Jonas Hübotter (ETH) - Test Time Inference

Active Inference

Active inference is a transformative concept in machine learning, emphasizing situational computation and dynamic learning environments. Jonas Hübotter compares this to Google Earth's variable resolution, where computational resources are allocated based on task complexity, allowing for more precise predictions 1. This approach contrasts with traditional models that rely on static computation, highlighting the potential for systems to adapt and learn continuously. Tim Scarfe notes that active inference could lead to distributed systems that remember and adapt based on past predictions, enhancing their learning capabilities 2.

   

Local Learning

The evolution of local learning methods showcases a shift from basic retrieval techniques to sophisticated test-time learning and transductive inference. Jonas Hübotter explains that early methods like nearest neighbor retrieval focused on finding similar data points, but modern approaches synthesize diverse information for more accurate predictions 3. This progression includes the development of kernel regression and locally weighted linear regression, which weigh data based on proximity to the prediction point 4. These advancements illustrate the balance between inductive and transductive learning, offering tailored solutions for specific tasks.