Inverse Programming for Deeper AI with Zenna Tavares, w/ Zenna Tavares - #114

Inverse Graphics

Zenna Tavares, a PhD student at MIT, explores the concept of inverse graphics, where 2D images are converted back into 3D structures. This process is akin to how humans perceive the world, inferring three-dimensional geometry from two-dimensional visual inputs 1. Zenna explains that this idea, though not new, has not been fully realized in computer vision, and his work aims to run rendering algorithms backwards to achieve this transformation 2.

We have a working prototype for a particular kind of graphics. So we can take in 2D images from a fairly primitive voxel renderer. So the input are images and the output are 3D voxel shapes.

--- Zenna Tavares

This innovative approach holds promise for scaling up to more realistic scenes, though it requires further development.

   

Parametric Inverses

Zenna introduces the concept of parametric inverses to tackle the challenges of non-invertible functions. By using a parameter alongside the output, it's possible to determine one of the multiple inputs that could produce the same result 3. This approach is exemplified through the absolute value function, where a parametric inverse can distinguish between positive and negative inputs 4.

We construct this new thing, we call it a parametric inverse. So we'll take in some value y, but also a parameter theta, and give you back one of the x's which map to y.

--- Zenna Tavares

This methodology is particularly useful in optimization problems, where it helps navigate the vast output space to find desired solutions.

   

Inverting Neural Networks

The application of parametric inverses extends to neural networks, where Zenna explores the potential of inverting these complex models. By breaking down neural networks into primitive operations, it's theoretically possible to reverse-engineer them, though practical challenges remain 5. Zenna envisions using this technique to improve models of the world, such as in physics simulations, by combining learned algorithms with traditional ones 3.

For me, I think the most interesting application is, again, inverting models of the world.

--- Zenna Tavares

This innovative approach could enhance the realism and accuracy of simulations, offering new insights into AI development.