#102 - Prof. MICHAEL LEVIN, Prof. IRINA RISH - Emergence, Intelligence, Transhumanism

Emergence

Emergent behaviors in machine learning models are akin to phase transitions in physical systems. Irina Rish explains that as the scale of data or model size increases, a critical point is reached where the model suddenly acquires new capabilities, a phenomenon known as "groking" 1. This behavior is not unique to machine learning but is observed in various AI problems, where a sharp change in performance occurs at certain critical values, similar to state changes in matter 1.

It's about, is there a one functional form to capture them all? But what I'm trying to say, those behaviors keep happening.

--- Irina Rish

Michael Levin adds that while these transitions are significant, the continuity of biological processes suggests a different perspective, emphasizing the gradual nature of biological evolution and development 2.

   

Scaling Laws

Scaling laws in machine learning reveal how model performance can dramatically shift with changes in size or data. Michael Levin discusses the concept of goals in systems, suggesting that understanding a system's goals can enhance our ability to predict and control it 3. He argues that while humans often seek clear transitions, biological and machine learning systems may not always provide such distinct boundaries 4.

I don't believe that there is an objective goal that is intelligible in the sense that we get to discover it once and for all and then we're done.

--- Michael Levin

Irina Rish notes that the sharpness of transitions in machine learning models can be compared to phase transitions in physics, where the size of the system influences the abruptness of change 4.