Martin Wattenberg: ML Visualization and Interpretability

Trust & Tools

Understanding machine learning systems involves calibrating trust rather than simply increasing it. Martin Wattenberg emphasizes that designers should aim to help users understand the appropriate level of trust to have in these systems, which are often probabilistic and not always accurate 1. He highlights the challenge of explaining these systems to users who might misinterpret their capabilities due to confident outputs 1. Wattenberg also discusses the complexity of terms like "understanding" in AI, suggesting that focusing on practical applications and usefulness can better approximate what people mean by understanding 2.

It's not like your goal as a designer is to increase trust in your system. It is to calibrate trust, make sure that people understand the right level of trust to have.

--- Martin Wattenberg

Tools developed at Google PAIR, such as Facets and SmoothGrad, aim to enhance model understanding by focusing on human-centered approaches and addressing potential misalignments 2.

   

Othello GPT

The Othello GPT project explores how language models can develop internal representations akin to world models. Martin Wattenberg describes training a model on Go game transcripts, which surprisingly learned to make legal moves without explicit knowledge of the game rules 3. This suggests that the model forms an implicit understanding of the game board, raising questions about the interpretability of language models' internal processes 3.

It does appear to be that there's a model in there.

--- Martin Wattenberg

Further experiments revealed causal links in the model's decision-making, even with positions not seen during training, highlighting the complexity of neural networks' memorization and generalization processes 4. Wattenberg advocates for a skeptical yet open-minded approach to understanding these phenomena, cautioning against letting preconceived notions influence interpretations 4.