SDS 539: Interpretable Machine Learning — with Serg Masís

Feature Importance

Understanding feature importance is crucial in interpretable machine learning, as it helps identify which inputs significantly impact model outcomes. Jon Krohn explains that feature importance techniques allow us to reverse engineer relationships in complex models, similar to how beta weights function in linear regression 1. Serg Masís adds that these techniques can alert us when inputs deviate from the training data, indicating potential issues with model predictions 1. He also emphasizes the ethical and financial implications of misinterpreting models, highlighting the need for clear understanding of model scope and limitations 2.

Our features have drifted from what our model was designed for.

--- Jon Krohn

The distinction between interpretable machine learning and explainable AI is often debated, but Serg believes they are interchangeable, cautioning against overconfidence in explanations 2.

   

Neural Visualization

Visualizing neural networks, particularly convolutional neural networks (CNNs), offers a fascinating glimpse into their inner workings. Jon Krohn describes how CNNs process information through layers, starting with simple features like lines and progressing to complex textures and shapes 3. This visualization helps in understanding how specific neurons specialize in detecting particular objects, such as dog faces or cars 3.

It's clearly specialized to detect dog faces, and so you can kind of see the canonical dog face that it's learned to detect.

--- Jon Krohn

He recommends Jason Jacinski's Deepviz toolbox for an engaging exploration of CNNs, showcasing real-time neuron detection and interaction 3.