717: Overcoming Adversaries with A.I. for Cybersecurity — with Dr. Dan Shiebler

Resilience

Resilient machine learning is essential in cybersecurity due to the constant evolution of threats. Dan Shiebler explains that attackers continuously adapt, necessitating robust systems that can handle data distribution shifts and feature dropouts 1. He emphasizes that understanding attacker behavior through cost-benefit analysis helps in designing effective defenses 2.

Attackers are changing what they're doing every week and every day in order explicitly to fight against the system that you're building.

--- Dan Shiebler

This adaptability ensures that cybersecurity measures remain effective even as new threats emerge.

   

Balancing Errors

Balancing false positives and false negatives is a critical challenge in cybersecurity. Dan Shiebler notes that while false negatives can lead to severe attacks, a high rate of false positives can render security measures ineffective as users may ignore alerts 3. He highlights the importance of minimizing both to maintain trust and functionality.

The worst thing that can happen is that you miss a really serious attack and it causes a lot of damage to customers.

--- Dan Shiebler

This balance is crucial for effective cybersecurity solutions.

   

Model Updates

Regularly updating machine learning models is vital to handle changes in data distribution and emerging threats. Dan Shiebler describes their auto retraining framework, which ensures models are retrained on different cadences to maintain effectiveness 4. This process involves data collection, feature extraction, training, and rigorous evaluation to prevent issues like false positives 5.

We maintain a large number of different machine learning models, which we retrain on different cadences.

--- Dan Shiebler

This systematic approach helps in adapting to new signals and maintaining robust cybersecurity defenses.