ML Models for Safety-Critical Systems with Lucas García - 705

Verification

Verification challenges in AI models for safety-critical systems are multifaceted, involving data independence and certification complexities. Lucas García highlights the importance of independent datasets and frameworks to achieve higher certification levels, such as D level certification, which can be enhanced through architectural mitigation 1. He explains that certification bodies like EASA are developing materials to guide AI verification processes, including the ML leap project, which serves as a comprehensive guide for AI verification techniques 2.

It's challenging. It's challenging area. There are some benchmarks out there for AI verification that are publicly available.

--- Lucas García

These resources are crucial for industries like aviation, where AI integration into safety-critical systems is increasingly prevalent.

   

Neuron Coverage

Neuron coverage is a critical concept in AI verification, akin to code coverage in traditional software testing. Lucas García describes neuron coverage as ensuring that AI models activate enough neurons during testing to guarantee robustness and reliability 3. This methodology is particularly useful in identifying vulnerabilities, such as adversarial attacks, which can exploit weaknesses in AI systems 4.

There is a technique called neuron coverage that has the basic idea to make sure that as you train your AI model, you want to make sure that you are covering enough neurons in the neural network.

--- Lucas García

By applying formal verification methods, developers can mathematically prove the robustness of AI models, ensuring they perform reliably in safety-critical applications.

   

Mitigation Techniques

Architectural mitigation techniques play a pivotal role in achieving higher AI certification levels in safety-critical systems. Lucas García explains that these techniques involve creating redundant systems with dissimilar components to enhance reliability and certification levels 5. By using different datasets, labeling tools, and AI architectures, developers can ensure that independent systems agree on outputs, indicating the intended result.

Architectural mitigation deals with the fact that if you have two dal D components sitting side by each other, you can actually target a DaL C certification.

--- Lucas García

This approach allows for the certification of more critical components, providing a blueprint for industries to follow in their certification processes.