Jordan Edwards: ML Engineering and DevOps on AzureML

Governance

Enterprise governance in machine learning involves creating cohesive strategies for managing models across large organizations. Jordan Edwards emphasizes the importance of a unified data catalog that integrates both data and models, ensuring all modeling work is tracked and managed effectively 1. He highlights the need for collaboration between Azure AI and Azure Data organizations to develop tools like Apache Atlas for enterprise-wide data governance. This approach aims to streamline operations and maintain compliance across various projects.

Most companies have a multi-cloud strategy, and it's completely accepted that you'll need to do ML things on Azure ML in a certain way.

--- Jordan Edwards

Jordan also discusses the challenges of deep learning, noting its overuse in situations where simpler models might suffice, and the necessity for specialized algorithms to optimize performance 2.

   

Testing & Explainability

Testing and explainability are crucial in the ML pipeline to ensure transparency and mitigate risks. Jordan explains that while interpretability methods are useful during the interactive phase, expert systems are necessary for the non-interactive phase to test models effectively 3. He advocates for a model risk management process that considers the unique circumstances of each model, especially in regulated industries where explainability is crucial.

We don't require humans to explain how they know things. We can't verbalize how we know things.

--- Jordan Edwards

Jordan also points out the importance of semantic understanding of data for adversarial testing, highlighting the need for more labeled data and advanced simulation techniques to improve testing methodologies 4.