Aaron Colak — ML and NLP in Experience Management

Deployment

Deploying machine learning models from conception to production presents significant challenges, particularly due to structural and production constraints. Aaron Colak highlights the importance of having a close connection between ML engineers and product teams to ensure models are designed with production constraints in mind. He emphasizes the role of ML engineers in bridging the gap between model design and practical application, stating,

To me, that's the essential role machine learning engineers should play.

--- Aaron Colak

Qualtrics employs a "trifecta model" where ML engineers, product engineers, and ML scientists work collaboratively to streamline the deployment process 1.

   

Customization

Customizing machine learning models for specific industry needs is crucial for effective sentiment analysis across diverse sectors. Aaron Colak explains that out-of-the-box models often fall short, necessitating customization through customer input and the combination of language models with customer-specific lexicons 2. He describes Qualtrics' approach of using universal and customer-specific models, allowing clients to bring their own data and define variables for predictive modeling 3. This flexibility empowers clients to tailor models to their unique requirements, enhancing the accuracy and relevance of insights.

   

Monitoring

Monitoring the performance of ML models in production is a complex task that involves various tools and considerations. Aaron Colak notes the importance of productivity-boosting tools like experimentation tracking and reproducibility, while highlighting the challenges in CI/CD processes due to the involvement of multiple personas 4. He also discusses the evolution of ML platforms, emphasizing the role of hyperscalers in standardizing workflows and infrastructure 5. Colak stresses the need for fairness and bias monitoring, acknowledging the societal and legal implications of these systems.