Machine Learning Street Talk (MLST) avatar

Dexa/Machine Learning Street Talk (MLST)

Learn more
Published Sep 12, 2024

Cohere's SVP Technology - Saurabh Baji

Cohere's SVP of Technology, Saurabh Baji, explores the company's pioneering strategies in deploying large language models for enterprises, stressing data security and enhanced performance through retrieval-augmented generation. He shares insights into effective model customization and optimization, illustrating practical implementations and cost efficiencies across diverse industries.
Episode Highlights
Machine Learning Street Talk (MLST) logo

Topics covered

Popular Clips

Questions from this episode

Episode Highlights

  • Resource Optimization

    Cohere's approach to resource optimization in AI model deployment is both innovative and pragmatic. highlights the importance of utilizing cloud infrastructure efficiently, drawing parallels with past experiences in serverless computing 1. By leveraging techniques like running multiple fine-tunes on a single GPU, Cohere maximizes hardware utilization while minimizing costs 2. This strategy ensures that customers can achieve high performance without the need for massive computational resources.

    We have literally customers who are running 50 fine tunes on one single GPU.

    ---

    Additionally, the use of serverless AI systems allows for flexible and efficient deployment, adapting to the varying needs of enterprises 3.

       

    Cost-Effective AI

    Cohere's focus on cost-effective AI solutions makes advanced technology accessible to a broader range of enterprises. emphasizes the balance between performance and affordability, ensuring that AI models are not only powerful but also economically viable 4. By offering full fine-tuning capabilities, Cohere allows businesses to customize models to their specific needs without incurring prohibitive costs 2.

    My job as SVP of engineering is to really make sure that we are applying this amazing technology in the way that customers find useful.

    ---

    This approach fosters independence among customers, enabling them to leverage AI technology effectively while maintaining control over their resources 5.

       

    Innovative Techniques

    Cohere employs innovative techniques to enhance the efficiency and performance of AI models. The ability to run multiple fine-tunes on a single GPU is a testament to their commitment to maximizing resource use 6. discusses the pragmatic implementation of retrieval-augmented generation (RAG), which integrates enterprise data with AI models to improve output quality 7.

    The retrieval part almost doesn't get enough credit with retrieval augmented generation.

    ---

    Additionally, the use of synthetic data solutions allows for effective model training, even with limited data, by focusing on imparting new abilities to the model rather than overwhelming it with unnecessary information 8.

Related Episodes