Ep 32: CEO and Founder of Pinecone Edo Liberty on Pioneering Vector Databases, Barriers to Productionalizing Models and Why What’s Happening with GPUs is Not Sustainable

Vector Landscape

Vector databases have become a crucial component in AI infrastructure, with a surge of startups and incumbents vying to store vectors. Edo Liberty explains that vectors are now a fundamental data type, essential for semantic representation and search capabilities 1. He emphasizes the unique nature of vector databases, which use numeric arrays as primary keys for data organization and retrieval. This specificity makes them distinct from traditional databases, which struggle to handle such data types efficiently.

What people don't understand about vector databases and why they're so unique is that that numeric array becomes in some sense the key in some conceptual way.

--- Edo Liberty

Edo also highlights the early stages of AI infrastructure, noting that while vector databases have seen significant innovation, there's still ample room for new applications and solutions 2.

   

Key Uses

Vector databases are pivotal in various applications, from Q&A and semantic search to anomaly detection and drug discovery. Edo Liberty notes that while text and image data are common, there's growing interest in multimodal applications involving audio and video 3. He stresses the importance of realistic expectations, as many companies still struggle with basic AI model training.

Can we already see amazing things with multimodal? For sure. Do I think multimodal is going to hit the kind of mainstream technology developer in the next yield to unlikely.

--- Edo Liberty

Edo also points out that Pinecone excels at scale, handling billions of vectors cost-effectively, which is crucial for large-scale applications like those developed by Notion and Gong 4.

   

Scaling Issues

Scaling Pinecone's technology presented significant challenges, especially during the free tier's peak usage. Edo Liberty recalls the team had to redesign their solution to handle the massive demand, leading to the development of their efficient serverless model 5. This transition, while painful, was necessary to maintain performance and cost-effectiveness.

We started like really spending millions of dollars a month on our free tier. It was complete insanity.

--- Edo Liberty

Edo advises founders to make such transitions early to minimize revenue impact and align with customer needs, even if it means short-term financial pain 6.

   

Tech Comparisons

When comparing vector databases to other technologies, Edo Liberty highlights the cost-efficiency and performance of smaller, open-source models over larger, expensive ones. He believes the market will gravitate towards solutions that balance cost, compute, and output quality 7. This shift is essential as running large models on GPUs is unsustainable both financially and environmentally.

You're not going to run a 100 billion parameter model for every API call on your platform. That's, you're going to just, it's just you're going to go bankrupt.

--- Edo Liberty

Edo also discusses the importance of having data ready for vector databases, emphasizing the need for efficient ETL processes and metadata management to optimize retrieval methods 8.