From NLP to LLMs: The Quest for a Reliable Chatbot

Complexity

Managing conversational complexity in AI involves a blend of deterministic logic and large language models (LLMs) to ensure seamless interactions. Alan Nichol explains that while LLMs handle the unpredictable elements of conversation, deterministic engines manage task logic efficiently. This combination allows for a structured approach to customer interactions, avoiding the pitfalls of relying solely on LLMs, which can lead to unpredictable outcomes 1.

The nice thing about the approach that we have is you say, look, all the messy, ugly, confusing stuff around, having a conversation, having a fluent conversation and dealing with digressions and people correcting themselves and having follow up questions and changing their mind about things and switching from one thing to another, all of that complexity is handled by the LLM and the task logic is just done simply deterministically.

--- Alan Nichol

Martin Casado highlights the challenges of ensuring AI doesn't produce nonsensical responses, emphasizing the importance of interfacing with formal systems to maintain reliability 2.

   

Prompting

Prompting techniques in AI are crucial for steering conversations effectively. Alan Nichol describes the 'prompt and pray' approach, where a text blob is used to guide AI responses, though this method lacks control over the output 3. Instead, he suggests using templates that dynamically pull in relevant context, a method known as retrieval-augmented generation (RAG), to improve precision.

I call this approach prompt and pray. Because you really don't have like control over the output.

--- Alan Nichol

Martin Casado adds that RAG involves querying a database for relevant information to include in prompts, enhancing the AI's ability to provide accurate responses 4.