Real-time conversational insights from phone call data

Unsupervised Models

Creating unsupervised models for language processing involves navigating complex challenges, such as handling diverse dialects and extracting meaningful insights from phone call data. Mike McCourt explains that their approach involves developing a lexicon that defines an average phone call, which is then specialized into various topics like appointments or purchases 1. This hierarchical splitting allows for the identification of consistent themes across diverse conversations. Daniel Whitenack highlights the difficulty of interpreting computationally distinct clusters, to which Mike responds by emphasizing the importance of human interpretability in their models 2.

We want the topics to be something that kind of floats above that data set and represents the themes that are consistent throughout it.

--- Mike McCourt

Their unsupervised model aims to automatically analyze calls and present recurring themes without human labels, making the process more efficient for users.

   

Language Patterns

Unsupervised methods are pivotal in uncovering fundamental language patterns, such as Zipf's Law, within conversational data. Mike McCourt describes a hierarchical model that starts with messy, idiosyncratic data and moves through layers of abstraction to identify common patterns 3. This process involves deriving probabilities for words, which follow a power law relationship, to create a lexicon that defines the language used in calls. Daniel Whitenack inquires about the application of topic modeling to large datasets, and Mike explains how their model ensures both computational and human interpretability 4.

The word choice follows Zipf's law, this funny distribution, and there's all this uniqueness and dialect and all that.

--- Mike McCourt

By aligning mathematical models with linguistic features, they achieve interpretable results that can inform predictive models.