Being Bayesian

Understanding Priors

Kyle Polich and Linh Tran explore the concept of priors in Bayesian statistics, emphasizing how prior beliefs are formed and updated. Kyle explains that priors are essentially initial beliefs based on past information, which are then adjusted as new data is gathered. This process is likened to understanding the preferences of Yoshi, their bird, where prior knowledge about her food choices is updated with each new observation 1. Linh adds that while Yoshi's reactions to food are informative, they are not always reliable indicators of her true preferences 2.

We take our prior beliefs, what we thought about whether or not she'd like the food, we update. In this case, given the absence of the squeal, she didn't make it. So we don't think squash is her new favorite food, even though it could be.

--- Kyle Polich

This highlights the importance of continuously refining priors to better predict outcomes.

   

Likelihood's Role

The role of likelihood in Bayesian predictions is crucial, as it helps refine posterior beliefs based on new evidence. Kyle illustrates this by comparing it to GPS systems, where initial location estimates are adjusted as more satellite data is received 1. Similarly, Yoshi's food preferences are updated based on her reactions, though these signals are not always perfectly reliable. Linh notes that Yoshi's excitement over food can sometimes be misleading, as it doesn't always correlate with her actual preferences 2.

Our beliefs increase because it's consistent with the available information we have, but not all the way, because it's not a perfectly reliable measurement.

--- Kyle Polich

This demonstrates how likelihood functions adjust beliefs while accounting for the uncertainty of observations.

   

Convergence Properties

Bayesian beliefs are known for their ability to converge quickly to the true distribution with sufficient data. Kyle discusses how, over time, repeated observations can lead to more accurate predictions about Yoshi's food preferences 3. Linh shares that Yoshi's behavior, such as throwing food she dislikes, provides valuable data that can significantly alter their beliefs. This iterative process ensures that their understanding of Yoshi's preferences becomes increasingly precise.

With more data, over time, bayesian beliefs will converge to the true distribution. This is one property of them.

--- Kyle Polich

The rapid convergence of Bayesian beliefs underscores their effectiveness in adapting to new information.