Deep Reinforcement Learning Primer and Research Frontiers with Kamyar Azizzadenesheli - TWiML...

Epsilon Greedy

The Epsilon Greedy method is a foundational exploration strategy in reinforcement learning, balancing exploration and exploitation. Kamyar Azizzadenesheli explains that this method involves choosing the best-known action with high probability while occasionally selecting random actions to explore new possibilities 1. However, this approach can lead to suboptimal choices, as it doesn't always account for the quality of actions, like choosing vodka over coffee in a coffee shop scenario 1. Despite its simplicity, Epsilon Greedy is widely used in deep reinforcement learning, though it has limitations in extending theoretical algorithms to practical applications 2.

   

Thompson Sampling

Thompson Sampling offers a more sophisticated alternative to Epsilon Greedy by incorporating uncertainty into decision-making. Sam Charrington highlights that this method can significantly reduce the number of trials needed for learning in reinforcement learning environments 3. Kamyar notes that while most deep reinforcement learning literature relies on Epsilon Greedy, adopting better exploration strategies like Thompson Sampling can lead to substantial improvements without altering the architecture 4. This approach allows agents to maximize information gain and minimize uncertainty, enhancing their understanding of the environment while optimizing rewards.