Edu: Product Experiments - Revealing The Science Behind Successful Apps

Success Stories

Andres Glusman shares compelling stories of successful experiments at Meetup that led to significant positive outcomes. One notable experiment involved introducing a checkbox during the signup process to deter predatory business meetups. Contrary to initial fears, this added friction resulted in a 16% increase in group signups and improved the bottom line 1. Yaniv Bernstein also recounts a split test at Airtasker, where testing different fee percentages revealed that a 10% fee had no negative impact on metrics, significantly boosting revenue 2.

I cannot believe that adding more friction, which is the opposite of what everyone says you should do, actually netted out more groups.

--- Andres Glusman

   

Learning from Failures

Learning from failures is equally crucial, as Andres emphasizes. He discusses the importance of monitoring experiments over extended periods to avoid negative long-term effects, such as increased churn due to unmet promises 3. Additionally, Chris Saad and Yaniv highlight the potential for upstream and downstream effects, stressing the need to consider the broader impact of any change 4.

Every solution is really creating, like the seeds for tomorrow's problem that you get to solve.

--- Andres Glusman

   

Optimizing Growth

Optimizing growth through experimentation is a key theme. Andres explains that running experiments allows companies to identify what truly drives user behavior, leading to more effective growth strategies 5. He also notes that while speed to launch is often prioritized, speed to results or knowledge is more valuable, as it provides actionable insights faster 6. Chris adds that focusing on time to truth rather than just shipping features can lead to more meaningful success.

When you run things as experiments, the upside is that you get lift and it works. When it doesn't work, you get learning.

--- Andres Glusman