PM Quest · Practice cases · Learning resources

Netflix Sign-Ups Surge, But Watch Time Drops - What's Going On?

Root Cause Analysis · advanced

Netflix recently conducted an A/B test on a new onboarding process. The test group saw a 30% increase in sign-ups, but paradoxically, average daily watch time per user dropped by 10%. You need to diagnose the root cause. Do not propose solutions yet; focus on identifying potential reasons for this anomaly.

Write your approach on PM Quest

Community approaches

LearnProduct’s approach

  1. As a first thing we need to analyse both the onboarding process metrics which includes the following metrics
  • Sign Up rate
  • Activation Rate(Total number of users completing the sign up and watching first video)
  • Total time spent on the platform
  • Total time spent on a video
  • drop off rate at which point of the platform
  • Ratings of the videos uploaded in netflix before and after new onboarding process
  • Age wise segment of users who sign ups and drops off
  1. Check the activation rate whether the issue is because of the high user sign up who has less intent to watch
  2. Analyse the metrics to find where the friction is happening.
  3. If the users are young and the activation rate is less it might be the issue with the show or movie which has been uploaded in the platform
  4. If the user drops off while watching a show or movie nevertheless of age the issue is in the show or movie being watched
  5. Check if there is any bug in the new onboarding process deployed causing the drop off rate