YouTube Long-Form Watch Time Drop Post-Shorts Tab Launch

Ishant Juyal’s approach

1. Segment the userbase into cohorts - new users and old users and see if the drop is in both the cohorts or only in one cohort.
2. Further segment the cohorts into smaller cohorts based on their activity level, their region, the device the use Youtube on and find if this problem exists for a specific segment or for everyone.

If the problem is with a certain cohort, watch the session recordings to evaluate what could have caused the issue - it can be technical, regional, something because of a third party etc.

If the problem exists for everyone, we track the user journey as funnel and see where the dropoff is happening. If there is a certain step where the users are dropping now that they were not before, we study that step specifically.

If the average watch time has dropped only but not the number of videos being opened, then we need to see what change is there in user behaviour after they start watching the video. Is there a layout change making them leave? Is there a new Shorts button on the layout that they are clicking and hence leaving the long form videos to go to shorts?

Need to see the user flow from the watch video screen in order to find the root cause.

Pavlo’s approach

I would check the metrics of how much time users spend in the 'Shorts' tab and the total app session time. Then we should compare previous total session duration (before 'Shorts' was rolled out). If the new duration is >= old duration, then we can conclude, that users are just more interested in 'Shorts' which is a good signal. Otherwise 'Shorts' might distract users from long-form content.