Pavlo’s approach
We have already done the cohort break down and found that only users within specific age group is affected by the update. That's a great clue. Next we should seek the following clarifications:
- What were the reasons to update the algorithm - should it improve recommendation accuracy, fix regulation issues or something else?
- What were the metrics expectations before the release?
Based on the answers we can assume what to do next during our investigation. If the algorithm update was needed to fix some inappropriate content recommendation for the specific age group, then session duration drop should be expected.
If the update had to improve the recommendations by making them more accurate, then we should check why the specific group was affected. Maybe the algorithm triggered some age specific content checks which leads to the issue.
We can gather the details about the update by communicate to the engineering team members who was responsible for the update, then make a decision either it's expected issue or it should be fixed/rolled out.