Diagnose a 20% Drop in Daily Active Users

Why This Is Asked

Metric drop diagnosis is the most common real-world analytics scenario. This question tests whether you have a systematic investigation framework — or whether you randomly pull charts until something looks suspicious.

Key Concepts to Cover

• Clarify before investigating — is the drop real, and what is the exact time window?
• Segment to isolate — platform, geography, user cohort, acquisition channel
• Check external factors first — holidays, app store outages, tracking issues
• Form hypotheses before looking at data — structured investigation beats fishing
• Root cause vs correlation — a metric in segment X also dropped because it's downstream of the root cause

Investigation Framework

1. Validate the signal     → Is the data pipeline healthy? Tracking issue?
2. Define the scope        → When did it start? Is it recovering? All users or subset?
3. Segment the drop        → Platform / geo / cohort / feature — which segment drives it?
4. Check external factors  → Holiday? App store outage? Competitor announcement?
5. Check internal changes  → Did we ship something? Push notification change? Algorithm update?
6. Form hypotheses         → Rank by likelihood. Test most likely first.
7. Confirm root cause      → Find the segment where drop disappears when excluded
8. Recommend action        → Fix, monitor, or acknowledge (if external/seasonal)

How to Approach This

1. Clarify First

Before pulling any data:

• Is this DAU vs same day last week, or DAU vs rolling 7-day average?
• Is this all users globally, or a specific region/platform?
• When exactly did it start — gradual or sudden?
• Were there any recent product changes, infra changes, or marketing changes?

2. Check Data Quality

A 20% DAU drop is large. Before assuming user behavior changed, check:

• Is the events pipeline healthy? Check Kafka consumer lag, dbt job status
• Are event counts down everywhere or just in specific tables?
• Did the tracking code change? Mobile SDK version update?

3. Segment Systematically

Break the drop down by:

• Platform: iOS vs Android vs Web — isolated platform drop → app store update, tracking bug
• Geography: One country or global? → local holiday, regional outage
• New vs returning users: New user drop → acquisition channel issue; returning drop → engagement/retention issue
• Feature area: Logged a specific feature? → recent feature change

Common Follow-ups

1. "The drop is isolated to Android users. What now?" Check: did we ship an Android app version recently? Is there a crash rate spike? Is the Google Play Store having issues? Did our Android tracking SDK update change event names?

2. "You've ruled out a tracking issue. What's your next step?" Look at the specific events that changed — login events, session starts, core action events — to identify which user behavior changed and when, then correlate with product or infrastructure changes at that timestamp.