Our best tutorial was actually broken (we just couldn't see it)
12 user complaints revealed what our analytics couldn't
A tutorial had a 5-minute average session time. The analytics looked great, the best retention of any page. During a review call, the product lead pulled up the dashboard: “See? Users love this one.”
We suggested adding a feedback widget. A small button: “Was this helpful?”
One week later, 12 users had left feedback. Not one was positive.
“Step 3 is missing.” “This example doesn’t work.” “I’ve read this four times and I’m still confused.” “Where’s the actual code?”
Those 5 minutes weren’t engagement. They were users scrolling up and down, re-reading, trying to figure out what was missing.
Good metrics, bad experience
The analytics showed what users did. The feedback showed why.
The team fixed everything in 48 hours: added the missing step, replaced the broken example, clarified the confusing parts, and added code where users expected it.
Session time dropped to 3 minutes. But now users were finishing successfully instead of giving up confused.
You need both
Quantitative data shows behavior. Qualitative data explains it.
Your analytics will tell you users spent 5 minutes on a page. Your users will tell you they spent those 5 minutes confused.
One number can mean two completely different things.