When users fall through the cracks

Google's one-question drop-off survey

Support for this issue comes from Attio - the AI-native CRM that builds itself. Connect your email and Attio maps your network, relationships, and deals so you can focus on the real work, not admin. Get 14d free here to see how it works ✨

Have you ever looked at a funnel chart, and wondered:

It’s one of the most common questions I find myself asking in product development:

You can dig more into data, but it rarely gives you the answer.

That’s where qualitative analytics stops. You can know the ‘what’ is happening, but not the ‘why’.

So, how do you work out what’s going on, and why users are falling through cracks in your funnel?

Enter: one-question surveys.

I was logging in with Google’s two-step verification the other day when my verification failed 🫠

I got to an error screen (shown when the verification between devices fails during sign-in).

At that very same moment, I saw a white box pop up at the bottom right of my screen:

It was a one-question survey asking me:

Followed by eight options:

• The instructions weren’t clear
• I don’t know which phone I should use
• I clicked on my phone or tablet’s name, but I didn’t receive a notification on that device
• I’m not familiar with QR codes
• The QR codes won’t scan
• I don’t want to enable bluetooth
• My phone is loading but nothing happens
• Any other issues (please explain)

The timing of the survey is perfect.

It appears right at the moment of failure when the frustration is still fresh.

In user research, the further you get from the event, the fuzzier the feedback becomes.

Here, the user’s memory, device setup, and emotion are all still active, which makes their answers far more reliable.

There’s a few other things that are executed really well here:

1. The design is simple: the white survey module stands out well. It’s big enough to be visible without getting in the way.
2. Low cognitive load: single-select options makes feedback easier. By giving likely causes, Google helps users express frustration without needing to articulate it precisely.
3. It’s hypothesis testing: each option of the survey maps to a product or engineering hypothesis: it’s clear the team have already brainstormed what might be going wrong here (bluetooth reliability, unclear language, device pairing).
4. The options are written in user-centric language: first person, layman language means it’s easy for users to self-select into an option. At the risk of sounding clunky, e.g. ‘the QR code won’t scan’, each option feels like a conversation with a struggling user.
5. Easy opt-out: There’s no pressure to respond. The small size and make it easy to avoid if you want. Within this flow, a compulsory question may hamper your chances to get people to sign in.

But even with all that done well, there’s a couple of things that are jarring.

1. The options are overwhelming: eight different failure options for one simple login feels a lot. As a user answering the survey, it feels like a long list to read.
2. Knowledge gap: there’s a risk here that people don’t know why the log in failed. When I read this list in the moment, I drew a blank. I didn’t have the technical know-how of exactly what went wrong.
3. The timing is a double-edged sword. Asking in the moment captures honest data, but it’s also the peak of frustration. The user may just want to get back in.

What I like about this example is how it forces us to think about research inside the product. Most teams collect feedback through emails or quarterly surveys, but some of the best insights come when users are mid-task - not when they’re reflecting after the fact.

Did this survey find out what it needed to? Maybe. It likely gathered honest, in-the-moment reactions, even if it stumped some users.

It shows how a tiiiiiny moment of friction can become a feedback loop if you design for it.
​

Thank you SO much for reading (all the way to the bottom, wow look at you go).

See you next week,

Rosie 🕺

​​