Anrew Chen is General Partner, Andreessen Horowitz @a16z and ran growth @uber earlier.
Here goes his tweetthread on user testing and why product teams need to go beyond the core users and think of edge cases as well.
1/ “There’s nothing like some user testing to remind you that you don’t know anything.” -@far33d, growth at @slack
2/ When you work on the same product every day for years, it’s easy to assume that everyone uses the product the same as you do. Of course that’s not true!
3/ IDEO has a wonderful phrase – “vuja de.” Deja vu is when you see something for the first time, but think you’ve already seen it. “Vuja de” is when you see something every day but can see it fresh, like it’s the first time
4/ And in fact, when you do the analysis, the most important user perspectives aren’t just your core users, but all the folks out on the edges who are casual, churned, or blocked somehow.
5/ Your core users are important, but it’s a small minority of the universe you should be focused on. If you’re a DAU, given a good DAU/MAU is usually 20-40%, that already puts you in a pretty hardcore user segment. You already visit more than the majority of your user base.
6/ But that’s not all! Usually the monthly actives are only 20-40% of everyone who’s ever signed up on your product. That means you might already be in the top 10% of all registered users
7/ And of course, imagine all the folks who’ve downloaded or visited your product and never signed up at all. Signup rates are often <10-30%. So being a DAU makes you in the top 1% of all prospects/visistors
8/ In other words, most of your users are folks who never stick to your product at all. And when you recruit those folks and put them in front of what you’re working on, it’s super painful. Thus @far33d’s tweet rings so true
9/ Growth teams often spend their time on the vast/craggly “edges” of a product where most users hit and bounce off, but never stick.
10/ These edges are often ignored – they are the notification landing pages, signup flows, email/push notifs, lost password flows, etc. – because moving these metrics even a tiny bit can make a huge difference
11/ Needless to say, these edges are often ignored because they aren’t “sexy” to work on – it’s not something that easily goes into your Dribbble portfolio.
12/ To attack these areas, user research can play a huge role, because these sidedoors and edges of the product are often areas that people just see once, as new users, or as long dormant users. You need a different perspective.
13/ The “unboxing” experience is critical for any product. For years, we cut through crappy anti-theft plastic packaging, before the golden age of consumer electronics packaging arrived. We’ve replaced thick manuals with instructions to just install an app. It’s great.
14/ The same is true for a purely digital product. When it goes right, it should be magical. But we also need to make sure people aren’t stuck on the little things – picking secure passwords, being confused about pricing, getting locked out of their accounts.
15/ We use our products every day but we call the experiences that the largest numbers of our users go through as “edge cases” – when the majority case is that people don’t sign up, that they try it once and bounce, that they first see the app through a random side door
16/ Repeat after me: Vuja de. It’s just fun to say 🙂
— Andrew Chen (@andrewchen) June 1, 2018