Recently, I was lucky enough to spend a couple of days with some users who were testing a career guidance tool we’ve been working on for several years with a long-term client of ours, Career Services.
The tool is called Career Quest. It’s delivered on CD-ROM and it’s highly likely that we’ll soon begin redesigning the product from scratch and making it available online.
It’s been available in NZ secondary schools (and also Career Guidance offices) for a number of years and we’ve been steadily refining and improving the product over this time.
The purpose of our User Testing sessions (held with 22 users across two days in five different locations) was simple enough: find out what users think about the product and how we could improve it.
As this was one of several rounds of testing that has been conducted over the years, there were no major content or usability surprises that emerged from what is (now) a reasonably mature and stable product – in fact, we (as designers and clients) were probably a lot harder on the product’s overall design and usability than the users were. We knew it better, and we already had years of research telling us where the major sticking points were occurring.
However, the real benefit from the two days of user testing was being out of the office for a couple days, sitting alongside users (predominantly Kiwi teenagers from a range of backgrounds) and hearing their candid and honest thoughts about what Career Quest meant to them (none of them had used the product previously). No emails. No databases. No wireframes. No strategies. No meetings. Just us (one tester) and them (sitting with one user).
Two things struck me about this experience:
1. How optimistic and positive the teenagers were (both about the software, but more importantly, about their futures).
2. The importance of user testing.
The first point was more a personal reflection made by me in response to the way ‘the yoof of today’ are portrayed in the mainstream media (can anyone else already hear drunken riots and sirens wailing?).
Try Googling: ‘NZ teens’ or similar and you’ll find a slew of headlines with numerous fatalistic and grim articles about narcissism, personal demons, drugs, depression, pregnancy, deafness (iPods), bullying and so on.
Now, there is nothing particularly scientific about some conversations with some teenagers and a random Google search. But there were five testers (two from Click Suite and three from Career Services), and the users themselves were not ‘cherry picked’ by schools to represent ‘just the good kids’. In fact, some were ‘grabbed from the corridors’ at the last minute to make up numbers.
So my first point is nothing earth shattering: it’s simply a timely reminder not believe everything you read (or watch), and especially not to be spooked by the New Zealand media who do seem to indulge in a certain degree of stoic negativity as they stagger around like groaning zombies on the gloomier side of the ‘reporting’ spectrum. Can you remember how many good news stories you’ve seen recently headlining either of the major dailies? Russell Brown’s Hard News is often a refreshing exception.
The second point is more directly relevant to the business of making interactive media. And that is quite simply: User Testing is utter GOLD. No matter how well (or badly) it goes there is simply nothing better than watching users ‘use’ and listening to their thoughts and feelings about why an interface is or isn’t working. For me, personally, it’s the most enjoyable (and sometimes hardest) part of what I do. If you’re a developer or designer or BA or content person and you haven’t done this recently then you simply MUST beg, steal, borrow or bulldoze your way in to a User Testing session near you.
User Testing provides the genuine human insights that you’ll never get from any behaviour tracking and reporting tools, focus groups, online polls, phrenology, Reiki, crop circles or any of the other many diagnostic tools that are used to assess the success or otherwise of what you make.
Phrenology. Early anthropologists couldn't wait to jump from their Syphilis-ridden vessels and mesaure the natives' heads. We User Testers are the same.
It’s not to say that quantitative analysis isn’t valid – it’s vital; it’s just really important to remember that it’s only part of the story.
The other great thing about User Testing is all the little comments that are clues as to the socio-cultural and geo-political compasses (or whatever you want to call them) that direct and inform your users’ worlds. How they react with their friends. Their language. Their body language. Their sense of humour. And the entire cast of heroes, villains and oddballs they mumble under their breath when they see a potential career option pop-up in front of them.
Interactive media is generally an interruption in someone’s life (regardless of whether it’s push content or pull content). Consuming or using it is generally not their reason for being. You have to know all this kind of stuff to understand how this interruption in someone’s life is actually making their life better, more entertaining, more informed, more efficient, less stressful and so on. Without it, you’re blind.
So there you have it. A couple of very long-winded and spectacularly unoriginal maxims from the coalface of interactive media: teenagers are actually fine (or at least, trying to come to terms with the same old issues they’ve always had); and, if you’re not doing regular user testing (face-to-face, one-on-one) then you should pack up your toys and go home.
Your users are waiting. Turn off your computer and go interrupt them now.
