User Testing: How to Start With Limited Resources
The internet is full of ultimate guides on user testing. There are tons of articles on how to improve user experience, how to find participants, how to properly analyse results and what testing tools are "the crème de la crème".
Those guides are elaborate, extensive and — well — long. There is, however, a problem. Inadvertently, they paint the same picture for user research as a whole. No wonder people start thinking that user testing itself is elaborate and lengthy.
You start feeling that you don't have enough money, time, personnel or skill to conduct proper user research with real users. This is not true.
Increased ROI, customer satisfaction, marketing potential and other perks of a good user experience can be available to you even when you have limited resources. Even when you have no prior experience conducting user research.
As you’ll soon see, user testing doesn't have to be a long and exhausting process. We'll share advice and shortcuts to save you time and money while ensuring your product has great user experience and receives all the benefits of user-centered design.
Let's dive in.
When to Do User Testing Test
The common answer would be: as soon as possible.
Yes, starting as early as possible helps to avoid rework by 50% and reduces the development costs up to 50%.
However, you should only start testing when you actually have something to test: a raw prototype of your product, maybe even a home page and a clear idea of what it does.
Maybe right now you're thinking, "But I want to test my ideas with my peers!" That’s fine, but bear in mind that testing ideas is user testing, but not usability testing — especially if your ideas are very abstract. Your peers haven't used your product yet and they don't know how it works. So while you can get some insights about your future product, those will have to do with its market value and interest, not usability and ease of use. You might ask, "But what if I talked to real people who like my ideas?" Well, that’s great! But make sure to avoid the pitfall of false confidence that everything associated with your product will be great — don’t take the early positive feedback as a sign that you can stop future testing altogether.
Back to "when to test". The right answer would actually be: Test your products as soon as you have something to test. This could be a storyboard, a paper sketch, a prototype, a landing page — something real and concrete, related to your product. Let's talk about that in more detail.
What to Test With User Testing
Here's the first tip to save you a lot of trouble: You don't have to test a final product. To do so would mean losing out on the development cost savings of UX research we stated earlier.
The previous section was all about testing something real, something concrete related to your product or service — even if it’s a paper sketch.
A Paper Sketch
All you need is a piece of paper and a pencil to draw your product's UI. It's raw, primitive and lacks visual design. But at the same time, it's cheap, simple and does the job of validating some early concepts about a particular product. It’s no wonder many interaction designers use paper prototyping as part of their design process.
For example, you show target users a paper sketch of your messaging app's user interface and ask them how they would send a message. Right after they ask, "But where do I type my message?", you instantly know that you have something to work on.
Test participants, of course, won't be able to click anything on a paper sketch. It's not exactly a human-computer interaction. But they can point things out. And this is how testing works in a nutshell: You give your target audience a series of tasks and then observe how they complete them.
The drawbacks are:
- You can only test paper prototypes in-person, so either your test subjects will be from the local area or you'll have to travel a lot.
- Testing on paper requires a lot of imagination from test participants, so insights may vary greatly from user to user.
A viable and still a low-cost alternative to a paper sketch is a digital prototype.
A Digital Prototype
Ok, we know what you're thinking: "Wait, don't I need UX designers and developers to create a digital prototype of my product? That doesn't sound cheap at all." It might have been that way several years ago, but these days nothing is further from the truth.
You don't necessarily need to hire designers or UX professionals to create a digital prototype, especially in the very beginning. In fact, you can do everything yourself. Use tools like:
- Adobe XD
- InVision Studio
These allow you to create digital prototypes and wireframes within hours, if not minutes. All of them offer easy-to-follow tutorials on interaction design and free UI-kits so you can quickly assemble a raw prototype of your product with pre-built elements. This is all without having to develop or draw anything.
You can even add interactions to your app design and create prototypes for both desktop and mobile platforms. You can do that yourself or hire someone from Fiverr to do it for you.
The drawbacks are:
- You'll have to spend a little time learning how to use user interface design tools.
- Creating a digital prototype will be more complex than a paper one.
That said, a digital prototype will save you lots of time in the future. Also, eventually, you'll have to convert a paper prototype into a digital one anyway.
For now, your main goal is to create a user interface for your product so that you can start testing a user's experience with actual users. It doesn't matter if it’s a paper prototype you hand-draw yourself or a digital mockup. What's important is who you're going to test it with.
Who to Test With
This is the question that many beginner practitioners of user experience testing are afraid of the most: Who should I test my prototype with? The answer is annoyingly simple: real people. More specifically, target users. If your future app is for dog grooming, find people who have dogs. If you have a car-parking service, test with people who have cars.
The second question is: How many people should I test with? Let's not go with the safe "It depends" answer. The number of users you should test with is five. Yes, the best results come from testing with no more than five users. Yes, there are debates around this number. And yes, there are exceptions to it as well.
But for now, let's say at least five. Testing your product with five users will surface 85% of user interaction problems with your current prototype.
So how do you find five target users for your user testing session? There are two options.
Find Participants Yourself
This option is cheap but takes a long time. It all comes down to how valuable your time is. You'll have to ask your team members, friends or even strangers on Craigslist (safety not guaranteed) to find test participants. You can also try looking for participants on social media.
Remember, you need to find five different users, so you can't test the prototype five times with the same person, no matter how much you like them.
Finding participants yourself has both advantages and disadvantages.
- It's cheap: You don't pay anyone to find participants for you.
- You have direct control: You have more control over who you want to test your product with and how.
- It's time-consuming: Finding participants whose profile is close to that of your end-user can take a lot of your time.
- It can lead to an unrepresentative sample: When finding participants, we're prone to take the easy route and interview our friends and family. While it's better than nothing, you must consider potential bias that comes from collecting data within your inner circle.
Utilise User Testing Platforms
Testing platforms will find test subjects for you. Although it's not always cheap, the pricing depends on the number of features the platform offers and the quality of screening for participants. You will have a lot of flexibility picking who you need to test your product with.
Most of the time, user testing platforms offer unmoderated usability testing, which means you have to specify in advance what tasks you want test users to perform.
Utilising user testing platforms comes with advantages and drawbacks.
- It's quick: You can save time because you don't have to recruit participants for usability testing yourself.
- It's easy: The testing platform will do all the searching for you.
- It's precise: Some platforms allow advanced screening of your testing candidates, and with a set of filters, you can set exactly what types of users you want to test your product with.
- Prices vary: You could see a huge range from platform to platform, but prices usually start at $50 per participant for tests with video recording and advanced candidate screening.
- It's unmoderated: You have to thoughtfully prepare a number of tests and tasks with your product beforehand, because you won’t be able to talk to participants in real-time and adjust testing script during sessions.
It's up to you whether you find test users yourself or utilise testing software. After you've finally found five participants, the key question is: How do I do the user testing itself?
How to Conduct User Testing
If you have limited resources, the most cost-efficient testing methodology is remote user testing.
With in-person testing, you have to search for suitable participants locally, spend money on traveling and rent or search for a place to conduct UX research.
With remote testing, all you need is Skype.
There are two testing methods available to you. With moderated user research, you conduct interviews personally. With unmoderated research, you create a set of tasks then analyse how successful real people were in completing them without your supervision.
Both methods have their advantages and disadvantages. And whether you choose moderated or unmoderated remote usability testing, you'll need to follow this blueprint.
Create a Test Script
There are two main reasons why you should have a script during user testing. Both are equally convincing.
First, to reveal usability problems, you need five users trying to do the same thing. If each of these users are doing their own, different tasks, there’s no way to understand whether the problems they encounter are common for everyone and not just that specific test participant.
Second, when you run user testing, most of the time things go in an unexpected direction. After all, you're looking for usability issues during the user journey, and you'll likely find them. If that happens, a test script helps you stay on track. Otherwise, you could get lost in a swamp of random behaviour data.
So you need a script — especially if you're just starting out with user testing. Your testing script should include realistic task scenarios that real users would perform with your app. It shouldn't give away how your interface should be used.
A poorly designed task might say, "Send a message using this upper menu", while a well-designed task would say, "Invite your friends to dinner".
Run It With Someone
It's good practice to test run your script with at least one person before you start testing your product with it. There are two reasons to do this:
- You'll get more comfortable with your own script and will be more confident during the user research. Conducting research with real users can be something new for you, and confident flow is important for a successful testing session.
- You'll find mistakes in your script before the testing begins. Scripts are not perfect. When you do a test run with someone, you might notice that certain tasks make no sense, and others tasks you forgot to test at all.
You'll get more comfortable and find mistakes. And if you do usability studies with a testing platform, like Teston.io, you won't have to schedule all five users in advance. for example, you can do a test run with one user and then, after fine-tuning your script, involve five more test users.
Before you start testing, make sure you have recording software on. It is almost impossible to capture everything during the interviews because you have so much else to focus on. However, make sure to notify your test participants that you're recording the session. Usually, people allow it without hesitation, especially if you assure them the video recording won't go public.
This isn't an issue with unmoderated testing platforms. All the participants are already aware that their sessions will be presented to the client, and the participants have agreed to that.
Now that you have a prototype to test, participants and a plan, finally the testing starts.
Remember, you don't need to be a user experience designer to start evaluating user needs. For now, follow these best practices:
- It's important to listen to what people say during the testing process, but it's even more important to observe what they do when they complete tasks. Those are never the same thing. If you want to ask participants something, make a note and ask them after they complete their tasks.
- Check your internet connection, Skype setup (or other programs you're using) and recording software.
- Have your testers use a web camera. Observing their emotions will later help you to understand what they're doing during research. It's not as advanced as eye-tracking, but at least you can see what they do when they’re not talking.
- Turn on your web camera before the testing phase starts to build a rapport with your users, but then turn it off so they won't be distracted or seek your approval during the test.
Notice that you have to worry about those steps only during moderated user research that you conduct yourself. Unmoderated testing services take care of all that for you.
After the testing sessions are finished and recorded, it's time to analyse the results.
How to Analyse the Results of User Testing
If you followed the steps above, you may already start to notice certain patterns with your product. For example, you'll see what tasks people had the most trouble with. It's important to make notes and document your analysis. Ideally, you'll perform user testing at each stage of your product development.
After you improve your prototype and add or remove certain features, repeat UX testing again. After that, you will be able to compare the results with previous UX research and see if you managed to create a better experience.
Make sure you identify hidden killer features. Those are the features that you didn't pay special attention to, yet they cover particular user needs. Take note of them as well and make sure your next prototype doesn't kill them. Don't forget to take into account ergonomics, e.g. when people complain about the ease of use of your products.
Finally, talk about your results with someone else, preferably someone familiar with your product. Perhaps watch testing sessions together and take note of interesting or surprising behaviour. It's always helpful to have a second opinion in order to objectively analyse the results of the user testing session.
Start Testing, Start Learning
Now you see that in order to get the benefits of user testing, you don’t need to spend all your money, quit your day job or hire professional UX designers.
Create a simple prototype, prepare a testing script, find participants and write everything down. Even in its simplest form, user testing will help you identify major usability issues and areas of improvement for your product.
From now on, we hope user testing won't seem like an exhausting process, but a fun, valuable and integral part of your product development. Good luck!