User testing agency: decide on evidence
Do you doubt how people understand your service, and do you need to make a call?
We're a UX/UI agency that has been doing UX research since 2013, with three doctors in cognitive engineering on the team. We put your product, or your new concept on mockups, in the hands of people recruited on precise criteria to match your target users. We moderate every session. You come away with the problems encountered, screen by screen, prioritised and illustrated with video and audio clips and verbatim quotes, so you can decide with your stakeholders based on what users actually did, and no longer only on what the project team, ours included, thinks.
When testing is the right call
Clients ask us for user testing in one of these situations.
- You have an existing product and you want to know how it performs in use: do people understand the journeys, where they are, where they can go, how to go back, and do they manage to make a decision.
- You know some things don't work very well, you have your own doubts, and you want to check that users run into the problems the same way.
- You lack knowledge of how certain categories of users, professionals or people with disabilities for instance, understand your service.
- Your service was designed over several years with the project team, without users being asked directly.
- You're preparing to launch a new concept, and you want to see, on mockups or on a prototype, whether people understand it.
Most often, a product that's already live
The most classic case is an existing product that we test to see how things go. We also test upstream, on mockups or on a prototype, when a new concept needs to be understood before it launches.
How we go about it
The protocol is built with you
Your project team is involved upstream: in defining the recruitment criteria and, almost every time, in the presentation of the test protocol. We show you the scenarios and the pages that will be explored, and we can go as far as the observation grid and the questions asked.
We recruit on behaviour
What interests us is what sets your users' behaviour apart, more than their socio-demographic profile. Digital confidence does it almost systematically, because you don't talk in the same way to someone discovering the conventions of digital and to someone who masters them. Four or five main behavioural characteristics make it possible to define the criteria we pass to the panel provider (the company that recruits the participants). Age isn't enough to tell confidence: according to Insee, 21% of 15 to 29-year-olds had weak digital skills in 2023.
With us, a test is always moderated
In an unmoderated test, the person is alone in front of their screen: you don't know what they're doing on the side, how they're carrying out the task, or even whether it's really them. Our sessions are therefore moderated by us from start to finish, which lets us observe, follow up and understand what's blocking.
Remote or in person, a matter of logistics and profiles
Remotely, recruitment is simpler and the profiles more varied, and generally it's the panel provider who checks the connection and the screen sharing. In person gives access to behaviour, to what happens a little before and a little after the task, with less varied profiles since people have to travel. It also lets us reach profiles we would have technical difficulties with remotely, for instance people with disabilities whose full use of their technologies we want to understand: there, screen sharing doesn't work. For the best quality test, we generally prefer in person, putting the resources into recruitment.
You can observe, without stepping in
From time to time, we offer your teams the chance to observe the sessions. Remotely, they connect as members of the project team, without being introduced as the client, so as not to bias the feedback. In person, very exceptionally, they sit behind a one-way mirror at the panel provider's premises. During the test they don't step in, we moderate it from A to Z. At the end of a session, we sometimes offer them, if they wish, the chance to introduce themselves and ask the participant the questions they care about.
We analyse on our side, we decide together
The analysis is carried out entirely by us, then presented to you. The presentation is where you can react, notably on the prioritisation, and nothing in it is final: the priorities and the recommendations adapt with you. Its purpose is to decide, not to redo the tests or the analysis on the spot.
Decide with the results of the user tests
The problems, screen by screen
We go through the problems encountered screen by screen, linked back to the users who experienced them, and we prioritise them, in the same spirit as our UX audits.
Tied to criteria where possible
When a problem can be linked to an ergonomic criterion, we do it, because it's more useful to your teams than the bare finding.
Video, audio and verbatim quotes
Much more than in an audit, we illustrate with short video or audio clips and verbatim quotes, which works very well with decision-makers.
Anonymised sessions
We anonymise the sessions so as not to tie the feedback to individuals, and so that nobody thinks a problem doesn't count, or on the contrary counts enormously, because of the profile of the person who ran into it.
The Banque des Territoires teacher portal
Teachers were creating an account on the Territoires Numériques Éducatifs portal without using it continuously, and the Banque des Territoires wanted to understand what was holding them back. We combined an ergonomic audit with remote user testing, and met 20 users across the whole study. The audit had flagged the lack of a breadcrumb, unnecessary steps and difficulties with dynamic elements. The tests confirmed these problems and brought to light the frustrations tied to them, such as the lack of clarity on the scheme and the uncertainty about the effort required. From this cross-check came 4 major blockers and two priority areas, the first hands-on use and retrieving resources, on which around ten page templates were redesigned.
Read the case studyUser testing for a public portal, an NGO, a hotel chain
WWF France: from informed click to committed action
Read the case study
Rediscovering B&B Hotels' online check-in through user tests
Read the case studyFrequently asked questions
Ten to twelve per typical profile. User testing is a qualitative method, which brings out trends rather than statistics, and we've long seen that they emerge clearly after eight or nine people. There's always one or two sessions that don't go as planned, and often one or two very atypical profiles: hence the margin. If your service has several target groups, individuals and professionals for instance, you need ten to twelve for each. Mixing profiles in a single group is possible, knowing that it dilutes the quality. Testing more people brings extra detail, but what most often interests a client is the big picture, and chasing the last details sends the budget up very sharply.
On four or five behavioural criteria, such as digital confidence, which we usually hand to a panel provider. They do the recruiting, and participants are compensated. The harder a profile is to recruit, the higher the compensation, and going through a panel provider lengthens the timeline. To give you an idea: on a recent campaign, with around twenty participants split between the general public and professionals and an online questionnaire on top, seven weeks went by between the kick-off meeting and the presentation of the results.
The budget depends first on the number of users we meet, then on how complex they are to recruit when we go through a panel provider, because participants have to be compensated and difficult profiles cost more to recruit. The panel provider also lengthens the timeline. Then comes the type of test: a test under controlled conditions, close to the lab, with very tightly framed recruitment, or guerrilla testing, where we go and meet people in the broad sense to get feedback quickly and cheaply, with quality that isn't the same. Each campaign is quoted individually, based on these three factors. As a guide, allow at least €10,000 excl. VAT, not including recruitment and participant compensation.
Most often on a product that's already live, to see how it performs in use or to check out doubts. But also upstream, on mockups or on a prototype, before a new concept launches.
Yes, and it's often the most practical option, because recruitment is simpler and the profiles more varied. A single campaign can combine both modes. For a public sector client, we ran most of the tests remotely and those with people with disabilities in person, because screen sharing doesn't let us fully understand how they use their technologies.
On computers and on smartphones. Tablets are much rarer. Most of the time, and notably remotely, we test on the participants' own devices, which lets us take in the diversity of usage contexts. In person, we test on our own machines, which gives us better control over the tests.
Before any collection, we give participants a data protection charter and gather their express consent, their parents' consent under 16, and an image and voice release under 18. We only collect what's necessary, with no needless demographic data. Each participant gets a unique identifier, the correspondence table is kept in dedicated storage, and audio and video recordings sit on an encrypted server, with named, logged access. When a panel provider recruits for us, they are our processor under the GDPR, bound by contract to protect this data. We keep the data for as long as the study needs, then destroy it. Participants exercise their rights with data@akiani.fr.
The problems encountered, screen by screen, linked to the users concerned, prioritised and illustrated with video and audio clips and anonymised verbatim quotes. Priorities and recommendations are then discussed at the presentation of the results.