Related methods
User testing
User testing means having target users assess the usability of your product or service. Depending on the goals, several approaches are possible.
Read more about User testingResearch protocol
UX method
Framing user research within a research protocol
The research protocol, or experimental design, is the document that gives traceability to a research process. User studies rely on the experimental method, which aims to answer research questions or validate hypotheses using methods that fit the goals. The research protocol gathers the description of the conditions, how the session runs and the setup in place to collect data during an experiment or a test in order to answer those questions.
What is a research protocol for?
- Stating the questions being asked and the goals of the study
- Spelling out the methods and the related goals
- Controlling biases (as much as possible) by precisely describing the instructions and the conditions under which data is collected
- Describing the resources used, the type of data collected and the analyses envisaged
Why build a research protocol?
- At kick-off, to frame a user study
- During preparation, to anticipate the data analysis
- For the analysis step, to make sure the questions asked are properly answered
- Once the study is complete, the protocol is an important record for history and for revisiting the results in the future
Setting up a research protocol
01
Familiarising with the topic
The first step when planning to launch a UX study is to gather the data and knowledge (readily available) on the topic, to kick off the thinking.
02
Research questions
In consultation with the project teams, the first task is to define what falls inside and outside the scope of a study.
03
The data collection setup
Once the research questions are clear, we can pin down which methods to favour. At this step, it's essential to factor in time constraints, budget, and also access to data (and sometimes to the users themselves). The protocol also describes the data that will be collected and the analyses envisaged.
04
Data data collection materials
We know what we're looking for and how we're going to look for it. What's left is to define the flow of a data collection session more precisely. At this step, we produce the data collection materials such as an interview guide, an observation grid, a user-testing scenario, and so on.
The ethics of user research
Whenever we ask a user to share information about their practices, we owe them complete transparency. First, we make sure to respect the principle of informed consent. This means clearly explaining the objectives of the request and confirming that the user is volunteering to take part. We then have a responsibility to protect any personal data we may collect. For this we rely on our GDPR charter, a data processing document specific to each study, an anonymisation procedure and managed archiving of the results.
Frequently asked questions
An experimental protocol is the document that describes, precisely and reproducibly, how a study will be conducted: objectives, hypotheses, target population, recruitment method, observation conditions, stimuli presented, metrics collected and analysis plan. It guarantees that several researchers can run the same study consistently and that the results are comparable over time.
Without a formalised protocol, each test session can drift subtly: slightly different questions, a changing order of presentation, varying stimuli. These variations make the results incomparable between sessions and between participants. The protocol is also useful for obtaining ethical approvals (necessary in certain academic or medical contexts) and for documenting the study for future replications.
A complete protocol includes: the research question and hypotheses, the description of participants (inclusion and exclusion criteria), the recruitment method, the study design (number of conditions, order of presentation), the material and stimuli used, the session-by-session procedure, the metrics and how they are collected, and the planned analysis plan. In critical studies, you add an analysis of potential biases and the measures to control them.
No. The user test is a method. The experimental protocol is the documentation that frames any study method, including user tests. A user test without a formalised protocol is still a user test, but its results will be less reliable and less comparable. Formalising the protocol is particularly important for comparative studies (A/B, before/after) where methodological rigour determines the validity of the conclusions.
By distinguishing what must be rigorous (the hypotheses, the metrics, the recruitment criteria) from what can be lightened (the formal documentation). A one-page 'light protocol' with the key questions, tasks and metrics is enough for most user tests in a sprint. Rigour does not require heaviness: it requires clarity about what you are measuring and why.
A few of our case studies
What travellers really look at between two stations
View the case study : What travellers really look at between two stationsCurious to learn more about us?
Everyone talks about user experience, service design, ergonomics...
It's not always clear, and you'd like to find out, or sharpen what you already know.