Feedback from the field
Building on the feedback of professionals in the field
To carry out this study, we meet with the project team to validate together the objectives, the expected outcomes of the project and the methodologies to put in place.
Following this strategic organisation phase, we talk to road-tunnel operators with varied characteristics.
An essential step to help us build familiarity with the topics covered and to understand the emergency-closure issue for tunnels.
After analysing these interviews, we formulate several hypotheses that feed into the construction of the experimental protocol. As examples, the readability of certain devices or the lack of awareness of the specific regulations for tunnels are taken into account.
To diversify the feedback, we include different road-tunnel configurations in this first part of the study.
For instance, the issues encountered are very different between structures located in urban environments or on motorways. The one-way (single direction) or two-way (two directions of traffic) nature is also a characteristic that influences the topic of the study. Other factors are also taken into account: signage devices installed, the region the tunnel is in, the types of users who use it, and so on.
Review of existing practice
Centralising the state of knowledge on the issue and identifying innovative devices
Desk review
To complement the field interviews, we examine different documentary sources.
We first look at the technical regulations related to signage at tunnel entrances. In a second step, we look for scientific studies on the topic of signage and understanding issues. We pay particular attention to tunnel-approach situations.
Evolution of signage devices
To feed the experimental phase, we also research the signage devices in use in other environments.
For example, we identify solutions deployed to regulate traffic at urban junctions: car-tram crossings. But also in peri-urban contexts with level crossings.
This research phase is carried out for France and abroad, where some tests are run at tunnel entrances.
Experimental protocol
Preparing the test environment, the tasks to be performed and defining the participants' profiles
Experimental plan
The findings gathered from the subject-matter experts allow us to frame part of the hypotheses to test in an experimental context. By cross-referencing them with the review of existing practice, we define a protocol structured around several pillars:
- a realistic dynamic environment,
- reaction time,
- elicitation interviews,
- drivers’ mental models,
- and risk awareness.
The aim is to expose participants to varied signage configurations while taking two types of measurement.
- First, we run a performance measurement in a hazard-detection task.
- Second, we gather subjective measures linked to the perceived danger and effectiveness.
Finally, to round out the results obtained, we run a semi-structured interview centred on different identified sub-themes.
Co-defining the scenarios
In this kind of experiment, scenarios drive everything. Taking the exploratory data into account, we draw up a list of scenarios that vary by the signage device used on one hand and the road context on the other. We meet again with the project team to iterate on and challenge this list of scenarios. A co-construction effort that requires taking many criteria into account:
- inclusion of existing devices,
- proposal of innovative directions,
- integration of rule-breaking behaviour by other road users,
- production time, and so on.
Producing the test materials
Beyond the classic materials needed for any user test, this experiment requires a more significant preparation and logistics effort. The test environment includes:
- the production of custom-built high-definition 3D sequences respecting a precise technical specification (thanks to our partner Tribia (opens in new window)!),
- the creation of an application to play the 3D sequences while capturing the user’s reaction-time data,
- the creation of a second application to record the subjective evaluations during self-confrontation.
Recruiting the user panel
For this project, we target participants who are all road users. The criteria are defined notably in connection with the participants’ driving-experience profile (types of trips, how long they have held their licence, driving frequency, and so on).
Three groups are made up to cover several profiles: less experienced drivers with recent knowledge of driving theory, experienced drivers with older knowledge of the Highway Code, and finally users for whom two-wheelers are the main mode of transport.
Since the tests take place in person at our Bordeaux offices, we recruit users from the Nouvelle-Aquitaine region.
User testing
Putting road users in front of the system under study
Detection task
The experiment starts directly with the hazard-detection task. Users are placed in front of a two-metre diagonal screen on which video sequences are played and have a keyboard in front of them. The instruction is to watch the sequences (more than sixty in total) and react by pressing the space bar if they detect that they are going to have to slow down.
The sequences are presented in a random order with a significant proportion of decoys (reference situations with no stop instruction). This task lets us measure detection performance (whether or not the user detects, reaction time and distance to the obstacle).
Subjective evaluations
Once the video protocol is over, the participants this time look at still frames of the signage tested. For each one, the user assesses dimensions linked to the dangerousness of the situation and the perceived effectiveness of the signage.
To weight the evaluations of the different signage types, we add a sequence-ranking task. We round out the numerical data with the reasoning that the participants detail when commenting on their choices.
A semi-structured interview then concludes the session. This time includes questions on the regulations or on attitudes in interrupted-traffic situations, in particular. It is also the opportunity to come back to all the situations tested and to open the discussion to similar situations the users have come across in real conditions.
Data analysis and recommendations
Working through the data and turning the results into recommendations
We centralise all the data measured during the experimental tasks in aggregated files that serve as the basis for analysis. The analysis then covers the results of the different tasks before cross-referencing the objective and subjective feedback. Finally, we compare the results with the different signage variants used in the protocol. The interview data deepens the argument by shedding light on the findings from the users’ point of view.
Ultimately, this study surfaces ten recommendations covering different aspects of the overall system. Each recommendation starts from an observation and offers a direction to improve, in the end, the road users’ understanding of the signage. We propose a priority level for each one to kick off the next deployment phase.
These recommendations take several forms: awareness-raising actions, setting up targeted experiments, evolving work processes, and so on.
The Akiani project team adapted very quickly to the world of tunnels and its specificities, thanks to a substantial initial effort of research and familiarisation in collaboration with the road-tunnel operators. For the rest of the study, their human-factors expertise allowed us to better grasp how drivers think while driving. The team, dynamic and responsive, was able to respond to our questions and requests throughout the study and provided us with high-quality outputs. Our challenge and what we expected from the study were well understood, and the objectives were met. Putting the deliverables provided to use will, in the end, help us advance safety in road tunnels.