It’s the fantasy of the moment. You describe a profile to an LLM, and it answers the way that person would. No more recruitment, no more scheduling, no more no-shows, no more incentives. A panel of 1,000 people in 20 minutes, for the price of an API call.
NVIDIA has just announced Nemotron, a model built to generate “synthetic personas”. In UX, a persona is the outcome of field research. You observe people, you identify recurring behaviours, you build an archetype that helps teams put themselves in users’ shoes. The Nemotron persona, on the other hand, is a profile generated by an algorithm from statistical data: the word is the same but the object has nothing to do with it. The line is blurring, and some of the people we talk to are starting to wonder why they should pay for field research when an AI can simulate the respondents.
We asked ourselves the same question — and we had what we needed to answer it.
We had the data to check
As it happens, in the summer of 2024 we ran a quantitative study for Cultura, with a panel of 1,000 French people, on their cultural practices and consumption habits. We had each respondent’s socio-demographic profile and their answers to about ten questions: income, cultural budget, reading frequency, satisfaction, second-hand buying.
We had reality. If you give exactly the same profiles to an LLM and ask it the same questions, does the fantasy hold up?
How we checked
We generated 3,000 synthetic respondents in March 2026, split across 3 conditions:
- In condition 1 (standard batch), we hand the profiles to Claude Sonnet 4.6 in batches of 5 with a standard prompt, and it processes them assembly-line style.
- In condition 2 (immersive Sonnet), we ask Sonnet 4.6 to embody each profile individually before answering. One by one, 1,000 times, with the same model but a different method.
- In condition 3 (immersive Opus), the same immersive approach with Claude Opus 4.7.
Generating 3,000 answers one by one in immersive mode takes time and costs money, but it’s the only way to find out whether the batch shortcut holds up.
The socio-demographic profiles (age, gender, socio-professional category, education) are injected directly from the real database. The LLM doesn’t invent them, it receives them. It’s then up to the model to simulate the answers to the attitude and behaviour questions.
10 variables compared, split into two categories:
- The recruitment variables (semi-deterministic) cover net monthly income, where the prompt injects the socio-professional/age bracket but the LLM picks the exact band.
- The simulated variables cover opinion on purchasing power, cultural leisure time, satisfaction, annual budget, budget change, reading frequency, number of books read per year, creative-hobby frequency, second-hand buying.
Tooling
For each variable and each condition, we measured the gap between real and synthetic distributions:
- On categorical variables, we used a Chi-square test of independence and a Cramér’s V (0 when the distributions are identical, 1 when they have nothing in common)
- On numerical variables, a Mann-Whitney U and a rank-biserial r
- To compare the conditions against each other on the same 1,000 personas, a Wilcoxon signed-rank test
- For the three conditions at once, a Friedman test
- And a Bonferroni correction over the 30 tests (10 variables × 3 conditions), with a threshold of 0.00167
What the numbers say
Overview
Cramér’s V measures the gap between the synthetic distribution and the real distribution. The lower it is, the more the synthetic resembles the real.
The lower the value, the closer the synthetic panel is to reality.
“Books read per year” is measured with a rank-biserial r (Mann-Whitney U), not Cramér's V: its values are not directly comparable with the other bars. “Books read per year” is measured with a rank-biserial r (Mann-Whitney U), not Cramér's V: its values are not directly comparable with the other bars.
View the data
| Variable | Batch (V1) | Immersive Sonnet (V2) | Immersive Opus | Best condition |
|---|---|---|---|---|
| Net monthly income | V = 0.375 | V = 0.416 | V = 0.329 | Opus |
| Opinion on income | V = 0.292 | V = 0.288 | V = 0.286 | Opus |
| Cultural leisure time/week | V = 0.391 | V = 0.236 | V = 0.259 | Immersive Sonnet |
| Cultural leisure satisfaction | V = 0.407 | V = 0.002 | V = 0.138 | Immersive Sonnet |
| Annual cultural-goods budget | V = 0.363 | V = 0.377 | V = 0.297 | Opus |
| Budget change | V = 0.469 | V = 0.463 | V = 0.484 | Immersive Sonnet |
| Reading frequency | V = 0.336 | V = 0.244 | V = 0.311 | Immersive Sonnet |
| Books read per year | r = 0.088 | r = 0.097 | r = 0.025 | Opus |
| Creative-hobby frequency | V = 0.526 | V = 0.266 | V = 0.395 | Immersive Sonnet |
| Second-hand purchase | V = 0.350 | V = 0.012 | V = 0.121 | Immersive Sonnet |
Average distance to reality
Leaving aside the recruitment variable (income), here is the overall performance across the 9 simulated variables:
The lower the value, the closer the synthetic panel is to reality.
View the data
| Condition | Mean effect | Median effect | Max effect | Variables won |
|---|---|---|---|---|
| Immersive Sonnet (V2) | 0.221 | 0.244 | 0.463 | 6/9 |
| Immersive Opus | 0.257 | 0.286 | 0.484 | 3/9 |
| Standard batch (V1) | 0.358 | 0.363 | 0.526 | 0/9 |
The standard batch doesn’t win on a single variable. Immersive prompting narrows the gap to reality by a third (from 0.358 to 0.221 on average).
What it looks like in practice
Effect-size figures don’t speak to everyone. If you’ve ever worked with questionnaire results, the gaps below should speak to you more. We show them as a gap to reality, since the client’s original distribution stays confidential.
Example 1: Cultural budget change (the worst case)
We asked respondents whether their cultural budget had changed compared with the previous year.
Zero = identical to reality; a bar pointing up over-represents the answer, down under-represents it.
View the data
| Answer | Batch (V1) | Immersive Sonnet (V2) | Opus |
|---|---|---|---|
| Less | −17 | −17 | −18 |
| The same | +39 | +39 | +40 |
| More | −10 | −9 | −10 |
| Don't know | −12 | −13 | −12 |
Cramér’s V: 0.463 to 0.484 depending on the condition. All significant (p < 10⁻⁹⁰). The LLM refuses to simulate a change in future behaviour. It crushes everything onto “the same”, and never says “don’t know”.
Example 2: Second-hand purchase (the best case in immersive mode)
We asked whether respondents buy second-hand cultural goods.
Zero = identical to reality; a bar pointing up over-represents the answer, down under-represents it.
View the data
| Answer | Batch (V1) | Immersive Sonnet (V2) | Opus |
|---|---|---|---|
| Yes | +32 | +1 | +12 |
| No | −32 | −1 | −12 |
In batch mode, the LLM over-represents second-hand buying by 32 points. In immersive Sonnet: the gap falls to 1.3 points, non-significant (Chi-square = 0.29, p = 0.59, V = 0.012). Immersive prompting erases the social-desirability bias on this variable.
Example 3: Creative-hobby frequency (the most contrasted case)
Zero = identical to reality; a bar pointing up over-represents the answer, down under-represents it.
View the data
| Answer | Batch (V1) | Immersive Sonnet (V2) | Opus |
|---|---|---|---|
| Never | −32 | 0 | −24 |
| Rarely | +18 | +10 | +17 |
| 2-3 times/week | +21 | +4 | +9 |
In batch mode, the LLM turns a mostly non-practising panel into an active one, with non-practitioners shrinking by more than 30 points. Immersive Sonnet, by contrast, reproduces the share of non-practitioners to within 0.3 points. It’s the variable where the gap between conditions is widest (V drops from 0.526 to 0.266).
Statistical significance
Across 30 tests (10 variables × 3 conditions), 27 are significant after Bonferroni correction (p < 0.00167). The 3 that are not:
| Variable | Condition | Chi-square | p | Cramér’s V |
|---|---|---|---|---|
| Leisure satisfaction | Immersive Sonnet | 0.01 | 0.93 | 0.002 |
| Books read/year | Opus | U = 473,335 | 0.35 | r = 0.025 |
| Second-hand purchase | Immersive Sonnet | 0.29 | 0.59 | 0.012 |
These are the only 3 variables where the synthetic is statistically indistinguishable from the real. 3 out of 30. All the remaining tests are significantly different.
The 3 conditions don’t do the same thing
The Friedman test compares the 3 conditions at once, on the same 1,000 personas. The result: on 9 variables out of 10, the 3 conditions produce significantly different distributions from one another (p < 10⁻¹² to p < 10⁻⁹⁷). The only exception: budget change (p = 0.067), where all 3 conditions fail in the same way.
| Variable | Batch | Immersive Sonnet | Opus |
|---|---|---|---|
| Net monthly income | 0.375 | 0.416 | 0.329 |
| Income opinion | 0.292 | 0.288 | 0.286 |
| Leisure time | 0.391 | 0.236 | 0.259 |
| Leisure satisfaction | 0.407 | 0.002 n.s. | 0.138 |
| Cultural budget | 0.363 | 0.377 | 0.297 |
| Budget change | 0.469 | 0.463 | 0.484 |
| Reading freq. | 0.336 | 0.244 | 0.311 |
| Books/year | 0.088 | 0.097 | 0.025 n.s. |
| Creative hobbies | 0.526 | 0.266 | 0.395 |
| Second-hand | 0.350 | 0.012 n.s. | 0.121 |
“Books/year” is measured with a rank-biserial r (Mann-Whitney U), not Cramér's V. The “n.s.” cells are not statistically distinguishable from reality.
That means the choice of prompting method has a measurable, reproducible impact. It’s not noise.
5 ways to get it wrong
All confirmed at n = 1,000, significant after Bonferroni correction.
-
The LLM doesn’t simulate change (V = 0.46 to 0.48). On budget change, the conditions pile almost every answer onto “the same”, about 39 points above reality. It’s the strongest, most stubborn bias: no condition corrects it.
-
The high brackets (income > €4,000, > 20h of leisure a week) and the low ones (< €1,000, life rated “very hard”) are systematically under-represented (V = 0.29 to 0.42). The LLM pulls everything back towards the average.
-
The LLM over-represents socially valued behaviours such as second-hand buying or cultural engagement (V = 0.01 to 0.35 depending on the condition). Immersive prompting reduces this bias significantly but doesn’t eliminate it on every variable.
-
LLMs produce personas who read more, do more, and report being more dissatisfied at not being able to do even more (V = 0.24 to 0.53). As if the synthetic personas were the idealised version of real people.
-
“Don’t know” is a sizeable share of real answers on budget and its change, and almost zero in the synthetic. The LLM always has an opinion. When a real respondent says “don’t know”, they teach us something; when the LLM never does, it hides something from us.
Where does the bias come from? We traced it to the source.
We observe that LLMs systematically over-estimate respondents’ cultural engagement. But is it the LLM projecting an engagement it wasn’t asked for, or is it the persona dataset that is already biased?
To find out, we had Gemini 2.5 Pro score the 1,000 Nemotron personas: for each persona, a score from 0 to 10 measuring the density of real cultural practices described in its profile. Not tastes, not intentions. Practices: “plays the piano 3 times a week”, “visits the Tours fine-arts museum every Sunday”.
The Nemotron dataset is biased at the source
View the data
| Engagement level | Score | Nemotron personas | Real French population (INSEE) |
|---|---|---|---|
| Low | 0-2 | 5.0 % | 30-40 % |
| Moderate | 3-4 | 24.1 % | — |
| Significant | 5-6 | 37.2 % | — |
| Intense | 7+ | 33.7 % | — |
5% of culturally inactive personas, versus 30 to 40% in the real French population. The Nemotron dataset contains almost no persona who does nothing. Everyone reads and has cultural activities — it’s an idealised panel rather than a representative one.
The LLM faithfully follows the dataset’s bias
We measure the correlation between the persona’s cultural score and the LLM’s answers (Spearman):
View the data
| Variable | Immersive Sonnet (rho) | Opus (rho) |
|---|---|---|
| Relationship to culture | 0.49 | 0.72 |
| Reading frequency | 0.33 | 0.44 |
| Creative hobbies | 0.38 | 0.37 |
| Books read/year | 0.42 | 0.50 |
| Second-hand purchase | 0.34 | 0.43 |
All the correlations are moderate to strong. The LLM doesn’t project a cultural engagement of its own. It reads the persona, and it follows it. If the persona describes someone who “visits contemporary-art exhibitions and attends engraving workshops”, the LLM answers “culture matters to me” — and it’s right to do so.
The problem is that Nemotron almost never generates a persona who “has no artistic interests” or who “watches TV in the evening and that’s enough for them”. As a result, the LLM never gets the chance to answer “I’m not interested in culture”.
93% dataset, 7% LLM
We cross-validated this diagnosis with two independent evaluator models (Gemini 2.5 Pro and Claude Sonnet 4.6). Inter-rater agreement: Cohen’s kappa = 0.77 (substantial agreement), Spearman correlation = 0.79. Both converge on the same finding.
The estimated contribution is 93% Nemotron and 7% LLM.
Bias already present in the personas generated by the dataset · Share attributable to the model that simulates the answers
View the data
| Source du biais | Contribution estimée |
|---|---|
| Nemotron dataset | 93% |
| LLM | 7% |
| Validation croisée par deux modèles évaluateurs indépendants (Gemini 2.5 Pro et Claude Sonnet 4.6) : accord inter-rater kappa de Cohen = 0.77 (accord substantiel), corrélation Spearman = 0.79. | |
It changes how we read the problem. The synthetic-panel fantasy doesn’t only run into the limits of the LLM. It runs into the limits of the dataset used to build the personas. NVIDIA calls them “synthetic personas”, but they are personas without ordinary people.
What we make of it
The idea has been circulating in the research community for a few years now, under the name “silicon samples”. Most published studies look at English-speaking populations and compare synthetic answers with opinion polls. What we wanted to do here was to check against real French data, on a questionnaire we designed ourselves, and on a topic where social-desirability biases are especially active.
The synthetic-panel fantasy rests on an implicit assumption: that an LLM trained on billions of texts “knows” how people answer questionnaires. That’s partly true, but what it knows is how people are supposed to answer, not how they actually answer.
An LLM doesn’t doubt, doesn’t change its mind, and doesn’t lie about its income. It doesn’t tick “don’t know” because it can’t be bothered or because the question makes it uncomfortable. It doesn’t over-report its leisure hours because it mistakes its ideal for its reality.
Synthetic panels aren’t useless. On 3 variables out of 10, the synthetic is indistinguishable from the real. Immersive prompting narrows the gap to reality by a third. That’s usable knowledge, and you can use it to pre-test a questionnaire or explore segmentations before going to check in the field.
But if you use them as a substitute for fieldwork, you confuse what people should answer with what they do answer. And that confusion, in user research, is exactly the problem we’re trying to solve.
