Synthetic Focus Group: Rehearse the Discussion Guide First

A synthetic focus group can rehearse the discussion guide before fieldwork: question order, stimuli, probes and timing. Nothing said there enters the report.

A round of focus groups is booked. The recruiter is filling the quotas, the viewing facility is confirmed, and the client team will sit behind the one-way mirror. The day before, the moderator is still reworking the discussion guide: the order of the concepts, the probe after a stimulus, whether the core section has enough time. In most projects nobody runs that guide in front of anyone before the first group starts. A synthetic focus group, a simulated discussion among language model personas, can give the guide that rehearsal, provided one rule holds: nothing said in the rehearsal enters the report.

Key Takeaways

• A discussion guide, also called a topic guide, is the outline of issues and questions that steers a qualitative group, as defined by the UK Association for Qualitative Research.
• A 2016 systematic review of interview guide development lists pilot testing as one of five phases. Commercial focus groups often skip it, and the first paid group becomes the pilot.
• In a 2024 comparison with five human focus groups and 23 participants, simulated groups covered most human opinions, then repeated the common ones and stopped producing new codes while people kept adding.
• That profile makes a simulated group useful for testing question order, stimulus sequence, probes and timing, and unusable as a source of findings.
• The rule that keeps the rehearsal honest: transcripts from synthetic personas never reach analysis, the topline or the report.

What a discussion guide is, and what it has to survive

The Association for Qualitative Research glossary describes the guide as an outline of key issues and areas of questioning. It also notes that guides vary widely in detail and in how flexibly they are applied.

A discussion guide is the moderator's plan for a focus group or depth interview: the sections, the questions inside each one, the stimuli to show, the probes to use and the minutes allotted to each part. The client usually approves it before fieldwork.

A probe is the follow-up question a moderator asks after the first answer, to get past the polite version. "What made you say that?" is a probe. "So you liked it?" is a leading question pretending to be one.

A stimulus is anything shown to the group, such as a concept board, a pack, an ad or a price card. The order in which stimuli appear shapes how each one is read.

A guide has to survive a room of real people, on the clock, with a client watching. Much of what goes wrong in a group traces back to a guide that was never tested against that room.

How the guide gets tested today

PERSONAL EXPERIENCE In twenty years of market research I have seen guides prepared with care and almost never rehearsed. The usual sequence looks like this.

1. The moderator drafts the guide from the brief, section by section, with time per section.
2. The client team reads it and returns comments, mostly about wording and about questions they want added.
3. The moderator merges the comments, and the guide grows by a few questions and loses no minutes.
4. Someone reads it aloud alone, or walks a colleague through it over coffee.
5. The first real group runs the guide for the first time.

Methodology does not treat step five as acceptable. A 2016 systematic review on developing a semi-structured interview guide, published in the Journal of Advanced Nursing, synthesized the process into five phases, and the fourth is pilot testing the guide before data collection. Academic teams pilot with a few volunteers. Commercial projects, on a tight timeline with the groups already booked, usually treat the first group as the pilot without calling it that.

Quantitative teams settled the equivalent question long ago. Nobody fields a programmed questionnaire without testing the script, as we argued in survey testing before fieldwork. The qualitative instrument rarely gets the same treatment.

What it costs when the first group is the pilot

PERSONAL EXPERIENCE The defects that a rehearsal would catch are predictable, because they come from the guide and not from the people.

For example, the core section runs long and the last concept gets a few rushed minutes. The second stimulus reframes the third, so the third is never read on its own. A probe written as a question with a built-in answer produces nods. An exercise instruction that seemed clear on paper takes longer to explain than the exercise takes to do.

Each of these costs something. The participants were recruited against quotas and paid an incentive, and their time went into exposing a flaw in the guide while the client watched. The fix creates a second problem. If the guide changes between group one and group two, the groups no longer answer the same instrument, and every comparison across them weakens.

The person who answers for it is the qualitative lead. They signed the guide, they sit with the client in the debrief, and they are the one explaining why the last concept got so little time.

What a synthetic focus group is

A synthetic focus group is a simulated discussion in which a language model plays several participants, each built as a persona, while a moderator, human or model, runs a discussion guide with them.

A synthetic persona is a model prompted and grounded to answer as a defined member of a target audience would. It is a different thing from synthetic data, which is a generated dataset that imitates the statistical shape of a real one. We drew that distinction at length in our article on synthetic sample augmentation.

Using simulated discussions as a cheap pilot is already in the literature. A paper on synthetic discussion generation, first posted in March 2025 and revised in April 2026, proposes them as low-cost pilot experiments. Its authors report that a simulation can reveal the limitations of a design before any human discussant is engaged.

What the research says a simulated group reproduces and misses

The most direct comparison comes from Focus Agent, presented at the ACM conference on Intelligent Virtual Agents in 2024. The researchers ran five focus groups with 23 human participants on digital well-being and simulated the same discussions with model personas.

Content analysis produced 39 distinct codes from the human groups and 47 from the simulations, and most opinions the people expressed also appeared in the simulated groups. The differences matter more for our purpose. The simulated participants repeated common opinions across sessions, and after a few rounds they stopped producing new codes. The human participants kept adding new ones. The authors read this as the model favoring common viewpoints, while people show more variance and individuality.

A second detail came from their own pilot. People do not always have something to say on every question, while simulated participants always generate content. The model moderator kept repeating questions when the human group went quiet, and the authors had to change its protocol so it would move on.

A third finding, from a separate line of work, sets the hard limit. A study of models standing in for human participants, run with 3,200 people across 16 demographic identities and four models, found that models tend to misportray demographic groups and flatten the variety inside them.

What a simulated group reproduces, and what it misses

Two published comparisons between simulated and real participants. Panel one, from Focus Agent, presented at the ACM conference on Intelligent Virtual Agents in 2024: five focus groups with 23 human participants on digital well-being were also simulated with model personas. Content analysis found 39 distinct codes in the human groups and 47 in the simulated groups. Reproduced: most opinions the people expressed also appeared in the simulation, which also added viewpoints nobody in the human groups raised. Missed: after a few rounds the simulation stopped producing new codes while the human groups kept adding them, and simulated participants always had something to say. Panel two, from Wang, Morgenstern and Dickerson on arXiv: four models compared with human studies of 3,200 participants across 16 demographic identities misportrayed demographic groups and flattened the variety inside them. A closing line says a simulated group is good enough to stress a discussion guide and never good enough to report. Every number belongs to the source named beside it and none of them is a Cassi.ai measurement.

More codes from the simulation does not mean more insight. The simulated groups repeated the common opinions, and the new ones kept coming only from people.

Put the three together and the profile is clear. A simulated group talks smoothly, covers the obvious ground and always has something to say. That is useless as evidence of what a target thinks, and useful as a stress test for a guide, because a guide that breaks even with cooperative participants will break harder with real ones.

Rehearse the guide, never the findings

UNIQUE INSIGHT The line between the two uses is the whole method. A rehearsal asks questions about the instrument. A study asks questions about people. The table shows what each part of the guide can and cannot learn from a synthetic run.

Sources for the limits in the right column: Focus Agent and the study on misportrayed and flattened groups. The mapping onto the parts of a guide is ours.

The last row is the rule. Synthetic answers about the category, the brand or the concepts are a byproduct of the rehearsal, and they are discarded with it.

The rehearsal, step by step

UNIQUE INSIGHT This is the procedure we recommend for one guide before a round of groups.

1. Freeze version one of the guide, with minutes allotted to every section, from the warm-up to the closing exercise.
2. Build the personas from the screener profile and from studies the client already owns, and label every one as synthetic.
3. Run the guide word for word, with the moderator reading it as written.
4. Run it again with a different mix, including one dominant persona and one that barely answers.
5. Log, section by section, the time used, where answers drifted off topic, which probes got yes or no, and where one stimulus leaked into the next.
6. Revise the guide and record each change, with the defect that motivated it.
7. Quarantine every transcript: it stays out of analysis, coding, the topline and the report.
8. Run every real group on version two, unchanged, so the groups stay comparable.

A guide rehearsal with a wall around it

A flow in five steps on the left, each with its output on the right. Step one, freeze version one of the discussion guide, with every section timed and every stimulus in order. Its output is the baseline, the guide the client approved, kept for comparison with the final version. Step two, run the guide twice with synthetic personas, word for word, with a dominant persona and a quiet one in the second run. Its output is the transcripts, marked in orange and quarantined: never coded, never quoted, never in the topline or the report. Step three, log the defects section by section: time used, leading probes, stimulus leaks and unclear instructions. The defect log is the only thing the rehearsal hands to the next step. Step four, revise and freeze version two, with each change recorded next to the defect that motivated it, in a change log kept with the study. Step five, run every real group on version two, unchanged across the round. Its output is the findings, and that is the only box that feeds the report. A closing band says every persona is labeled synthetic and no synthetic answer is presented as a real participant’s answer. The figure contains no measured numbers.

The orange box is the one that protects the client. A synthetic quote in a debrief deck reads exactly like a real one.

Step seven is the one that protects the client. A quote from a synthetic persona that slips into a debrief deck reads exactly like a quote from a person, and nobody in the room can tell the difference afterwards.

What the rehearsal will get wrong

UNIQUE INSIGHT A rehearsal with model personas is generous to the guide, and that bias is predictable.

Silence will be underrepresented, because simulated participants always produce content, as the Focus Agent pilot found. The rehearsal will not show where a real group stalls.

Timing is an estimate. Focus Agent itself allotted time by text length, at roughly one hundred words per simulated minute. Treat the rehearsal's clock as a test of whether the guide fits on paper, and keep a margin for the room.

Group dynamics are thin. Dominance, social desirability and the participant who changes the room's mind are human behavior, and the identity study shows how much variety a model compresses. The rehearsal checks the instrument. The room still has to be moderated.

Where Cassi.ai comes in

cassiai.com is a software engineering company specialized in the pains of market research, innovation and insights, founded by Cassiano Albuquerque after twenty years in the field. We build for research agencies and corporate insights teams.

Synthetic Lab is a Cassi.ai product that turns the research already done into synthetic personas of the target, for synthetic focus groups and quantitative studies with quotas.

Synthetic Lab is our product for research with synthetic personas. Its Mirror Room puts twelve personas of a target in a synthetic focus group with a moderator, while the team watches from behind the glass and can step in with a question. The personas are grounded in studies the client already owns, and a copilot drafts the discussion guide, which a person approves before anything runs.

The product has a rule that matters here: it never presents a synthetic answer as a real consumer's answer, and every output is labeled synthetic. For guide rehearsal we go one step further and keep the transcripts out of the deliverable entirely. When a decision needs real people, the Mirror Room is where the guide gets tested, and the facility is where the findings come from. Our evidence on when synthetic respondents can screen concepts is in the fidelity data article.

FAQ

What is a synthetic focus group?

A simulated discussion in which a language model plays several participants built as personas, while a moderator runs a discussion guide. The Focus Agent study compared one with five human groups and found broad overlap on common opinions and much less variety.

Can a synthetic focus group replace a real one?

No. In the published comparison, simulated participants stopped producing new codes after a few rounds while people kept adding them, and a study with 3,200 participants found models flatten the variety inside demographic groups. Use it to test the instrument, never to report what a target thinks.

How do you pilot a focus group discussion guide?

Run the guide word for word before fieldwork, log time per section, leading probes and stimulus order effects, then revise and freeze it. Pilot testing is the fourth of five phases in a 2016 review of interview guide development.

What should a discussion guide rehearsal check?

Question order, the sequence of stimuli, the wording of probes, the minutes per section and the clarity of exercise instructions. It should never be used to check which concept wins.

Can synthetic transcripts go into the report?

No. Keep them out of coding, the topline and the debrief. A synthetic quote in a deck reads like a person's quote, and that is how a rehearsal turns into a fabricated finding.

The first group of a round has one job, which is to tell the client something about people. A guide rehearsed beforehand lets it do that job, and the rehearsal stays in the drawer where it belongs.

If your next round of groups is booked and the guide has never been run aloud, send us the guide and two past studies of the same target. We will set up the personas and run the rehearsal with your moderator, so the version you freeze has been run once before the client watches it.

About the author

Cassiano Albuquerque is the founder of cassiai.com. He has spent twenty years in market research, was ESOMAR's representative in Brazil and speaks at IIEX, ESOMAR and ABEP. He now builds AI systems for research teams. To discuss rehearsing a discussion guide with synthetic personas grounded in your own studies, contact him on LinkedIn or see the cassiai.com portfolio.

Published by Cassi.ai. Read the full article at https://www.cassiai.com/blog/synthetic-focus-group-discussion-guide.