Ad Hoc Research Requests: Triage Before the Next Wave

Ad hoc research requests arrive between waves with a meeting already booked. A triage in three exits: the archive, a labeled persona hypothesis, or fieldwork...

The quarterly wave of the usage and attitude study closed three weeks ago, and the debrief went well. On Tuesday the brand manager writes to the insights team. The agency wants to drop the scent claim from the refill pack. The meeting to decide is on Thursday. Would the price-sensitive buyers who already use the refill still pick it up without that line? Nobody asked that in the last wave. The next one fields in eleven weeks. Most ad hoc research requests look like this one. Most get answered with an opinion, or with a slide built for a different question. There is a third way, provided the answer is labeled for what it is.

Key Takeaways

  • The UK Association for Qualitative Research defines ad hoc research as a one-off study for a specific purpose. A tracker is the opposite, continuous research.
  • In a poll Stravito ran during a session with Quirk's, 45 percent of attendees said their top challenge was being outnumbered by stakeholder requests. Another 36 percent said it was finding research they know exists.
  • In a working paper by Brand, Israeli and Ngwe, fine-tuning a model on past survey data improved its match with human answers in the same category and population. It did not help for new categories or for differences between segments.
  • That boundary gives a triage with three exits. The archive answers, a persona built from the company's own studies offers a labeled hypothesis, or fieldwork decides.
  • A synthetic answer never enters the deck as something a consumer said.

What an ad hoc research request looks like between waves

The Association for Qualitative Research glossary defines ad hoc research as work designed for a specific purpose and a specific client. It runs as a one-off study, or a programme of studies, and not on a continuous basis.

An ad hoc research request is a question that reaches the insights team outside the research calendar. It is usually tied to a decision that already has a date. The decision is tactical: a claim, a pack variant, a promotion mechanic, the order of two benefits in a radio spot.

A wave is one round of fieldwork in a continuous study, such as a brand tracker or a quarterly usage and attitude survey. The questionnaire is fixed months ahead, so a question that appears between waves has nowhere to go until the next one.

These requests are not rare and not unreasonable. They come from people who respect research enough to ask for it. The problem is arithmetic. The calendar has four waves a year, and the brand team has a meeting every week.

How the question gets answered today

PERSONAL EXPERIENCE In the insights teams I have worked with, the routine is the same whatever the category. Nobody designed it. It grew around the gap between the calendar and the meetings.

  1. The request arrives by email or chat, phrased as a question and not as a decision.
  2. An analyst searches shared folders for a past deck that mentions the topic.
  3. The closest slide turns up, from a study two waves back, with a different base and a question worded for another purpose.
  4. The analyst pastes it into a reply with a caveat that nobody reads in the meeting.
  5. When no slide exists, the insights lead gives an informed opinion. Or the question goes into the next wave, and the decision is made without it.

The pressure behind that routine has been put into numbers, by a vendor and with a sample nobody published. Stravito ran a poll during a session with Quirk's on the top challenges for insights teams. Being outnumbered by stakeholder requests took 45 percent of the votes. Finding research the team knows already exists took 36 percent. Together, the two answers took 81 percent of the votes.

Listen Labs, another vendor, gives the queue a name. Its playbook on research team backlog defines a research backlog as the open requests that exceed a team's capacity. Backlog age is how long the oldest open request has waited.

What it costs when the answer is an opinion or an old slide

The old slide is the more dangerous of the two, because it looks like evidence. It carries a logo, a base size and a chart. It also carries a field date from two waves ago and a sample that may not include refill users. And its question asked about the scent, not about the claim on the pack. In the meeting, all of that compresses into "research says".

The opinion is at least honest about what it is. Its cost is different. The decision rests on the most senior person's intuition, and the insights team becomes the department that could not answer in time.

The third outcome costs the tracker. A question that keeps coming back gets added to the next wave. The questionnaire grows by two items. The comparability of the series pays for one meeting. We wrote about what that kind of change does to a tracker in our article on brand tracker significance.

PERSONAL EXPERIENCE The person who answers for all three outcomes is the insights lead. They sit in the room on Thursday, and they forwarded the old slide. In six months, they will be asked why the refill launch was decided on a guess.

What a model built from past studies can and cannot answer

A language model on its own answers like the average of what it read. The more useful question is what happens when it is given the company's own studies of the same category and the same buyers.

A synthetic persona is a model prompted and grounded to answer as a defined member of a target audience would. It is a different thing from synthetic data, which is a generated dataset that imitates the statistical shape of a real one.

Grounding is giving the model specific source material, here the company's past studies of the target. The model is told to answer from that material and not from general knowledge.

Fine-tuning is further training of an existing model on a small, specific dataset, such as answers from past surveys. Its outputs then move toward that data.

The clearest evidence comes from a working paper by James Brand, Ayelet Israeli and Donald Ngwe, Using LLMs for Market Research, listed in the Marketing Science Institute library. The authors queried models for dozens of answers to each survey question and compared the resulting estimates of willingness to pay with human studies. Out of the box, the estimates were sometimes comparable to human results, often inaccurate, and in some cases had the wrong sign.

Then they fine-tuned the model on previous survey data from similar contexts. Alignment with human answers improved for existing features and for new ones. That held as long as the product category and the population stayed the same. It did not improve for new product categories, and it did not improve for differences between customer segments. The authors conclude that the model belongs as a supplement to human studies and never as a substitute.

The Harvard Business School AI Institute summary of the same line of research adds a concrete failure. Models tend to rate novelty higher than people do. So synthetic customers liked pancake-flavored toothpaste at first, until fine-tuning on a real study corrected them. The summary quotes the authors directly: for anything beyond early-stage, high-level trend detection, human research remains essential.

A separate study of models standing in for human participants ran with 3,200 people across 16 demographic identities. Models misportrayed demographic groups and flattened the variety inside them. That is the same boundary seen from another side. A persona is weakest exactly where a segment differs from its neighbors.

Where past survey data helped the model, and where it stopped

A summary of the working paper Using LLMs for Market Research by James Brand, Ayelet Israeli and Donald Ngwe, listed by the Marketing Science Institute, which compared willingness to pay estimated from language model answers with human studies. A band at the top says that out of the box the estimates were sometimes comparable to human studies, often inaccurate, and in some cases had the wrong sign. Below, the results after fine-tuning the model on previous survey data, in two columns. Improved, in blue: existing features in the same product category and the same population as the survey data, and new features still inside that category and population. Did not improve, in orange: new product categories that the past surveys never covered, and differences between customer segments. A source line says the authors conclude the model is a supplement to human studies, not a substitute. A closing line says that inside the territory of your own studies the right output is a labeled hypothesis and outside it the right output is fieldwork. The figure reproduces findings and no numbers, and none of it is a Cassi.ai measurement.

The studies you already own define where a persona can offer a hypothesis. A new category or a difference between segments sends the question to the field.

Triage in three exits

UNIQUE INSIGHT The boundary in that paper sorts most ad hoc questions on its own. A question inside the category and buyers the company has already studied can get a hypothesis from personas built on those studies. A question that leaves that territory goes to fieldwork. And before either, somebody checks whether the answer is already sitting in the archive.

Kind of questionExitWhyWhat the requester receives
What did the last wave say about refill users?ArchiveThe answer exists, measuredThe page, with field date, base and question wording
Would refill users still pick the pack without the scent claim?Persona hypothesisSame category and buyers as the company's studiesA labeled hypothesis and what fieldwork would need to confirm it
Has awareness moved since the last wave?FieldworkChange over time has to be measuredA slot in the next wave or a quick quantitative study
Which segment reacts more to the new claim?FieldworkSegment differences did not improve with fine-tuningA study design with a base per segment
How would buyers react to our first pet food line?FieldworkA new category sits outside the studies the personas came fromA study design, with the archive as context

The limits in the third column come from Brand, Israeli and Ngwe and from the study on flattened identity groups. The mapping onto kinds of request is ours.

Read the second row carefully. The persona exit exists for questions that are narrow and tactical, inside known territory. On its own, such a question would never justify a study. It is also the only row whose answer has to be labeled as something other than a measurement.

The triage, step by step

UNIQUE INSIGHT This is the sequence we recommend for every request that arrives between waves.

  1. Rewrite the request as a decision with a date: "On Thursday we decide whether the refill pack keeps the scent claim."
  2. Search the archive first. Read field date, base and wording before quoting anything, as we argued in reusing past market research.
  3. If the archive answers, send the page and stop.
  4. Check whether the question stays inside the category and the buyers covered by the studies you own. If it does not, it goes to fieldwork.
  5. If it does, ask personas built from those studies. Run the same question several times and keep the spread of answers.
  6. Label the output a synthetic hypothesis and name its source studies. Write down what fieldwork would need to show to confirm it.
  7. Log the request, the exit and the decision, then compare with the next wave when it fields.

Three exits for a question that came up between waves

A decision flow in four steps, asked in order, where the first exit that fits ends the triage. Step one asks whether the question is already answered in the archive, after reading field date, base and question wording. If yes, the exit is the archive: send the page with its field date, base and wording, and stop. Step two asks whether the question stays inside the category and the buyers of the studies the company owns. If no, the exit is fieldwork: a slot in the next wave or a quick study of its own. Step three asks whether the question is about a difference between segments or a change over time, since both have to be measured with people. If yes, the exit is fieldwork, with a study design that has a base for each segment. Step four, when none of the earlier exits applied, is to ask personas built from those studies, running the same question several times and keeping the spread of answers. Its output is a persona hypothesis, labeled synthetic, with the source studies named and with what fieldwork would need to show to confirm it. A closing band says no synthetic answer enters the deck as a percentage of consumers or as a verbatim quote. The figure contains no measured numbers.

The persona exit is the last one asked, and the only one whose answer is labeled as something other than a measurement.

Step five is where the tests from our article on segmentation study personas apply. A persona that changes its answer between runs is telling you the studies behind it do not cover the question.

Step seven is the one most teams skip, and it is the one that makes the method improve. After a year, the log shows which kinds of question the personas handled and which ones the next wave contradicted.

What a persona hypothesis can carry into the meeting

UNIQUE INSIGHT A labeled hypothesis helps a decision meeting because it states its uncertainty. An old slide never does. It also has firm limits on what it can say.

It can say how buyers like those in the studies would probably read the claim. It can list likely objections and wording that risks confusion. It cannot report a percentage of consumers, because no consumer answered. It cannot appear as a verbatim in a deck. Once a synthetic quote leaves its file, it reads exactly like a real one. And it cannot settle a difference between two segments, which is precisely where the evidence says these models fail.

The wording on the slide matters more than it seems. Try this line: "Hypothesis from synthetic personas built on waves one to four, not measured with consumers." A brand manager can act on it, and an auditor can trace it. "Consumers said they would still buy it" is a fabricated finding, whatever the intention behind it.

Where Cassi.ai comes in

cassiai.com is a software engineering company specialized in the pains of market research, innovation and insights, founded by Cassiano Albuquerque after twenty years in the field. We understand research first, and we automate research projects with or without language models.

Synthetic Lab is a Cassi.ai product that turns the research already done into synthetic personas of the target, for synthetic focus groups and quantitative studies with quotas.

Synthetic Lab is our product for research with synthetic personas, built for tactical questions that would never justify fieldwork. It builds personas of one target and grounds them in the studies the company already has. The team can interview them one to one, or put twelve in a synthetic focus group. Every insight points to the interviews and sources that support it.

The product has two rules that match this triage. It never presents a synthetic answer as a real consumer's answer, and every output is labeled synthetic. And it never replaces fieldwork in a decision that needs a real sample. When the question leaves the territory of the studies you own, it goes to the field, now with a sharper hypothesis to test. Our data on when synthetic respondents can screen concepts is in the fidelity data article.

FAQ

What is an ad hoc research request?

A question that reaches the insights team outside the planned research calendar, tied to a decision with a date on it. The AQR glossary defines ad hoc research as one-off work for a specific purpose, as opposed to continuous research.

How should an insights team prioritize ad hoc requests?

Rewrite each one as a decision with a date and check the archive first. Then ask whether the question stays inside the category and buyers your studies cover. Inside, a labeled persona hypothesis can help. Outside, it needs fieldwork.

Can synthetic personas answer questions between tracker waves?

Only as a labeled hypothesis, and only inside the category and population of the studies they were built from. In the Brand, Israeli and Ngwe paper, fine-tuning on past survey data improved alignment there and did not for new categories or segment differences.

Does fine-tuning on past survey data make a model reliable?

More reliable within the same category and population, according to the same paper, including for new features. It did not fix new categories or differences between customer segments, and the authors treat the model as a supplement to human studies.

Can a synthetic persona's answer go into the deck?

As a labeled hypothesis with its source studies named, yes. As a percentage of consumers or a verbatim quote, no, because no consumer answered.

Before the next request arrives

UNIQUE INSIGHT The gap between four waves a year and a meeting every week will not close by itself. What changes is whether the answer that fills it says where it came from. A dated page from the archive, a hypothesis labeled synthetic, or a study design are all honest answers. An old slide forwarded without its field date is the only dishonest one, and it is the most common.

Does your team still answer between-wave questions with the closest slide in the folder? Take the last five requests and sort them into the three exits. We can build the personas from the studies you already own and run the next request with you, labeled from the first line.

About the author

Cassiano Albuquerque is the founder of cassiai.com. He has spent twenty years in market research, was ESOMAR's representative in Brazil and speaks at IIEX, ESOMAR and ABEP. He now builds AI systems for research teams. To talk about answering between-wave questions with personas grounded in your own studies, contact him on LinkedIn or see the cassiai.com portfolio.

Published by Cassi.ai. Read the full article at https://www.cassiai.com/blog/ad-hoc-research-requests-synthetic.