Virtual Shelf Testing: What It Measures and What It Inflates

Virtual shelf testing gets closer to the real store than a survey on where shoppers buy from the shelf, and still inflates how much they buy. How to read it.

A category team wants to change the shelf. The brand moves up from the bottom shelf, two slow lines leave, and a new flavor goes in as a block beside the leader. Before the retailer resets a single store, somebody has to say what shoppers will do. Will they find the product, pick it up and keep it in the basket? Virtual shelf testing is one way to answer before the reset. It answers some parts of the question far better than others.

Key Takeaways

• In a Wageningen study in Appetite, 2016, 21.6 percent of milk and biscuit purchases came from the top shelves in a real store, against 33.4 percent in a virtual store and 50.0 percent with pictures.
• The same study found that virtual store shoppers still bought more, chose more national brands and reacted more strongly to price promotions than shoppers in the real store.
• Stated purchase intentions carry systematic biases that do not cancel out in aggregate, per Sun and Morwitz, 2010.
• Eye tracking at the shelf shows that attention does not always become choice. Top and middle shelves gained attention, and only the top shelf carried through to brand evaluation.
• Read a virtual shelf as a comparison between layouts. Do not read it as a forecast of volume or promotion lift.

What a planogram decision actually involves

A planogram is the diagram that fixes where each product sits on a shelf. It sets which shelf, how far left or right, how many facings and next to what. A facing is one unit of a product turned toward the shopper, so three facings means the pack appears three times side by side. Changing either one changes what a shopper sees in the few seconds spent at the fixture.

The decisions that follow from a planogram are concrete. Move a brand to eye level or leave it low. Cut a line to make room. Block a range by brand or by flavor. Decide what a shopper buys when the usual item is missing.

The academic evidence says position matters, and in a specific way. Chandon, Hutchinson, Bradlow and Young ran an eye-tracking experiment later published in the Journal of Marketing. The number of facings had a strong effect on brand evaluation, entirely through its effect on visual attention. Top and middle shelf positions gained more attention than low ones, and only the top shelf carried through to evaluation. Their summary line is the one a category team should keep: gaining attention at the shelf is not always sufficient to drive sales.

How the shelf question is answered today

PERSONAL EXPERIENCE In my experience of consumer goods and retail projects, the shelf question usually gets one of three answers. Teams still lean on the first two.

1. Ask in a survey which product the respondent would buy, sometimes with a picture of the shelf.
2. Take the decision in the category review, on the judgment of the most senior people in the room.
3. Negotiate a test with the retailer, reset the shelf in a handful of stores, wait for the sales read and compare with control stores.

The first is fast and measures what people say. The second is faster and measures what executives believe. The third measures what shoppers do, and it needs store access, a retailer willing to reset the fixture, and weeks before a read is clean.

Why asking is the weakest of the three

A purchase intention question measures a statement about a future choice, made away from the shelf. Marketing science has studied that gap for decades.

Sun and Morwitz, in the International Journal of Research in Marketing in 2010, put it in their first lines. Intentions data often contain systematic biases, intentions change over time, and they may not accurately predict actual purchases. Their introduction adds the point that matters for a planogram. Individual discrepancies do not cancel out in the aggregate, so average stated intention and the share who actually buy drift apart.

The executive meeting has the same problem in a different form. The judgment is real expertise, and it is still a statement about shoppers made away from the shelf. A brand only gets one launch on a fixture. When the reset goes wrong, the category manager who took the planogram to the retailer answers for it, and the retailer remembers.

What a virtual store gets right, and what it inflates

A virtual store is a simulated shop, on a screen or in a headset, where participants walk an aisle, look at products, fill a basket and check out. The software logs what they looked at, what they picked and how long each step took. Virtual shelf testing is the use of that store to compare shelf layouts before a physical reset.

The most useful evidence on how close it gets comes from a study that ran the same shopping task three ways. Van Herpen, van den Broek, van Trijp and Yu, at Wageningen University, published it in Appetite in 2016. Shoppers bought fruit and vegetables, milk and biscuits in a physical store, in a virtual store, or from pictures of the shelves.

The figures are the study's, repeated here so they can be read without the chart. In the physical store, 21.6 percent of milk and biscuit purchases came from the top shelves. The virtual store gave 33.4 percent and the pictures gave 50.0 percent. Purchases from the left side of the biscuit shelf went from 35.5 percent in the store to 53.3 percent in the virtual store and 66.7 percent with pictures. The average number of milk products selected went from 1.19 in the store to 1.53 in the virtual store and 2.58 with pictures.

The same shopping task, three ways

Three panels from a study by van Herpen, van den Broek, van Trijp and Yu, published in Appetite in 2016, comparing the same shopping task in a physical store, in a virtual store and with pictures of the shelf. Panel one, purchases taken from the top shelves for milk and biscuits: 21.6 percent in the physical store, 33.4 percent in the virtual store, 50.0 percent with pictures. Panel two, purchases taken from the left side of the biscuit shelf: 35.5 percent in the physical store, 53.3 percent in the virtual store, 66.7 percent with pictures. Panel three, average number of milk products selected: 1.19 in the physical store, 1.53 in the virtual store, 2.58 with pictures. In every panel the virtual store sits between the physical store and the pictures. A closing line says the virtual store moved shelf behavior toward the store and did not stop people buying more. The figures come from the published abstract and none of them is a Cassi.ai measurement.

In all three panels the virtual store lands between the real store and the pictures. Closer is not the same as equal.

The authors read that as virtual reality improving realism in responses to shelf allocation. They were just as clear about the limit. The virtual store did not remove the other lab effects. Participants in both the virtual and the picture conditions bought more products and spent more money on biscuits and produce. They also chose more national brands and reacted more strongly to price promotions than shoppers in the real store.

That is the whole method in two sentences. A virtual store moves shelf behavior a long way toward the store, compared with pictures. It does not make people shop with their own money and their own week in mind.

What each method measures

UNIQUE INSIGHT The table sets four ways of reading a shelf side by side, by what each one can and cannot tell a category team.

Sources: van Herpen and colleagues, Appetite, 2016 for pictures, virtual and physical store; Sun and Morwitz, 2010 for stated intentions. The last column and the reading across rows are ours.

How to run a shelf test you can defend

UNIQUE INSIGHT The test setup follows from the evidence. Use the virtual store for what it measures well, which is relative behavior between layouts, and keep everything else out of the conclusion.

1. Write the decision first, in one sentence: move the brand to eye level, delist two lines, or block by flavor. The same discipline that makes a dashboard start from the decision keeps a shelf test honest.
2. Build the current shelf and the proposed one with the same products, prices and facings. Change only what the decision changes.
3. Give each participant a shopping task with a budget, so the basket has a reason to stop, and split participants between the two shelves.
4. Add a mission with the usual item out of stock, because substitution is where a planogram wins or loses shoppers.
5. Compare the two shelves on where purchases came from, time to find the product and what replaced a missing item.
6. Report the difference between shelves and leave the absolute basket size out of the conclusion.
7. Confirm the winning layout in stores when the decision is large enough to need a sales number.

How to read a virtual shelf test

Two columns. The left column, read it as a comparison of: where on the shelf people buy, top, middle or bottom, left or right, current layout against proposed; time to find the product, meaning how long the shopper stays at the fixture before choosing; what replaces a missing item, shown by the out-of-stock mission; and attention against choice, meaning products that are looked at and not bought. The right column, do not read it as a forecast of: how much people buy, since virtual shoppers bought more products than real ones; how much they spend, since spend was higher for biscuits and produce; promotion lift, since reaction to price promotions was stronger than in store; and brand share, since virtual shoppers chose more national brands. The right column comes from van Herpen and colleagues, Appetite, 2016, comparing a virtual store with a physical store; the left column is Cassi.ai's reading of what that evidence supports. A closing band says that when the decision needs a sales number, the winning layout goes to a store test.

The left column is what a category team needs to choose a layout. The right column is what still needs a store.

Specialist vendors already sell this. One case published on Greenbook this year describes controlled virtual and in-store experiments used to validate planogram and fixture changes before roll-out. The growth figures in that case are the vendor's own. They are the kind of claim to put through the questions we would ask any research AI vendor: measured against what, in which stores, by whom.

Where Cassi.ai comes in

cassiai.com is a software engineering company specialized in the pains of market research, innovation and insights, founded by Cassiano Albuquerque after twenty years in research. We work with research agencies, consumer goods brands and retail insights teams.

Shopper Insight Lab is a Cassi.ai product that simulates purchases on a virtual shelf: participants shop an aisle with a cart and a budget, and the dashboard shows attention, choices and trade-offs.

Shopper Insight Lab is our virtual aisle. The participant picks an avatar, gets a budget and works through three missions: open choice, a reduced budget, and the usual item out of stock. The shelf records time at the fixture, how long the cursor stays on each product, and what goes into the cart and comes back out. The team gets a shelf heat map, a reading of attention against choice by product, and the most frequent substitutions. A session takes five to eight minutes in a desktop browser.

Two limits come with that design, and we state them before anyone asks. Attention here is measured by cursor and time on a desktop screen, which is a proxy and not eye tracking. And a desktop store is closer to the desktop virtual store protocol of Ploydanai and colleagues than to the immersive setup in the 2016 study. Its numbers are a comparison between shelves, never a sales forecast. The out-of-stock mission is the part we would put first in front of a category team. It shows which product takes the brand's place when the brand is missing, which no survey question can show.

For the concept stage before any shelf exists, our article on synthetic respondents in concept testing covers the screening step.

FAQ

What is virtual shelf testing?

The use of a simulated store, on screen or in a headset, to compare shelf layouts before a physical reset. Participants shop a virtual aisle while the software records what they look at, pick and buy. Ploydanai and colleagues published a protocol for running a desktop virtual store as a research tool.

Is virtual shelf testing accurate compared with a real store?

On where shoppers buy from the shelf, much closer than pictures. In van Herpen and colleagues, top-shelf purchases were 21.6 percent in store, 33.4 percent in the virtual store and 50.0 percent with pictures.

What does a virtual store overstate?

How much people buy and how they respond to price. In the same 2016 study, virtual store participants bought more and spent more on some categories. They also chose more national brands and reacted more strongly to promotions than shoppers in the physical store.

When should you still run an in-store test?

When the decision needs a sales number, such as a forecast of volume, a promotion lift or a case for the retailer's buyer. A virtual test can choose which layout goes to the store test.

How is a virtual shelf test different from asking purchase intent?

A survey records a stated choice made away from the shelf, and stated intentions carry systematic biases that do not cancel out in aggregate. A virtual shelf records a choice made at a shelf, under a budget, with the product next to its competitors.

A virtual shelf answers where shoppers will look and buy on the new layout compared with the old one. It does not answer how much they will buy, and a test that keeps those two questions apart is one a category team can take to the retailer.

Is your next planogram change being decided on a survey question or in a category review? Build the current shelf and the proposed one, and look at the out-of-stock mission before anyone signs.

About the author

Cassiano Albuquerque is the founder of cassiai.com. He has spent twenty years in market research operations, from fieldwork and tabulation to the reports that reach steering committees, and now builds AI systems and research software for insights teams. To see a category shelf built in Shopper Insight Lab, contact him on LinkedIn or see the cassiai.com portfolio.

Published by Cassi.ai. Read the full article at https://www.cassiai.com/blog/virtual-shelf-testing.