Fuel price testing — let's talk about control

In this series of four blogs, we’ll be delving into the logic of price testing to help you understand how to run an effective test that boosts your fuel pricing strategy.

For many fuel retailers who’ve always held the same price position, trying something new is daunting. Margins are typically tight, so mistakes can be costly. And so, although many would agree that price testing is a good, forward-thinking approach, it’s often avoided, with Pricing Managers choosing to continue to trust in the position they’ve always held.

But price testing doesn’t have to be risky.

Kalibrate views price testing as trial and control ― not trial and error.

You won’t always get it right, but if you manage a price test in a controlled manner and measure it well, you’ll know what works and what doesn’t so you can always make informed decisions.

The following advice will help you to set stringent controls around any fuel price test that you’re considering. Implementing tight controls will help you to minimize risk.

Control your test site selection

To provide statistical viability in your test, only test one change at once, and always select sites that are stable. If a site has gone through any significant change recently, including changes to your own facilities, at local competitor sites, or in the external environment, it will not be a good candidate for a price test.

You need to have a reliable base to measure against. Ideally, you want just one variable from the entire ecosystem of your site to change — so you can pinpoint whether fluctuations in volumes or margins are truly the result of your test.

Create a control group

Without a control group, you’ll be able to tell if your performance has increased or decreased compared to where you started, but you won’t know if you’ve performed better or worse than if you hadn’t run the test at all. That’s where a control group comes in. In order to reach a dependable conclusion, you need a control group to measure against.

Pairing a test site with a control site will help you to measure whether performance changes are the result of your price test or other environmental factors.

So how do you select sites for a control group?

You need to select pairs of sites that are as similar as possible in the following areas

• Number of competitors
• Market behavior
• Volume
• Margin
• Daily trends

But they need to be far enough apart geographically so that a price change at “site A” won’t impact performance at “site B”. Once you have a suitably similar pair you put one in the control group and one in the test group.

You’ll need enough pairs of sites to allow statistically significant results. When running tests for clients, Kalibrate typically uses 30 sites in each group. Kalibrate’s Global VP for Customer Success, Beth Thompson, explains;

“We tend to select 30 sets of pairs for the price tests we run. That’s because over a 12-week period some sites, and their paired site, will have to be excluded. If at some point during the 12-week test period there’s a stock shortage, facilities get damaged, or there are roadworks in the local area, the performance of the site will be altered — but not as a result of the price test. So this site, and it’s partner in the control group, should be excluded from the test so as not to skew the results.”

This doesn’t mean you can’t run tests if you’re a smaller operator. But you need to ensure your test sites are statistically viable. If you have 10 sites and run a test at two sites, you can presume that the results are a good representation of the rest of your group. But if you have 100 sites and test on two, it’s unlikely that your results will be applicable across your whole network.

Use the “12-week rule”

The testing process can be an anxious time for Pricing Managers and there’s always an urge to react based on an immediate change. But a price test needs sufficient time before a conclusion can be drawn.

You’re likely to see a change as early as the first two weeks, particularly if you’ve made a significant strategy change, but making decisions on that two-week change could be detrimental. Kalibrate’s Client Success Consultant, Elizabeth Kershaw, suggests that 12 weeks is the optimum time for a price test.

“12 weeks is the perfect period as it gives enough time for consumer perception to be altered and for your competition to notice and respond to your change in price position. Any shorter and you won’t really know if your results are sustainable.

“We regularly run price tests with our clients, as well as pricing trials for prospective clients and always stick to the ‘12-week rule’ — it’s the quickest way to reliable results.”

Create an exit strategy

Remembering the logic of the 12-week rule, the immediate reaction from consumers and competitors is are not always consistent over time — so, set an exit strategy and stick to it. Exit strategies are critical, not only to protect the business in the case of a poor market response, but also to agree the criteria that allows the test to continue.

Before you start the price test, define what your exit criteria should be. This prevents you “losing your nerve” should you see the needle going in the wrong direction, especially at first, and to help weather calls from other parts of the organization who may be concerned by loss of volume. You should typically try to work out, based on your estimated unit margin gain/loss, what your breakeven volumes should be to remain margin neutral.

Use this to set your “worst case scenario” volume cut-offs. Calculate a margin cut-off too, to ensure you prevent a downward pricing spiral with your competitors. Rather than setting an absolute margin cut off, consider using a differential to your network average margin or nearby sites, to take margin conditions into account.

Download “The logic of price testing” white paper for more detail, or contact Kalibrate for advice and assistance in setting up suitable controls for your price test.