How To Use Heuristics To Win at CRO: A Mumsnet Casestudy

Often as a new CRO program commences there is a desire to get a test winner out of the door fast – to get everyone excited about the program and start proving its value to the business quickly.

CRO can, and should be a fast moving program. Insights in, learnings out. Rinse and repeat.
So showing off how quickly CRO can move and deliver learnings and value back to the business at the beginning of a program is only ever a good thing.

A full analytics deep dive and hypothesis formation to build a CRO roadmap is a significant piece of work. So launching the first test whilst this is taking place is a good way to keep the momentum up and show off the velocity benefits of CRO to the business. This challenge can be navigated with the correct use of heuristic analysis.

So First of All, What Exactly is Heuristic Analysis?

‘Heuristic analysis is an expert based analysis that uses experience-based techniques for problem solving, learning, and discovery. Its results are not guaranteed to be optimal.’ Conversion XL.

The final sentence is key ‘It’s results are not guaranteed to be optimal’ and this is something we have written about before in 3 guaranteed (or not) tricks to help you win at CRO. What has worked before for one site won’t necessarily work for the next site and there really are no guaranteed ways to win based on heuristics alone. Bringing together a collection of insights from a suite of analytics tools is the secret for guaranteeing success in CRO.

But sometimes heuristics may be all we have access to. We were recently briefed by Mumsnet, the UK’s biggest network for parents to help improve how product review pages work for Mumsnet users.

We had one month to prove the value of CRO – we had analytics access but were not in a position to get heatmaps and session recordings data as well as budgets and timescales didn’t allow for a user testing study. So we were left with a heavy reliance on heuristics.

Applying Your Heuristics Framework

Using a methodical heuristics framework is important. Below is the high level framework we use:

• What if any elements are causing friction to conversion?
• How is the clarity of the page? Is the content high quality? Are the next steps to conversion highly visible and clear?
• Is there anything on the page that is a distraction from conversion?
• Is there a sense of urgency to convert?
• What do we know about the audience on this website?
• What have we seen work for other clients that this site could benefit from?

When applying this framework to the Mumsnet product reviews pages where the objective was to improve usefulness by making it easier for users to read the content and where relevant, click through to the retailer site that sells a product they are interested in, we formed two key observations and hypotheses:

Heuristic Analysis 1

How is the clarity of the page? Is the content high quality? Are the next steps to conversion highly visible and clear?

Observation: The CTA to click through to the retailer site is only accessible at the bottom of the page. It is not visible or accessible across the full depth of the page.

Hypothesis: Using a sticky CTA that makes the link through to the retailer site accessible across the whole page will improve visibility of this journey and increase traffic to retailer sites.

Control

Variation

Heuristic Analysis 2

How is the clarity of the page? Is the content high quality? Are the next steps to conversion highly visible and clear?

Observation: The list of the top products was text heavy and the links through to the retailer sites were getting lost

Hypothesis: Structuring the top products in a table with a greater level of spacing between each product and showing the retailer links in a bold purple font would increase the visual clarity of the retailer links, improving the visibility of this journey and increasing the click through rate.

Control

Variation

Results From Heuristic Analysis

The variation delivered a 20% increase in click through rate to the retailer sites validating that the variations improved the product reviews experience for users. 5% of traffic clicked the new sticky CTA and the re-designed top products component delivered a 30% increase in clicks compared against the control. This was a very simple test based on some basic heuristic observations of the page.

Heuristic analysis is a great tool for delivering fast test plans when access to the full analytics suite is limited or in the process of being set up. Using heuristics is risky and wins can’t be guaranteed but using a solid framework for your heuristics will maximise your chances of success. Getting a winning heuristic based test out of the door in the first month of a CRO program can be an invaluable way to show off the velocity of learnings a CRO program can deliver and get the business excited and engaged.