Business Performance Conversion Testing

Challenging your routines for performance optimisation

When we do something for long enough, we start to find ourselves falling into routines. It’s how we get through the day, the week, life, without having to over-analyse everything. The problem is those routines aren’t always for the best. We take shortcuts, connect dots where they shouldn’t be connected and hold on to conventions.

We believe what we’re doing is right. It only backfires when we’re completely wrong. Most of the time we don’t really notice, as the consequences are either too small or we don’t know what we’re missing.

In business though, it can make the difference being bad, good or even great at your job. You can continue to move along being average, getting average results for your average company or you can stop and question what you’re doing.

Let me give you an example. One of the websites I work on has a daily email process that has traditionally taken a long time to run due to the volume of recipients. It’s a core process, with a big return for the business. We’re pleased with the contribution it provides. We accept that it’s going to take all day to run, but that’s fine as long it gets sent to the customer.

There are a number of big assumptions in play here: 1) that’s how long it has to take to get through that volume 2) the business impact is optimal 3) daily is the important time factor for the customer.

Then one day, we upgraded the hardware running our core email process. Overnight, a 12 hour process was reduced to 4 hours and all emails are now delivered before 9 am instead of 4 pm.

Take a look at what happened to the traffic (split by hour of the day)…

Why was the behaviour so dramatically different?

Well, in a moment of serendipity, the hardware upgrade occurred at about the same time new customer research landed on my desk that revealed distinct time of day differences in user needs and behaviour. To quote a few lines…

Before work: This is the most common time to check email alerts. Many jobseekers rely solely on alerts to learn of opportunities, saving them having to check the source repeatedly. Reviewing alerts often takes place on the move using mobile or tablet devices to bookmark interesting roles and positions for further research at a later time.

So our acceptance of the previous process – the routine we had established – had in fact throttled user behaviour. Customers had wanted their email early, before work, and we’d only fulfilled that for a minority. Removing this limitation meant we could deliver on their needs and they responded eagerly.

This was fantastic news for the business, but I’d have to admit, it was a stroke of luck rather than genius. We made a change from a backend technical need and we were fortunate to see a positive business performance uplift. With the user research to hand we were able to understand the reason behind the performance change. But it could so easily have been very different – what if we’d set the process to run at the end of the day instead? It also made us realise that we’d been sitting on a missed opportunity.

That said, this was actually a very positive experience, as we got to learn some valuable lessons that influence our thinking and approach going forward:

  1. Just because something is done one way it doesn’t mean it’s the best way or that it will continue to be the best way. Things change.
  2. Don’t assume you know what’s best for customers. It’s important to keep searching for insights. Ask them what they want.
  3. Bear in mind that what people say and what they want isn’t always the same thing. You need to verify that the actions match the words. You can mitigate the risk of change by running A/B or multi-variant tests – show a sample of your audience the new version and everyone else gets the default, existing version. Opinions lose out to results.

Those might seem rather simplistic – and perhaps obvious – suggestions, but can you honestly say you’re talking to your customers often enough or testing your changes to see if they’re actually improving your offering?

We’re all pushed for time, so going with what we believe is right or maintaining our routine is often seen as the quickest and easiest way of getting things done. But so much of this is a mind-set rather than reality. We need to change the way we think. Research and testing need to inform our decisions, not routines and opinion.

If you want to give your routines and assumptions a rude awakening, get yourself along to the Conversion Conference in London on 27-28th November. The past two years it has proven to be the most valuable conference I’ve attended for challenging my views on business performance. You’ll come away exhausted, but full of questions and ideas.

You’ll look even more of a genius in the office if you use this discount code – ITSDIGITALMARKETING2012 – when booking to get a 15% discount on your tickets.

Leave a Reply

Your email address will not be published. Required fields are marked *