Conversion Testing Innovation

Impact of TV imagery on website conversion rates

It’s an unwritten law: spend big £/$ on a TV campaign and your website traffic will go through the roof.

Your overall cost per visitor will naturally follow suit. So how do you get more out of the traffic you’re buying in?

The answer is pretty simple – conversion rate optimisation. Test and tweak your pages and processes until you can get more people doing much more of the things you want them to do.

A case study in conversion rate optimisation

The company I work for – – is into its 2nd year of TV advertising. The traffic levels during the campaigns have been incredible (see graph below for search query peaks during two TV bursts).

UK search trends for ‘jobsite’ – Google Trends

Whilst looking at the data, I wondered what the impact on conversion rates would be if we used the same imagery from the TV ad on the site’s homepage search box. Would it encourage more searches?

As we were about to run another burst of TV advertising in January 2010, we had a prime opportunity to do some conversion testing to find out.

If you’ve worked on a brand’s website during its TV campaign you’ll be familiar with the following traffic pattern: a tremendous surge during the ad period, followed by a gradual, not sudden, drop during the subsequent weeks – known as the ‘Halo Effect’. In our case, TV ads aired throughout January, with the halo effect running into February.

The conversion test

I wanted to test 3 things:

  1. What impact would TV imagery have during the TV period?
  2. What impact would it have during the halo?
  3. What impact would it have during a ‘normal’ month?

When testing, you need to identify which conversion metric you are looking to improve. Given the nature of our site, my preference tends to be job applications or CV (resume) uploads. However, in this instance, I felt there were too many variables that could influence either metric (particularly the fact that jobseekers often leave the website to prepare their application before returning in a separate session to apply). So for this test I chose to track on site job searches.

Could switching the search box imagery from the default stock photo of people over to the TV image (featuring British actor, Max Beesley) actually increase the number of visitors that become searchers?

Rather than a simple A/B test (Max vs the People), we ran multivariate tests, featuring variations of the search box heading, the search field label and the imagery. This would give us insight into which element(s) improved the number of searches.

Results during TV

As expected, the winning page combination during the January TV campaign was a version that featured Max Beesley. Based on the visitor figures for the month, running this version on 100% of visits would have resulted in an additional 11,328 job searches.

The % uplift over the default version was not dramatic – a 1.3% increase – but sufficient enough to be noticeable when you’re dealing with a highly trafficked website.

What was clear with this test, however, was that the heading ‘Find a Job’ outperformed the default heading ‘Job Search‘, with an 87% chance of beating the original headline (on the default version).

Into the Halo Effect

Things began to change in February with the ‘halo effect’. Traffic to the site was still high, TV had stopped but other digital brand advertising (PPC, Mobile pre-rolls) continued.

The same Max variant continued to outperform the original, but its impact had begun to decline (down to +1.0% over the original combination). In fact, it had slipped into 2nd place, behind a variant using the original stock library image (+1.18% over the original combo). Based on February traffic figures this new lead variant would have generated an additional 8,181 searches if the creative had run for 100% of visits.

The impetus for this performance increase was the heading ‘Find a Job’ (+1.09% over the original heading) – the call to action was more of a factor than the choice of image at this stage.

Interestingly, we also saw a rise in the performance of a 2nd heading – ‘Search Jobs’ – amongst the successful variants (+0.93% over the original)

Under ‘normal’ conditions

As we moved into March, there was enough distance from the high profile ‘Above the Line’ activity to consider it a ‘normal’ environment.

Of the 19 variants we were testing, only 3 outperformed the original – and just barely. All three made use of the stock library imagery. The best performing variant was almost identical to the original, with the only difference being a heading of ‘Search Jobs’ instead of ‘Job Search’ (a simple switch of word order creating a call to action instead of a description).

This variant increased searches by 0.32% over the original – extrapolated to 2,338 extra searches in March.

When you look at the individual components’ performance, ‘Find a Job’ was still marginally ahead of ‘Search Jobs’ (+0.54% to +0.52% improvement on the original), but in combination with the other elements, the latter featured in the best performing variant throughout March.

Interestingly, if we’d used the best performing TV imagery variant in March, it would have resulted in 1,169 FEWER searches than the original.

Learnings from conversion testing

Overall, we can conclude from this test that the use of TV imagery within your search box DOES improve your conversion ratio of visitor to searcher during your campaign. However, it should be coupled with a strong call to action to maximise its impact. In this case study, it provided the possibility of 22,000+ additional searches over the 3 months.

So what else can we take from this test?

In general terms, I would recommend the following tips:

  • Continuous testing is important – external influencing factors, e.g. advertising campaigns, news, events and seasonality, can impact the conversion performance of your site. Test and change your design accordingly
  • Give consideration to your use of imagery – its not there just for aesthetics. It can be a powerful aid to conversion
  • Use clear Calls to Action in your copy, especially titles and/or buttons. Stick to a single call to action in a creative to avoid mixed messages

In regards to Jobsite, we’ll be returning to TV again next month (May ’10). My specific recommendations are:

  • Use TV imagery and ‘Find a Job’ title throughout the broadcast period
  • Continue with same creative for first fortnight of halo effect period
  • Switch back to original creative thereafter but with a call to action in the title – either ‘Search Jobs’ or ‘Find a Job
  • Continue testing – introduce new variant options, such as alternative imagery, text, buttons or even background color.

As with all conversion testing, this isn’t a blue print for success with your own website. You’ll have your own set of external factors to contend with, your own site design and product offering to influence results. Set up your own optimisation tests and see how you can improve your own conversion rates.

I’d love to know about your successes with conversion rating optimisation – what has been your most interesting or successful conversion test? Anyone done any conversion testing with websites & TV with a story to tell? Please do share your comments or experiences.


Sticky Fingers: Our Children’s Technological Future

I have sticky finger prints smeared across my flat screen TV.

They’re not mine, I might add. Rather they belong to my two year old son. Unlike his other random markings around the house (walls are a particular favourite), these are deliberate and with purpose.

He tries to operate the TV like he would my iPhone. That is, he competently swipes and pushes objects (buttons) on the flat screen as a means of navigating to the content he wants.

When the TV doesn’t respond he gets annoyed and airs his displeasure. For instance, a favourite programme of his starts and ends with the characters entering a lift (elevator) – the top floor housing a fun play room. One day, as the programme was coming to an end and the doors to the lift closed on the departing characters, he began to furiously ‘swipe’ the doors on the screen, yelling “no, no, no!” – desperately trying to open the doors so the programme wouldn’t end.

But end it did, and in tears.

It struck me watching this, that he’s two years old and already he’s frustrated with the speed of technology advancement. To him, there is no reason why a TV shouldn’t work in the same way as Daddy’s iPhone.

Which makes me wonder, how fast will technology evolve in just the first couple of decades of his life?

Flash back

When he’s old enough to understand, he’ll probably laugh when I tell him about the technology I had when I was growing up.

Two main broadcast technologies, TV and Radio – three channels on the former and mainly national stations on the latter. As a kid, the height of cool was owning a twin deck radio cassette player (for recording the Top 40 Charts on Sundays). And…, well, that was about it for the early years.

Then came the technology revolution, as the (top-loading, not front loading) video player burst on to the scene to change the entertainment world forever. Not that our family was an early mover on this one. We were well into the Late Majority before I discovered Vader was actually Luke’s father.

Consumer-owned computers appeared in my teens in the 90s but I barely touched them until I left college (mid 90s). Then of course technology exploded again, as we marched in the new millennium, with the next wave of game changers – the iPod and iPhone.

The odd thing for him is that these new technologies are all standard devices. They are so everyday in our house that he doesn’t understand why the other (older) shiny things don’t work in the same way.

Flash forward

Being fortunate (unfortunate?) to have a parent working in technology, there is a good chance that he’ll adopt emerging technologies and activities quite quickly. From a school perspective, it’ll be interesting to see how it develops.

I still remember the excitement of being allowed to use a calculator in class for the first time, I can’t imagine being able to open up a browser to access the web via the school WiFi.

Much will depend on education funding, but isn’t it likely that within just a few years all school children will be working from laptops or iPad-like devices in class, rather than with books and pens?

Much of this technology is already available, but what else is to come? How about:

  • User Generated Content Story books, placing the child within the (e)book (viewed on a Kindle-esque device, naturally)
  • Desks and ‘blackboards’ using Minority Report style interfaces?
  • 3D experiences of faraway places, visiting the Pyramids or back in time to ‘witness’ historical events?
  • Virtual classrooms, with remote teachers delivering lessons by teachercam or holographic projection?
  • Collaborative projects using wiki-based platforms?
  • Start-ups formed as part of the Business studies or Economics curriculum (with Intellectual property rights shared with the school, of course)

All of these things are achievable within this next decade. I can’t even comprehend what it will be like by the time he reaches his teens.

I have no doubt children of his generation will readily adopt any new technology and application that emerges – perhaps the biggest question will be how will we keep up?

Business Performance Innovation

Innovate for your brand’s survival

Let’s be honest, you can milk your Cash Cow for quite some time. However, like oil, one day it’s going to run out. Then what?

Well, by that point it’s too late. One, or several, of those pesky start-ups who launch good looking websites with shiny, multi-featured products, will have figured out how to monetise their passion and will be waving at you in slow-mo as they pass you by.

You’ll be left to wonder what happened and at some point you’ll look at your new market leaders and will lament “we could have done that”.

Ideas needed – apply within

So, you’ve seen the future and you don’t like it. What are you going to do?

Firstly, look to your customers. And particularly to those who aren’t your customers. Don’t start building anything until you understand what they need. Not necessarily what they say they want – or what you want – but what they need. There is a big difference. Answering a need secures a customer for the long term, answering a want will have you following fads and wasting resources.

Secondly, embrace the creativity in your workplace. Great ideas are not limited to the executive suite. The Support Assistant on the phone speaking to the irate customer may have the greatest insight of all in your business. A cliché it may be, but employees are the business’ greatest asset. Tap into it. Treat your employees like citizens, let them join the cause.

This is a cultural thing. Much like your brand in the social sphere, is your business happy to relinquish ‘control’ to the ‘masses’?

Consider it. It can produce exciting results.

A case in point

At the 2010 Onrec Awards for the online recruitment industry, won in two categories – one of which was for its candidate services. Amongst the products included under that banner, were Jobs-by-Twitter, BeMyInterviewer and RecruitRank.

Jobs-by-Twitter is an API integration between Jobsite and the micro-blogging platform. It was created as an experiment to understand how we can reduce the searching workload of jobseekers by delivering relevant jobs to a platform they were already using daily. It was also a direct response against the torrent of untargeted job tweets gushing into the Twitter sphere

BeMyInterviewer is an interactive interview practice service that utilises video to enable jobseekers to rehearse with top industry professionals, including the likes of Dragons’ Den mogul Duncan Bannatyne.

RecruitRank is a jobseeker feedback system, enabling applicants to rate recruitment agencies for the customer service they received. It came in response to research that revealed over half of jobseekers find the process of job hunting frustrating and demoralising.

All three products came to life through the creativity of Jobsite employees – be it initial concept or enabling the idea to flourish into a fully featured – and useful – product for our customers.

It was possible because the culture promotes such creativity – to step beyond the paint-by-numbers approach that has littered the industry with commodity products.

Be a market leader

Seth Godin, in his book ‘Linchpin’, defines a Hierarchy of Value as follows:

Lift, Hunt, Grow, Produce, Sell, Connect, Create

For your company to be (or continue to be) successful it will need to deliver at multiple levels across the hierarchy. However, to lead your market into the future you need to excel at creation, at innovation.

Take a look at what you offer your customers. How is that different to what your competitors provide? Now find out what your customers actually want and throw some grey matter at finding solutions. It’s crowd-sourcing on a company level. Get everyone in a room – physically or figuratively – and throw away the job titles. Leave them at the door, they’re not needed here.

Now start creating.