(Editor’s note: When we find something of such significant value to our audience from another source, we want to share it. This article appeared in Chief Content Officer magazine, which excerpted it from Brian Massey’s great blog, Conversion Sciences.)
Marketers have always relied on testing. But let’s be honest; it’s probably only in the last few years that they’ve begun discussing conversion rates rather than golf scores over beers.
And yes, I’ve got the data to back that up. According to Econsultancy, in the last five years, the number of companies using A/B testing has more than doubled. Two-thirds use A/B testing, making it the top optimization method in practice today. Compare that to five years ago, when only a third of businesses were testing.In the last 5 years, the number of companies using A/B testing has more than doubled via @Econsultancy Click To Tweet
You might say it’s the golden age of conversion optimization. Cool thought, I know. And it sounds like it should benefit businesses across the board. But that’s not what we’re seeing.
Whenever any tactic becomes a “thing,” it gets adopted by newbies and wannabes. Beware. You could be paying good money for website optimization services from an agency that just learned last month how to run a test.
The truth is, conversion rate optimization (CRO) is hard. You can’t learn it in a month, and you won’t be an expert until you’ve done it for years.
Let me say that again: It takes YEARS to become a pro.
Lots of agencies are making mistakes without even knowing it because they’re so new to the game. Here are some of the mistakes I see most often:
1. Best-practices landing pages
A best practice is NOT the same thing as conversion rate optimization. PPC agencies, SEO agencies, UX, and UI people – they’re all claiming to do CRO.
But calling something CRO doesn’t make it so. Here’s what Brian Massey told me the other day:
Here at Conversion Sciences, we’ve stopped doing best-practices consulting because it’s so unreliable. Even if someone asks for it, we won’t sell it to them. As brilliant as we are, when we implement best practices, we’ll be wrong on half of them. Every audience is different. You have to test to know what’s working. Period. End of story.Every audience is different. You have to test to know what’s working. End of story. via @bmassey Click To Tweet
Agencies fall into the best-practices trap. They hear something is working on another website, and adopt it, no questions asked. If a landing page built on best practices gets the results you’re looking for, call it dumb luck.
2. Tests for the wrong things
When you rely on hearsay rather than data, it’s easy to make another mistake: Testing the wrong things.
Some agencies try to shave expenses by cutting out the data-collection tools – things like click testing, heat maps, and user-session recording tools. As a result, they don’t have the data to make smart decisions about what to test. These agencies pick something out of the blue to test instead of using analytics to figure it out.
In other words, they’re testing for the sake of testing.
Science should be based on hypotheses, not guesses or busy work. So, you should ask, if your tests aren’t based on data, what’s the point?
Alex Turnbull, Groove’s founder, gives a great example of this. He lists some tests that are often considered “easy wins” (e.g., slightly changing the listed price or changing the color of a button); but for Groove, he says, those tests were pointless.
Typically, those tests are the first tests newbies try to run, not because they’re relevant to the website or the audience, but because they seem like easy wins. Remember: Trust the data, not someone else’s results.
RECOMMENDED FOR YOU: Data-Driven Content Strategy Meets Content Marketing [Essential Template]
3. Reliance on self-reported data
Data is important. But you can’t depend on just any data. Self-reported data – such as responses from focus groups, user testing, surveys, and forum feedback – is gathered from people’s stories, not their behavior.
The problem is people lie.
They may not mean to, but they do. If you ask them how they spend their money, they give a best-case scenario or what they wish to be true. Not the absolute truth. Compare their answers to your analytics and you get another story. The real story.
That’s why we don’t put much stock into self-reported data. Qualitative data (self-reported) is great for generating hypotheses, but it needs to be validated with testing.
Here’s where you need to be careful: A lot of agencies (especially UX and UI) redesign a site using only self-reported data. A true conversion scientist uses analytics and split testing to verify assumptions before deciding they’re true.
RECOMMENDED FOR YOU: How to Uncover Critical Content Marketing Insights Using Google Analytics
4. Lack of understanding the scientific process
Agencies are time-and-materials companies. They bill by the hour. Understandably, they want every hour of their employees’ time to be accounted for and assigned to a winning project.
The problem: This focus on the bottom line can dampen results.
Scientists need time to be curious, follow their hunches, and understand the reason things are happening. A successful agency needs to give them that time, even if some of those hunches turn out to be pointless. In the long run, it’s cheaper to eliminate hypotheses early, before testing. In other words, a few hours of analysis beats two weeks of testing every time.
True inspiration requires time. Time to follow dead ends. Time to dive into the data. Time to think and ask questions. If your agency doesn’t allow that, be aware, you’re probably not getting the best results.
5. A completely done-for-you model
This one sounds more like a premium service than a mistake. But when it comes to CRO, it reads more like a mistake.
Some agencies believe they have more job security if they make the client completely dependent on them. So they do it all – collect the data, make the hypotheses, and, supposedly, deliver results. Alone. There’s no collaboration with clients, which means they’re only using half the information they should be using to create hypotheses.
Here’s the thing: The best results come when the agency and client work together. The agency has the expertise to collect the data, but the client has the intimate knowledge of the customer. It takes both. If your agency is doing everything for you, it may be creating issues rather than solving them.
6. Failure to influence the client culture
Similarly, some agencies appear to collaborate with the client, but they draw the line before influencing client culture. In reality, there’s a huge advantage to having an agency work so closely with you that it changes the way you do things.
True collaboration involves getting together on a frequent basis and discussing ideas. Over time, you begin to see the thought process that goes into each website optimization effort. You begin to understand how to make decisions based on data and to value the insight numbers can give you.
When that happens, whether you consider yourself a numbers person or not, you’re hooked. That’s the point at which you stop making random marketing decisions. Instead, you call your agency and ask what data needs collecting and when you can start testing. (Congratulations, you’re a conversion geek!)
7. No CRO staff
This is a biggie. An agency that doesn’t staff for CRO shouldn’t offer CRO. The best conversion scientists are skilled in two areas: They’re good with numbers and they’re excellent communicators.
Getting high marks in high-school math isn’t enough. Conversion scientists are masters of data and statistics. They know when numbers are reliable and when they’re not. So they know how long to run a test and when the results are statistically valid. They know when the math is bad, which means you can be sure you’re getting positive results.
Being good with numbers isn’t everything. Great conversion scientists are also excellent communicators. All too often, web developers are recruited to do analytics, and they understand the numbers … but not much else.
It takes a conversion optimizer to turn data into stories. Frankly, that’s where the magic happens.
At Conversion Sciences, the team members spend much of their time going through the numbers to tease out the stories. If there’s a hole in the plot, they design a test to figure out what’s missing. The goal is to find the story in the data — and tell that story well.
If you think about it, conversion optimizers are really fortune tellers. They predict the future based on the data your site gives them. Is your agency converting analytics to customer stories? If not, you may be dealing with web developers rather than conversion optimizers.
8. No test before going live
Professionals test and validate everything before going live. That avoids costly mistakes like Finish Line’s 2012 web redesign, which cost the chain around $3.5 million in sales and a huge hit to its reputation. Four days before Black Friday, Finish Line launched a freshly redesigned website, supposedly planning to “reinvent the shopping experience.” Instead, customers complained about lost orders and other technology glitches, and Finish Line had to revert to the old design just prior to the holiday shopping season.
Granted, that level of mistake isn’t likely for smaller brands, but bad usability can still impact reputation and profitability.
My guess is that a brand agency was responsible for that redesign. It would have been smarter to work with conversion optimizers who understand how to use data to decide on incremental changes, validating each one before moving on to the next.
9. Rookie mistakes
Since CRO is now a “thing,” everyone and their office cat offer website optimization services. Most don’t know the difference between conversions and sales, which means they’re making a lot of mistakes.
Now don’t get me wrong. We all make our fair share of junior mistakes when we’re starting out – things like delivering results without statistical validity, not analyzing traffic enough, and the like.
But this is the golden age of conversion optimization. Don’t you want pro results?
Again, not everyone who claims to be a conversion optimizer is. Unless your team is experienced and has a structured approach for improving conversions, it is likely making some mistakes that could be easily avoided — if the team was more experienced.
Bottom line: Mistakes are more common than not. That’s because website optimization is hard work. If you want to get the big results CRO promises, you need an agency that has the experience and know-how to do it right. Period.
Red flags in optimization
As Brian, founder of Conversion Sciences, said earlier, the firm no longer offers best-practices consulting because best practices can be unreliable. For example, a best practice says videos are good and sliders or carousels are bad on the home page. Sliders distract visitors, are hard to read, yada, yada, yada. Not so, according to a DeviceMagic case study, published by VWO.
The DeviceMagic test pitted a slider against a video to see which would work better on the home page. The team was pretty sure sliders were a better fit for their purposes, but it knew better than to make the change without testing. The verdict? The much-maligned slider was the clear winner. Conversions from home page to sign-up page increased 35% and subsequent sign-ups increased 31%.
Costing £150 million ($226 million), the U.K.-based major retailer Marks & Spencer website redesign took two years and led to an 8% decrease in online sales. Based on the fact that the project took two years, it’s a fair guess that most decisions were made from self-reported data plus the design team’s own opinions. It’s unlikely A/B testing was involved because testing delivers incremental changes rather than one massive change.
DigitalTonic says it well in its analysis of the failure: “Drastic changes can never be monitored meaningfully and you won’t be able to separate the variables that are causing the positive or negative impact on conversions. With testing on your current site prior to redesign, you will hit a local maxima meaning that you have optimized the site as best as you can in its current incarnation. It’s at this stage that you would take the learnings and move towards the global maxima with the redesign process.”
This article originally appeared in the February issue of Chief Content Officer. Sign up to receive your free subscription to our bimonthly, print magazine.
Cover image by Joseph Kalinowski/Chief Content Officer
Comments