Ahhhh, metrics and analytics. For those of us that aren’t deeply involved in conversion (cough cough making money), we resort to labels for the “numbers people.” Some call them analysts, technical folks or SEO / analytics gurus and tend to think that this isn’t our job as marketers.
Friends, you are mistaken. Marketing without data is like diving into a pool with no water – it ain’t pretty. Gone are the days of creating messaging with a finger in the wind or creating ideas in your fancy ivory tower.
Good marketers cater to potential customers by learning. What works? What do they want to hear? What do they do with your product? Where do they go on your site? When do they buy? How do they buy? Where do they buy?
If you can’t answer a few of those questions, do a Google search for your local unemployment office and save that in your GPS.
An A/B Testing Case Study
At Phonebooth, we launched a new website late last month after months of research, analysis, content creation, planning, stakeholder meetings and other things you do to try to make sure everyone is on board. One major element of our redesign was to become masters of testing. We hadn’t entirely neglected it in the past, but decided to make testing a focus.
Our first test was the main hero image (an elegant title for the stuff below the top nav / header). We had research telling us that customers buy our business phone solution because of value. Saving money makes people happy and we had data to confirm that.
We established five different messages with similar styles, colors, fonts, etc. The main variance was the actual copy.
- Control: $20 per user / month
- Variation B: Save up to 60%
- Variation C: A usability version better explaining our product with a few key bullet points
- Variation D: Save up to $2500 per year
- Variation E: Free Your Voice – a creative concept
Visual Website Optimizer was our tool of choice and we built out the different variations. Each website visitor was randomly assigned one of the variations and that became their homepage thanks to a cookie. Every visit to the homepage would give each user their “unique” homepage and we had several goals to track.
Testing is nothing without goals
Metrics are great. Measuring the correct metrics is even better. It is critical to have a thorough understanding of what you are measuring and what it truly means. We created three distinct goals to track for this test.
- Engagement – this is defined as clicking anything on the website. What this means is clearly debatable and it may be completely irrelevant, but I wanted to see how many folks clicked something on the page in a digestable way.
- Enters the Store – we sell our business phone solution online. Tracking the funnel is extremely important and we wanted to see how many folks clicked into the store.
- Buys Phonebooth – does this need an explanation? This is the most important metric for us. While entering the store and engagement are both important… frankly my dear, I don’t give a damn. I want money. It helps to keep a roof over my head
Pretty simple, huh? Now, we’re ready to see what happens. From here, we start our test early on a Monday morning – a clean week.
Have some fun and test yourself
Before you run a test, I highly encourage you to make an assumption. Guess what you think will perform the best and why. Heck, create a contest amongst your team and have everyone pick their favorite. Give a prize to the winners!
Leveling the playing field for testing will help everyone understand that a website isn’t a symphony of dorks and geeks coding and designing away to make something that they think looks good. It is science. Well, if done correctly that is.
While my opinion is arguably worth two cents, I thought that our control would perform the best. $20 is a really low number. It is inexpensive for a business phone solution. Plus, I had stared at this version for two months. It was stuck in my head, but I thought that folks would gravitate to a killer price and buy our product.
The Results
I was wrong. For those that know me, that happens a lot. Most of the time, I’ll even admit it.
As you can see, the visits were pretty evenly distributed. The dollar value there is the rolled up version of dollars per visitor (dollars of Phonebooth bought / visitors). Save up to 60% is clearly our most compelling value proposition at this point for driving revenue.
Our control, $2500/year and creative Free Your Voice concept have about half of our traffic clicking something on the page. Not dollars, but it is cool to see. Lastly, every single variation gets more folks into the store. While this does create a larger funnel, we care most about the folks actually buying. The test does need more time to run, but let’s zoom in on folks buying Phonebooth.
Holy kittens, unicorns and zombies (these things work in presentations – not sure about in copy?)! That is a 328% improvement in revenue per visitor over our control and I’m clearly an idiot!
The + or – number shows that there is a great deal of variance and the test needs to run longer, but wow. The control is doing well, but it is slightly eclipsed by a text focused description of the product. Save up to 60% looks like the clear winner here!
What should we learn from this?
Test! Be creative. Create multiple versions of everything – buttons, landing pages, banners, images, layouts, etc. Limit your tests to one thing at a time to try to reduce any other variables. No matter how smart you are, odds are you don’t think exactly like all of your customers. Testing is an easy way for us to figure out what works and what doesn’t.
From there, the burden is on us as marketers to make adjustments with the knowledge we have. We filter it into copy, campaigns, communicate it with Sales and Product and continue to try to make incremental improvements to make more money and improve the customer experience.
Remember, bringing in more money than you are paid is somewhat important in your career. 😉
What are you A/B testing?
Have you had any successful tests? Are there tests every website should do? Any recommendations? Please share in the comments as I am becoming a testing fiend.
Image source: Shutterstock.com