• How to Run an A/B Test to Improve Your CRO

How to Run an A/B Test to Improve Your CRO

Want to increase conversions, but not sure where to begin? What if there’s something about your website, landing pages, or marketing emails that’s negatively affecting your conversion rate? Before you can improve your CRO (conversion rate optimization), you first need to figure out what’s wrong. And A/B testing can help.

A/B testing involves comparing the performance of two different versions of something, (like a web page, email, or ad), to find out which one performs better. It can be an effective method for determining which version drives the most conversions. Which means that A/B tests should play a major role in your conversion rate optimization plan.

A/B Testing Basics

Before you start A/B testing (sometimes called split testing), you need to analyze user behavior on your site. You can use heatmaps and on-page surveys to do this. The goal is to determine why people aren’t converting. Maybe your CTA (call-to-action) button isn’t visible enough, or your sign-up form is too long.

Use this information to form a hypothesis. What changes do you think will drive more conversions? For example, your hypothesis may be that, “Changing the color of the CTA button will increase conversions.”

The next step is to create a variation (of your landing page, CTA button, etc.) based on your hypothesis. Then test how well it performs compared to the original. The variation that drives the most conversions is the version you need to implement in your CRO strategy.

In some cases, you may find little or no difference between how well the two variations perform. In which case you’ll need to create a new hypothesis to test.

What Should You Test to Improve CRO?

Split testing can help you analyze the performance of pretty much all aspects of your website, emails, and marketing campaigns. However, you probably don’t have the time or budget to test each and every element. Instead, try testing the elements that are likely to have the biggest impact on your conversions. Here are 7 elements you can A/B test for conversion rate optimization:

1. Test Your Headlines

Headlines are one of the first things visitors will notice on your site. So your landing page should have a snappy and relevant headline that immediately catches their attention. It should be compelling enough to make visitors want to read more, and eventually convert. In a BEHAVE (previously WhichTestWon) case study, changing the headline for a business course landing page helped increase conversions by 6.49%.

Here’s the variation that won:

BEHAVE landing page example - increasing CRO

And here’s the other variation:

BEHAVE landing page example 2 - improving CRO

If you compare the two headlines, you can see that the winning variation includes a specific dollar amount. The headline helps visitors visualize the amount of money they could make per month rather than just making their “first dollar.”

2. Test Your Sales Copy and Product Descriptions

While your headlines should attract visitors, your sales copy should close the sale. The copy on your home or landing page should inform visitors about the features of your product/service, and how they can benefit from it. It should be educational and enticing at the same time. The same goes for your product descriptions.

Take a look at how the Buffer homepage is all about the visitor. Instead of talking about what the tool can do, it focuses on what people can do using the tool.

Buffer homepage - conversion rate optimization strategy

And the description of Buffer’s features isn’t about what the tool does. It’s about how users can benefit from using it.

 Buffer's customer reviews - cro strategy

3. Test Your Design and Layout

Your website design may look amazing, but it’s possible that it’s too distracting. It may be preventing visitors from focusing on the action you want them to take.

Your site design should ideally be clean and minimal, with a focus on the most important elements – like sales copy and CTA. It should also be easy for users to navigate, which means you may also need to build a responsive website to adapt to different devices.

Apply the data from your user behavior tests to build different design and layout variations for A/B testing.

Let’s say you ran a heatmap test, and found that your site visitors are rarely navigating the area where you’ve placed your sign-up form. Then you could test the form in an area where visitors are most active.

Optimizely conducted a case study on the complete redesign of their learning hub, Optiverse. They completely redid the layout, and made some changes to the site copy.

Here’s what the old website looked like:

Optimizely case study on complete redesign Optiverse - A/b Testing

And here’s a look at the new website design:

Optimizely new website design, Optiverse - A/b Testing

As you can see from these two variations, the old design had a lot of text spread out over the entire page. Nothing really stands out. But in the new design, the text no longer takes up the entire page.

The new design also has clear, eye-catching boxes that highlight the three main features of Optiverse. If you take a look at the original design, there was no way for visitors to know that Optiverse offered these features.

The overall design is visually appealing. As a result of the website redesign, Optiverse experienced a 133% increase in entrances among logged in users. They also increased new member registrations by 464%.

4. Test Your Graphics and Images

Your choice of graphics and images can have a huge impact on your conversions. The purpose of graphics and images is to attract visitors, and highlight the benefits of your product/service.

They shouldn’t distract users from the action you want them to take. Instead, they should guide visitors towards the next action. This is why it’s crucial that you do A/B testing on your images/graphics as part of your CRO plan.

Maybe the background image on your landing page or homepage isn’t appealing. Or maybe you’re using low-quality product images that don’t display your product’s best features.

Sometimes, your page may even perform better without graphic elements, like in one BEHAVE case study. Out of the two variations shown in the image below, the one without the graphic resulted in more clicks. In fact, there was a 59.9% increase in their click-through rate with this variation.

BEHAVE case study - split testing

The best practices for website graphics and images may differ from industry to industry. If you’re selling consumer goods, you’ll need a high-quality image that properly highlights the product and its features. If you’re in the service industry, using images of real people may work in your favor.

Regardless of industry, images shouldn’t just be decorative. They should support your claims, and strengthen the desire to buy.

To find out what works best for your website conversion optimization, you’ll need to test different variations of your images or graphics.

OptinMonster uses fun graphics to illustrate each feature of their tool. Their graphics are engaging, and make a huge difference to the page design itself.

OptinMonster uses fun graphics on landing page - Conversion strategy

5. Test Your Forms

Another crucial element to split test is your sign-up form. Maybe you want people to sign up for a subscription, or you need them to sign up before completing a purchase. Either way, the form design and copy could have a huge impact on your conversion rate.

Like images and graphics, there is no foolproof method for optimizing your sign-up form. So split testing it should be included in your conversion rate optimization plan.

Optimizely conducted a case study on Trunk Club, a subscription service for clothing. The website originally had a simple sign-up form with minimal design.

Here’s what it looked like:

Trunk Club sign-up form testing - a/b test

They tested a variation that had more steps than the original, but which also had a stronger visual presentation. The new sign-up process places more focus on each question. And users have to select one of several images to answer the question.

Here’s what the first step looks like:

Trunk Club new sign-up process - conversion rate optimization plan

As you can see in the screenshot above, each image includes a brief description of the style. The new sign-up process is more engaging to users, and helped Trunk Club increase conversions by 133%.

Although the popular belief is that short and simple sign-up forms are better, that’s not always the case. It all depends on how you execute them, which is why A/B testing your sign-up process and sign-up forms is crucial.

6. Test Your CTA

Your call-to-action is one of the most important elements to test for CRO (conversion rate optimization). The tricky part is that there are several aspects to test for your CTA.

Your CTA copy, button color, button size, and button placement all play important roles in your conversions. You’ll need to run A/B tests to cover all of these aspects in order to truly boost your conversions.

The first aspect to test is the copy. Since your CTA copy should guide people towards an action, it should be short, direct, and actionable. Preferably, it should also focus on what the user is going to get, instead of what you want them to do.

For example, “Get Your Free Guide,” will typically perform better than, “Download Your Free Guide.”

BEHAVE conducted a case study on the two CTA variations shown below. One invites visitors to get a quote, and the other invites them to get pricing.

BEHAVE case study on two CTA variations - conversion rate optimization tips

The first variation outperformed the second, and increased form submissions by 104%. The hypothesis was that since it’s expensive to get water treatment services, the idea of getting a “quote” would be more effective for building trust.

Your CTA copy will greatly depend on what you’re selling or offering. Come up with several variations, and test how each performs.

Testing other aspects of your call-to-action, such as button size, color, and placement is also necessary for CRO. You should test all of these elements to ensure that your CTA button is prominently displayed.

First of all, your color choice should blend well with the other colors on your page. But it shouldn’t blend so well that it’s difficult for visitors to notice. The best thing to do is to choose a color that complements, but contrasts with your main page color. You can then test a few colors to see which one seems to drive the most conversions.

Check out the CTA button below from Olark, for example. The color green instantly stands out so visitors won’t miss it. At the same time, the hue complements the bluish tone of the background.

testing CTA from Olark - conversion rate optimization

As for button size, it should be big enough to be noticeable. But it shouldn’t be so big that it overshadows every other element on your site. You can use the data from your user behavior research to determine the best place to put your CTA button.

7. Test Your Marketing Emails

Your A/B tests shouldn’t only focus on your website elements. A/B testing is just as important for your marketing emails. Your email marketing campaigns are a crucial part of your conversion rate optimization strategy.

You should test several aspects of the emails you send out to prospects. The most important elements to test are the subject lines, images/graphics, CTA, and email copy. Additionally, you should try sending your emails out at different times of day, and on different days of the week to determine the best times and days to send them.

The subject lines of your marketing emails are the first thing recipients will notice. And they play a major role in whether or not people open your emails.

Here are some best practices for email subject lines:

  • Use Short Subject Lines – Keep your subject line short so that recipients can see the entire thing, no matter what device they use. Most mobile devices only show between 33 and 38 characters, according to Listrak. Try to keep your subject lines to 35 characters or less. If you need to go over 35 characters, make sure that your main message is in the first 4 to 6 words of your subject line.
  • Make Them Personal – Personalizing your subject lines can make a huge difference in your open rate. When you run an A/B test to improve CRO, try to include each recipient’s first name at the beginning of your subject lines. You can also try using “you” or “your” in your subject lines to see which performs better.
  • Add Emojis – Surprisingly, adding emojis to your subject lines could significantly boost your conversions. In fact, several studies have found that adding emojis to subject lines increased open rates, as well as unique clicks and click-through rates. A BEHAVE case study also proved this, with the emoji subject line outperforming the subject line with no emojis.

Marketing Emails A/B testing


These are some of the most effective CRO tips for boosting your conversions with the help of A/B testing. In addition to the elements mentioned above, you can conduct further tests based on what’s important for your brand. You can split test different ad copies, pricing structures, offers, etc. for maximum conversions, engagement, and ROI.

What’s the first thing you plan to A/B test? Let me know in the comments. And if you need any help driving more conversions, make sure you get in touch with me.

Shane Barker is a digital marketing consultant who specializes in sales funnels, targeted traffic, and website conversions. He has consulted with Fortune 500 companies, influencers with digital products, and a number of A-List celebrities.


  1. […] make your job more efficient, you need to identify which areas need improvement, and which elements need testing. And that’s where analytics enter the […]

  2. […] a relevant trust seal next to the CTA button usually relieves customer anxiety. Run a test and compare the conversion rates between the page with a trust seal, and the one without. This will help you determine whether or […]

Leave A Comment