50% Boost of Conversion Rate with A/B Testing

On the InDefero website, you can see at the top a series of links to access the different parts of the website. In my case, I wanted to improve the percentage of people accessing the page with the plans.

Before my tests, my data from Google Analytics gave me for the month of July:

• Home page: 3,849 page views.
• Plans: 824 page views (21.41% of the home page views).

I got everywhere that just changing a link in your website could drive the conversion up, so I decided to follow the same approach and used the base case plus 2 alternatives for my link to the plan page from the homepage:

1. Free Hosting base case for 50% of the visitors.
2. Pricing and Signup first alternative for 25% of the visitors (coming from the GitHub page).
3. See Plans & Pricing second alternative for 25% of the visitors (coming from a blog post about 37signals).

And the results are really nice!

Every morning, my AB/Testing framework is sending my an email report with the status of my tests, this test is the first real test, so I was eager to get the first statistically relevant results. It came this morning:

 Test: (1) Link to plans
 Alt: (1) Link to plans - Free Hosting
    Raw: 186, 42
    Conversion rate: 22.58%
    Recommended sample size: 210, 585, 5268
 Alt: (1) Link to plans - Pricing and Signup
    Raw: 91, 25
    Conversion rate: 27.47%
    Recommended sample size: 162, 450, 4056
    Z-Score: 0.87
    Confidence: 80.91%
 Alt: (1) Link to plans - See Plans & Pricing
    Raw: 96, 33
    Conversion rate: 34.38%
    Recommended sample size: 117, 325, 2933
    Z-Score: 2.06
    Confidence: 98.01%

As I have a high level of conversion anyway, I do not need to run the test for too long to get meaningful results. What is really important is:

• The 22.58% conversion rate of the base case is in line with the Google Analytics stats.
• The See Plans & Pricing link is really better with a 50% increase.
• The See Plans & Pricing is even better than the base case as the people clicking are in the mood to look at something costing money, as it is clear that a paid offer is available.

The next step is to use this alternative all the time and to improve the sign up page. If you are interested, you can do some split testing statistics here.

Loïc,
Aug 25, 2009

Read more news with tag: InDefero, A/B Testing, ...