17 May

Faster A/B Testing

We’ve all read about A/B testing. How it can help you increase conversion four quadrillion percent, twice. The problem is you need a few (a lot actually) conversions to be able to make a choice. So for example, when I run an A/B test on my ebook sales page, I let it run for a fews weeks until I get a few conversions (sales).

That is a very slow process. Also, you can’t run multiple A/B tests at the same time, as it would mix up the results of each test.

When I launched my class, there was quite a lot of things I wanted to test on the page. The problem was, I could only get 20 conversions (sales) and I had to get results fast. There was no way I could wait weeks for results.

So here’s what I did.

Track scrolling

Instead of tracking sales or clicks, I tracked scrolling. I assume if someone finds the headline or copy interesting, he will scroll to read more. The goal of the sales page is to get people to read it till the end.

Fortunately, I’ve been using Optimizely which makes tracking custom events pretty easy. Here’s the code to track scrolling past 25% and 90%.

var scrolledTo25Pct = false, scrolledTo90Pct = false;
  var bottom = $(window).height() + $(window).scrollTop();
  var height = $(document).height();
  var percentage = Math.round(100*bottom/height);
  if (percentage > 25 && !scrolledTo25Pct) {
    scrolledTo25Pct = true;
  if (percentage > 90 && !scrolledTo90Pct) {
    scrolledTo90Pct = true;

Using that technique, I was able to tweak the headline and copy, and get results in a few hours instead of weeks.

40% increase in conversions

My tests totalled a 40% increase in people that clicked the Register button.

I must admit A/B testing sometimes feels like black magic. For eg.: switching to a blue Buy button on my book sales page yielded a 440% improvement in sales… ?! But running lots of smaller tests seems like a better thing to do than depending on a full conversion such as sale or sign up.

Optimizely is pretty awesome! Give it a try.

Like what you just read? Subscribe to my newsletter to learn how I make a living selling my products online.

Or if you're into coding, join my free class on rebuilding a Ruby web server.

blog comments powered by Disqus