The What And How of Better A/B Testing

Posted by | · · · · | Blog · Conversion Rate Optimization | No Comments on The What And How of Better A/B Testing

If you’re currently A/B testing using Google Content Experiments (Formerly Google Website Optimizer), a home-grown system, or simple pre vs. post analysis using your analytics data, this is an intervention. There is a better way to optimize your website


Now, we use Google products all the time (including Content Experiments); and we’ve even been guilty of doing some pre vs. post analysis now and then. But even if you’re happy getting by with these free tools/techniques, they’re not always the best.


To run an A/B test with Google Content Experiments, for example, the process may look something like this:

  • Brainstorm and identify hypothesis to test.
  • Brief designer.
  • Designer comps up page/site variation(s) to test.
  • Comps revised/approved and sent to developer/webmaster.
  • Developer/webmaster codes up variation(s).
  • Variations revised/approved.
  • GA Content Experiments code sent to developer/webmaster and added to variation(s).
  • Variation(s) pushed live to site.
  • Test runs.
  • Results, analysis, reporting, etc.
  • Test complete.
  • Developer/webmaster removes GA Content Experiments code, rolls out winning changes to site, redirects variation page(s) as appropriate.
  • Rinse and repeat.



Key point: How long did all of that take? Aside from the actual time the test was live, that could have easily been two weeks to a month (or more). And that’s if it all went off without a hitch!


So what if the process looked more like this:

  • Brainstorm and identify hypothesis to test.
  • Marketing team (with designer, as needed) makes changes to the site in real time using a browser-based tool.
  • Test pushed live to site (without involving developer/webmaster).
  • Test runs.
  • Results, analysis, reporting, etc.
  • Test complete.
  • Developer/webmaster rolls out changes to site.
  • Rinse and repeat.



The time spent from brainstorm to active testing in this second scenario could literally be half an hour or less (depending on the changes). Crazy awesome stuff – and this is totally possible now!


And let’s talk impact: if we assume that it takes two weeks for a test to be completed – start to finish – we’ve just doubled/tripled the number of tests we can run in a given time period in comparison to the first scenario. If you’re testing the right things, that means more learning, more conversions, better user experiences… faster.


You should be doing this. If something’s holding you back, call us!

Share on FacebookShare on Google+Tweet about this on TwitterShare on LinkedInPin on Pinterest

No Comments

Leave a comment