Abba - JavaScript a/b testing

Abba is a self-hosted a/b testing framework built to help improve conversion rates on your site.

Tuning signup and payment forms can have huge effects on your business, and it’s a good idea to supplement design and copy changes with data on how differences actually effect conversion.

We built this tool for internal use at Stripe, and today we’re open-sourcing it for everyone. The key design goal of Abba is to have a great JavaScript API (unlike some of the other a/b testing services out there). For example:

<script src="//localhost:4050/v1/abba.js"></script>

<script>
  Abba('test name')
    .control('test a', function(){
      // Show test a
      $("#test-a").show();
      $("#test-b").hide();
    })
    .variant('test b', function(){
      // Show test b
      $("#test-b").show();
      $("#test-a").hide();
     })
    .start();
</script>

<script>
  $('button.complete').click(function(){
    // Successful conversion, mark the test as complete
    Abba('test name').complete();
  });
</script>

Here we’re setting up a control called test a, and a variant called test b. Sometimes the callback associated with the control will be invoked, and sometimes the variant’s callback will be run. Stats are sent back to the analytics server, hosted by you, where they are displayed in an admin interface.

You can see conversion rates per day for a configurable time period and even slice data by browser type. The different variants can be weighted such that some appear more times than others. Lastly we calculate a z-score for the displayed conversion rates, indicating the statistical probability that a test’s results are accurate.

See the GitHub project for the source and more information.

 
1,743
Kudos
 
1,743
Kudos

Read this next

Killing a library

A programming library’s main mission should be to make itself redundant, and with that in mind that I’m announcing the deprecation of Juggernaut. Juggernaut was one of the first solutions to pushing data in realtime to... Continue →