Abba - JavaScript a/b testing

Abba is a self-hosted a/b testing framework built to help improve conversion rates on your site.

Tuning signup and payment forms can have huge effects on your business, and it’s a good idea to supplement design and copy changes with data on how differences actually effect conversion.

We built this tool for internal use at Stripe, and today we’re open-sourcing it for everyone. The key design goal of Abba is to have a great JavaScript API (unlike some of the other a/b testing services out there). For example:

<script src="//localhost:4050/v1/abba.js"></script>

  Abba('test name')
    .control('test a', function(){
      // Show test a
    .variant('test b', function(){
      // Show test b

    // Successful conversion, mark the test as complete
    Abba('test name').complete();

Here we’re setting up a control called test a, and a variant called test b. Sometimes the callback associated with the control will be invoked, and sometimes the variant’s callback will be run. Stats are sent back to the analytics server, hosted by you, where they are displayed in an admin interface.

You can see conversion rates per day for a configurable time period and even slice data by browser type. The different variants can be weighted such that some appear more times than others. Lastly we calculate a z-score for the displayed conversion rates, indicating the statistical probability that a test’s results are accurate.

See the GitHub project for the source and more information.


Now read this

Adobe’s 9% Flash tax

Nicolas Cannasse reports on an announcement by Adobe about a new ‘feature’ for Flash: a 9% tax of your net revenues for using certain features. Basically, what it means is that you’ll have to get a license from Adobe and pay them 9% of... Continue →