What resources are there for A/B split-testing in Python?

Question

There are several A/B split testing modules/plugins for Rails.
http://github.com/paulmars/seven_minute_abs
http://www.bingocardcreator.com/abingo
http://vanity.labnotes.org/
etc.

Is there anything similar for Python?

Answer 1

You can look at SimpleAB library. It's pretty simple but flexible tool to organize your content in A/B test. Currently SimpleAB has several ways to create test class:

• SimpleAB test. This implementation of AB Test provides way to implement alternatives as methods with names A, B, ..., Z.

>>> import simpleab
>>> class MyTest(simpleab.SimpleAB):
...     name = 'MyTest'
...     def A(self): return 'Side A'
...     def B(self): return 'Side B'
...     def C(self): return 'Side C'
...
>>> myab = MyTest()
>>> myab.test()
'Side A'
>>> myab.current_side
'A'
>>> myab.test(force_side='C')
'Side C'

• ConfigurableAB test. This implementation of AB Test provides way to configure test name, sides and selector instance. If selector isn't specified random selection will be used.

>>> improt simpleab
>>> import random
>>> myab = simpleab.ConfigurableAB(name='MyTest',
...             sides={'A': 'Side A', 'B': 'Side B'},
...             selector=lambda: random.choice(['A','B']))
>>> myab
<ConfigurableAB [name: MyTest, sides: ['A', 'B']]>
>>> myab.test()
'Side A'
>>> myab.current_side
'A'

Actually the lib doesn't have solid support for data storage and analytic facilities yet, but it allows to implement this stuff quickly. That I think will be done soon :)

Answer 2

It's only at version 0.1.2 so far, but Swab looks promising. Example of testing two sizes of a form button:

from swab import Swab
s = Swab('/tmp/.swab-test-data')
s.addexperiment('button-size', ['default', 'larger'], 'order-completed')

Answer 3

I guess I'm a little late to the party -- but if you'll forgive the shameless plug, please check out Dabble, my own A/B framework. It works quite nicely for web frameworks using class-based views, supports filesystem or mongodb storage, and generates results for you.