0

The individuals are given as GUIDs and I need to split them by a criterion into A/B to run experiments. What is a good function for spreading them into equal buckets?

Andrei
  • 13
  • 3
  • Is there anything about the space of GUIDs that can be split into roughly two equal groups, without actually having to keep track of your buckets. What about the last bit? Or pretty much any evenly divided numeric characteristic. – Nathan Cooper Sep 18 '17 at 22:19
  • 1
    It almost certainly depends on how the GUIDs are generated. Trying to follow a blanket rule without understanding your data is a good way to screw up your results. – Bernhard Barker Sep 19 '17 at 00:26
  • The guids are the regular .Net NewGuid function. I always asuned that Guid generation is standard so I never questioned it. – Andrei Sep 19 '17 at 02:21
  • If you already have a population, then do some testing on the existing GUIDs to see if there is something you can use. Perhaps the last bit will work. Or perhaps you can use the last bit of the `time-low` field (see https://en.wikipedia.org/wiki/Universally_unique_identifier#Format). If you don't have an existing population, generate a few thousand GUIDs and perform some analysis to see if you can come up with a good differentiator. – Jim Mischel Sep 19 '17 at 13:37

1 Answers1

0

The most basic approach is to calculate a random number between 0-1. If the number is > 0.5, serve the variation. Otherwise, serve the control.

Alternatives include simply rotating back and forth between control and variation, or ensuring a relatively smooth distribution amongst all variables, depending on what you're tracking.

Some good resources to check out include this post from Twitter, and the Wikipedia post on A/B testing gives a good overview as well.

Bricky
  • 2,572
  • 14
  • 30
  • The populations is fairly small, a few thousands and grows at a very slow rate. Also there is no switching the population from A to B. Once it is treatment it can't go back. That is why the random number approach is not doable, they need to always be in the same group. Thanks for the links, I will go through them tonite. – Andrei Sep 19 '17 at 02:28
  • If you want it to be permanent then you'll have to attach it to their user and check which version to serve when they visit your site. But, you're still randomly assigning either version A or B. You're just doing it once per user. – Bricky Sep 19 '17 at 02:42