1

In Firebase for iOS I’m running two tests (#2 and #3) with the same target action, 'do N'.

I have created an Audience "Fans" to use as a condition. Fans = users who have done N at least once in the previous app version.

Test#2 targets 100% users in the audience "Fans". Test#3 targets 100% users NOT in the audience "Fans".

From the docs we know that users are attached to audiences permanently

Specifically, users become permanent members of an audience after they are assigned to it.

So I would expect that users in the test#2 and test#3 do not intersect. However, bigquery shows that sometimes an event "do N" has corresponding user property as "firebase_exp_2" and "firebase_exp_3" at the same time – https://www.dropbox.com/s/2yyqcelbf8dryvc/Screenshot%202018-06-09%2017.49.33.png

How can this be possible? Moreover, the remote config options are not the same for these experiments. How do I know, which variant a user actually has?

Thanks in advance

Mikhail Berlyant
  • 165,386
  • 8
  • 154
  • 230
Anastasia Nesmeyanova
  • 11
  • 2
  • How are you defining the "User not in audience Fans" test? AFAIK, Remote Config doesn't have any kind of "target users not in audience" capability – Todd Kerpelman Jul 17 '18 at 21:36
  • @ToddKerpelman I set the parameters for one test as 'includes all of' audience and for the second one as "does not include all of" audience option. What am I missing? – Anastasia Nesmeyanova Jul 19 '18 at 08:05

1 Answers1

0

As far as I know, you should now use user audiences in a/b test as it takes a lot of time for a user to become a part of audience. Also in your case, a user can be in no audience (and be part of test2) and then become a member of your fans (and so be part of test1). You should use user properties for a/b tests, they work faster and also you can configure a property like "times_user_have_done_N" and target users who have done it > or < than x

Dave
  • 1
  • 1