0

The following equation provides the test statistic used in the Test of Proportions.

For a proposed a/b test, I am attempting to show the minimum value needed for the treated group (p2) to show a statistical significance at a 95% confidence level. In other words, I am trying to solve this equation for p2. Given that I know my total population size, the percent that will be treated, and Z-value, this would seem to be straightforward. However, I am getting stuck on the algebra.

I have written an R script that will run through a range of values for p2 until a p-value of a given confidence is met, but that is a sloppy way of solving the problem.

Mark Wagner
  • 363
  • 2
  • 7

1 Answers1

0

I wouldn't bother doing this algebraically (or however that's spelled).

Notice that if

Z = diff(p) / se(p)

then

0 = diff(p) / se(p) - Z

The uniroot function can then do the work for you. you provide the values of everything but p2, and uniroot will seek out the value that resolves to 0.

zdiff <- function(p2, p1, n1, n2, alpha = 0.025) 
{
  ((p1 - p2) - 0) / sqrt(p1 * (1-p1) / n1 + p2 * (1-p2) / n2) - qnorm(alpha, lower.tail = FALSE)
}

uniroot(f = zdiff,
        p1 = .5, n1 = 50, n2= 50, 
        interval = c(0, 1))

$root
[1] 0.311125

$f.root
[1] -1.546283e-06

$iter
[1] 5

$init.it
[1] NA

$estim.prec
[1] 6.103516e-05

So with equal sample sizes of 50 and p1 = .5, p2 would have to be less than 0.311125 to generate a statistically significant result at the two-sides alpha = 0.05 level.

Benjamin
  • 16,897
  • 6
  • 45
  • 65
  • I agree - solving algebraically is a pain (at this point, not even sure it can be solved). Thanks for sharing the uniroot function, was unaware that it existed (and learned about optimize function in the process). A lot more efficient than the rscript I had set up. Thanks. – Mark Wagner May 12 '16 at 00:30