3

I am now trying to estimate the sample size needed for A/B testing of website conversion rate. pwr.chisq.test always gives me error message, when I have small value of conversion rate:

# conversion rate for two groups
p1 = 0.001
p2 = 0.0011

# degree of freedom
df = 1

# effect size
w = ES.w1(p1,p2)

pwr.chisq.test(w,
               df = 1,
               power=0.8,
               sig.level=0.05)

**Error in uniroot(function(N) eval(p.body) - power, c(1 + 1e-10, 1e+05)) : 
  f() values at end points not of opposite sign**

However, if I have larger value for p1 and p2, this code works fine.

# conversion rate for two groups
p1 = 0.01
p2 = 0.011

# degree of freedom
df = 1

# effect size
w = ES.w1(p1,p2)

pwr.chisq.test(w,
               df = 1,
               power=0.8,
               sig.level=0.05)

Chi squared power calculation 

      w = 0.01
      N = 78488.61
     df = 1   sig.level = 0.05
  power = 0.8

NOTE: N is the number of observations

Peter Pan
  • 65
  • 5

1 Answers1

2

I think there is a "numerical" explanation to that. If you take a look at the function's code, you can see that the number of samples is computed by uniroot and is supposed to belong to an interval whose boundaries are set to 1e-10 and 1e5. The error message states that this interval does not give you the result: in your case, the upper limit is too small.

Knowing that, we can simply take a wider interval:

w <- 0.00316227766016838
k <- qchisq(0.05, df = 1, lower = FALSE)
p.body <- quote(pchisq(k, df = 1, ncp = N * w^2, lower = FALSE))
N <- uniroot(function(N) eval(p.body) - 0.8, c(1 + 1e-10, 1e+7))$root

The "solution" is N=784886.1... that's a huge number of observations.

Vincent Guillemot
  • 3,394
  • 14
  • 21