Confidence intervals in java, testing the random pick of an element in a list of objects

Question

So I have this method that picks at random an object from a list of 2 objects. I would like to write a junit test (@Test) asserting based on a confidence level that somehow there's a 50% chance for each of the 2 objects to be picked.

The piece of code under test:

public MySepecialObj pickTheValue(List<MySepecialObj> objs, Random shufflingFactor) {

    // this could probably be done in a more efficient way
    // but my point is asserting on the 50% chance of the 
    // two objects inside the input list
    Collections.shuffle(objs, shufflingFactor);
    return objs.get(0);
}

In the test I would like to provide 2 mocks (firstMySepecialObjMock and secondMySepecialObjMock) as input objects of type MySepecialObj and new Random() as the input shuffling parameter, then assert that the firstMySepecialObjMock happens to be the choice 50% of the times and secondMySepecialObjMock happens to be the choice in the other 50% of the times.

Something like:

@Test
public void myTestShouldCheckTheConfidenceInterval() {

    // using Mockito here
    MySepecialObj firstMySepecialObjMock = mock(MySepecialObj.class);
    MySepecialObj secondMySepecialObjMock = mock(MySepecialObj.class);

    // using some helpers from Guava to build the input list
    List<MySepecialObj> theListOfTwoElements = Lists.newArrayList(firstMySepecialObjMock, secondMySepecialObjMock);

    // call the method (multiple times? how many?) like:
    MySepecialObj chosenValue = pickTheValue(theListOfTwoElements, new Random());

    // assert somehow on all the choices using a confidence level
    // verifying that firstMySepecialObjMock was picked ~50% of the times
    // and secondMySepecialObjMock was picked the other ~50% of the times
}

I am not sure about the statistics theory here, so maybe I should provide a different instance of Random with different parameters to its constructor?

I would also like to have a test where I could set the confidence level as a parameter (I guess usually is 95%, but it could be another value?).

What could be a pure java solution/setup of the test involving a confidence level parameter?
What could be an equivalent solution/setup of the test involving some helper library like the Apache Commons?

score 1 · Accepted Answer · answered Dec 31 '16 at 15:49

First of all this is the normal way to pick random elements from a List in Java. (nextInt(objs.size() produces random integers between 0 and objs.size()).
```
public MySepecialObj pickTheValue(List<MySepecialObj> objs, Random random) {
    int i = random.nextInt(objs.size());
    return objs.get(i);
}
```
You can read in Wikipedia about how many times you should perform an experiment with 2 possible outcomes for a given confidence level. E.g. for confidence level of 95% you get a confidence interval of 1.9599. You also need to provide a maximum error say 0.01. Then the number of times to perform the experiment:
```
double confidenceInterval = 1.9599;
double maxError = 0.01;
int numberOfPicks = (int) (Math.pow(confidenceInterval, 2)/(4*Math.pow(maxError, 2)));
```
which results in numberOfPicks = 9603. That's how many times you should call pickTheValue.

This would be how I recommend you perform the experiment multiple times (Note that random is being reused):

Random random = new Random();
double timesFirstWasPicked = 0;
double timesSecondWasPicked = 0;
for (int i = 0; i < numberOfPicks; ++i) {
    MySepecialObj chosenValue = pickTheValue(theListOfTwoElements, random);
    if (chosenValue == firstMySepecialObjMock) {
        ++timesFirstWasPicked;
    } else {
        ++timesSecondWasPicked;
    }
}
double probabilityFirst = timesFirstWasPicked / numberOfPicks;
double probabilitySecond = timesSecondWasPicked / numberOfPicks;

Then assert that probabilityFirst, probabilitySecond are no further than maxError from 0.5

I found a BinomialTest class in apache-commons-math but I don't see how it can help in your case. It can calculate the confidence level from the number of experiments. You want the reverse of that.

Confidence intervals in java, testing the random pick of an element in a list of objects

1 Answers1