Run espresso test multiple times

Question

Sometimes I faced with rare bug in my application. But I can't reproduce it as it's very rare. So, I decided to write simple espresso test:

@RunWith(AndroidJUnit4::class)
@LargeTest
class MainActivityTest {

    val password = "1234"

    @Rule @JvmField
    var mActivityRule: ActivityTestRule<MainActivity> = ActivityTestRule(MainActivity::class.java)

    @Test
    fun checkNotesListNotEmpty() {
        onView(withId(R.id.password_edit_text)).perform(typeText(password))
        onView(withId(R.id.notes_recycler_view)).check { view, noMatchingViewException ->
            if (noMatchingViewException != null) throw noMatchingViewException
            assertThat((view as RecyclerView).adapter.itemCount,  Matchers.`is`(1))
        }
    }
}

How can I loop this test and stop it when matching fails?

Try `fun checkNotesListNotEmpty() throws InterruptedException` — piotrek1543, Aug 28 '16 at 16:58
@piotrek1543 the main problem is that I can't find a way to run Espresso test multiple times(in loop). How can I do it? — Alexandr, Aug 28 '16 at 17:09

score 19 · Accepted Answer · edited Aug 07 '18 at 11:49

Use @Repeat annotation:

@RunWith(AndroidJUnit4.class)
public class MyTest {

    @Rule
    public RepeatRule repeatRule = new RepeatRule();

    @Test
    @Repeat(100)
    fun checkNotesListNotEmpty() {
    }

But you have to implement it yourself:

Repeat.java:

import static java.lang.annotation.ElementType.ANNOTATION_TYPE;
import static java.lang.annotation.ElementType.METHOD;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Retention( RetentionPolicy.RUNTIME )
@Target({ METHOD, ANNOTATION_TYPE })
public @interface Repeat {
    int value() default 1;
}

RepeatRule.java:

import org.junit.rules.TestRule;
import org.junit.runner.Description;
import org.junit.runners.model.Statement;

public class RepeatRule implements TestRule {

    private static class RepeatStatement extends Statement {
        private final Statement statement;
        private final int repeat;    

        public RepeatStatement(Statement statement, int repeat) {
            this.statement = statement;
            this.repeat = repeat;
        }

        @Override
        public void evaluate() throws Throwable {
            for (int i = 0; i < repeat; i++) {
                statement.evaluate();
            }
        }

    }

    @Override
    public Statement apply(Statement statement, Description description) {
        Statement result = statement;
        Repeat repeat = description.getAnnotation(Repeat.class);
        if (repeat != null) {
            int times = repeat.value();
            result = new RepeatStatement(statement, times);
        }
        return result;
    }
}

@Alexandr Did you check that? Accept some answer to make this site better :) — klimat, Sep 06 '16 at 17:23
Remember to add the @Rule annotation. I forgot it couldn't understand why it wasn't working... — Johan Ferreira, Mar 19 '21 at 14:41

score 17 · Answer 2 · edited May 03 '22 at 03:49

Solution presented by @mklimek in Kotlin.

How to use:

@Rule @JvmField
var repeatRule: RepeatRule = RepeatRule()

@Test
@RepeatTest(100)
fun checkNotesListNotEmpty() {

RepeatRule.kt

class RepeatRule : TestRule {

    private class RepeatStatement(private val statement: Statement, private val repeat: Int) : Statement() {
        @Throws(Throwable::class)
        override fun evaluate() {
            for (i in 0 until repeat) {
                statement.evaluate()
            }
        }
    }

    override fun apply(statement: Statement, description: Description): Statement {
        var result = statement
        val repeat = description.getAnnotation(RepeatTest::class.java)
        if (repeat != null) {
            val times = repeat.value
            result = RepeatStatement(statement, times)
        }
        return result
    }
}

RepeatTest.kt

@Retention(AnnotationRetention.RUNTIME)
@Target(AnnotationTarget.FUNCTION, AnnotationTarget.ANNOTATION_CLASS)
annotation class RepeatTest(val value: Int = 1)

Does not work with compose tests or probably with tests using `runTest`: Only a single call to `runTest` can be performed during one test. — Vladimir, Jun 20 '23 at 08:16

score 3 · Answer 3 · answered Aug 29 '16 at 05:57

Can't you have a separate test which loops that one? (if it is a one-of-its-kind situation):

@RunWith(AndroidJUnit4::class)
@LargeTest
class MainActivityTest {
    ...

    @Test fun checkNotesListNotEmpty() {...}

    @Test fun loopCheckNotesListNotEmpty() {
        while(true)
            checkNotesListNotEmpty()
    }
}

score 1 · Answer 4 · edited Oct 24 '17 at 00:28

1

Android Studio allows you to run a test N times or run until it fails. Click on "Edit configurations" for your test rule and search for the "Repeat:" setting.

Screenshot

edited Oct 24 '17 at 00:28

Enamul Hassan

5,266
23
39
56

answered Oct 23 '17 at 23:52

Rukmani Ravi

29
1

20

This only works for java unit tests not Instrumented Tests. – hopia Mar 16 '18 at 03:21

Piotr Zawadzki · Answer 5 · 2018-11-06T18:36:58.767

0

I had the same issue and as a result I've created a library to run test multiple times: https://github.com/stepstone-tech/AndroidTestXRunner

Once you run it, you can check the tests results for any failures.

edited Nov 06 '18 at 18:36

answered Nov 06 '18 at 18:31

Piotr Zawadzki

1,678
1
18
24

score 0 · Answer 6 · answered May 24 '23 at 03:24

I used parameterized test instead with an unused parameter.

@RunWith(Parameterized.class)

public class SampleParameterizedTest {

@Parameter(value = 0)
public int mTestInteger;

@Parameters
public static Collection<Object[]> initParameters() {
    // return a list with number of times you want this test to execute. Now it runs every test in this class 2 times with 2 different params..
    return Arrays.asList(new Object[][] { { 0 }, { 1 } });
}

Run espresso test multiple times

6 Answers6

Linked