Description
As I'm working on a React Native Typescript project, I'm having trouble implementing e2e testing on my CI.
I am running Detox locally on a Mac with Catalina 10.15.6. The tests pass consistently (after I fixed flakiness noted here).
When I push my changes, Circle CI runs automatically (on a Mac), building correctly. Then, the tests all fail due to issues with the matchers (which work correctly locally).
Reproduction
- Run Detox Build & Test locally.
- Initiate Circle CI, configured to do both the above.
Expected Behavior
If the tests pass locally and all the matchers work, the same will occur on a Circle CI machine.
Environment
- Detox: ^17.0.0
- React Native: 0.62.2
- Node: 12.18.0
- Device: iPhone 11 Pro (Simulator)
- macOS: Catalina 10.15.6
- Xcode: 11.5.0
- Jest: ^24.9.0
- Jest Circus: ^26.1.0
- Typescript: ^3.8.3
- Yarn: 1.22.4
Logs
Here is the output from my failed test on Circle CI:
yarn run v1.22.5
$ /Users/distiller/project/node_modules/.bin/detox test --cleanup
detox[12206] INFO: [test.js] cleanup=true reportSpecs=true useCustomLogger=true forceAdbInstall=false DETOX_START_TIMESTAMP=1599242684729 node_modules/.bin/jest --config e2e/config.json '--testNamePattern=^((?!:android:).)*$' --maxWorkers 1 e2e
detox[12207] INFO: [DetoxServer.js] server listening on localhost:49498...
detox[12207] INFO: [AppleSimUtils.js] xx.xxx.xxxxx launched. To watch simulator logs, run:
/usr/bin/xcrun simctl spawn 03E73092-B137-4FA2-B3F9-EE1901FCCC1E log stream --level debug --style compact --predicate 'process == "xxxx"'
detox[12207] INFO: Project Test Suite is assigned to 03E73092-B137-4FA2-B3F9-EE1901FCCC1E {"type":"iPhone 11 Pro"}
detox[12207] INFO: Project Test Suite: should enter an invalid email and error message should appear
detox[12207] INFO: Project Test Suite: should enter an invalid email and error message should appear [FAIL]
detox[12207] INFO: Project Test Suite: should enter a registered email and pass the email input screen to login code entry
detox[12207] INFO: Project Test Suite: should enter a registered email and pass the email input screen to login code entry [FAIL]
detox[12207] INFO: Project Test Suite: should enter a valid email and invalid login code then error message should appear
detox[12207] INFO: at e2e/firstTest.e2e.js:176:19
DO NOT REMOVE
detox[12207] INFO: at e2e/firstTest.e2e.js:123:19
Something went wrong: Error: Test Failed: Timed out while waiting for expectation: TOBEVISIBLE WITH MATCHER(identifier: == “xxxx”) TIMEOUT(1s)
TIP: To print view hierarchy on failed actions/matches, use loglevel verbose and above.
at processTicksAndRejections (internal/process/task_queues.js:97:5)
detox[12207] INFO: Project Test Suite: should enter a valid email and invalid login code then error message should appear [FAIL]
detox[12207] INFO: Project Test Suite: should enter a valid email then a valid login code and end up at the testing site preferences screen
detox[12207] INFO: at e2e/firstTest.e2e.js:200:19
DO NOT REMOVE
detox[12207] INFO: at e2e/firstTest.e2e.js:123:19
Something went wrong: Error: Test Failed: Timed out while waiting for expectation: TOBEVISIBLE WITH MATCHER(identifier: == “xxxx”) TIMEOUT(1s)
TIP: To print view hierarchy on failed actions/matches, use loglevel verbose and above.
at processTicksAndRejections (internal/process/task_queues.js:97:5)
detox[12207] INFO: Project Test Suite: should enter a valid email then a valid login code and end up at the testing site preferences screen [FAIL]
detox[12207] INFO: Project Test Suite: should attempt to advance past testing site preferences screen without entering preference
detox[12207] INFO: Project Test Suite: should attempt to advance past testing site preferences screen without entering preference [FAIL]
detox[12207] INFO: Project Test Suite: should rank a testing site and advance to the availability page
detox[12207] INFO: Project Test Suite: should rank a testing site and advance to the availability page [FAIL]
detox[12207] INFO: Project Test Suite: should toggle a time slot, collapse an accordion, and then click submit
detox[12207] INFO: [AppleSimUtils.js] xxx.xxxx.xxxx launched. To watch simulator logs, run:
/usr/bin/xcrun simctl spawn 03E73092-B137-4FA2-B3F9-EE1901FCCC1E log stream --level debug --style compact --predicate 'process == "XXXXX"'
detox[12207] INFO: Project Test Suite: should toggle a time slot, collapse an accordion, and then click submit [FAIL]
FAIL e2e/firstTest.e2e.js (174.609s)
Project Test Suite
✕ should enter an invalid email and error message should appear (8205ms)
✕ should enter a registered email and pass the email input screen to login code entry (8695ms)
✕ should enter a valid email and invalid login code then error message should appear (10658ms)
✕ should enter a valid email then a valid login code and end up at the testing site preferences screen (13953ms)
✕ should attempt to advance past testing site preferences screen without entering preference (4959ms)
✕ should rank a testing site and advance to the availability page (5033ms)
✕ should toggle a time slot, collapse an accordion, and then click submit (9286ms)
● Project Test Suite › should enter an invalid email and error message should appear
Test Failed: Failed expectation: TOBEVISIBLE WITH MATCHER(identifier: == “xxx”), View “<RCTUITextField: 0x7f9fb28ef600>” is not visible at point “{"x":59.333333333333343,"y":23.333333333333371}”; Window “<RCTRedBoxWindow: 0x7f9fb24b4650>” is above the tested view's window and its transparency around point “{"x":98.000001271565765,"y":309.66666730244958}” is below the tested threshold (0.5)
TIP: To print view hierarchy on failed actions/matches, use loglevel verbose and above.
● Project Test Suite › should enter a registered email and pass the email input screen to login code entry
Test Failed: No elements found for “MATCHER(identifier: == “xxx”)”
TIP: To print view hierarchy on failed actions/matches, use loglevel verbose and above.
● Project Test Suite › should enter a valid email and invalid login code then error message should appear
TypeError: Cannot read property 'XXXXX' of undefined
80 | it("should enter a valid email and invalid login code then error message should appear", async () => {
81 | const {
> 82 | constant,
| ^
83 | constant2,
84 | } = await someFunction();
85 | await expect(constant).toBeVisible();
at _callee4$ (firstTest.e2e.js:82:7)
at tryCatch (../node_modules/regenerator-runtime/runtime.js:45:40)
at Generator.invoke [as _invoke] (../node_modules/regenerator-runtime/runtime.js:274:22)
at Generator.prototype.<computed> [as next] (../node_modules/regenerator-runtime/runtime.js:97:21)
at tryCatch (../node_modules/regenerator-runtime/runtime.js:45:40)
at invoke (../node_modules/regenerator-runtime/runtime.js:135:20)
at ../node_modules/regenerator-runtime/runtime.js:145:13
● Project Test Suite › should enter a valid email then a valid login code and end up at the testing site preferences screen
Test Failed: No elements found for “MATCHER(text == “xxxx”)”
TIP: To print view hierarchy on failed actions/matches, use loglevel verbose and above.
● Project Test Suite › should attempt to advance past testing site preferences screen without entering preference
Test Failed: No elements found for “MATCHER(text == “xxxx”)”
TIP: To print view hierarchy on failed actions/matches, use loglevel verbose and above.
● Project Test Suite › should rank a testing site and advance to the availability page
Test Failed: No elements found for “MATCHER(text == “xxxx”)”
TIP: To print view hierarchy on failed actions/matches, use loglevel verbose and above.
● Project Test Suite › should toggle a time slot, collapse an accordion, and then click submit
Test Failed: No elements found for “MATCHER(identifier: == “xxxx”)”
TIP: To print view hierarchy on failed actions/matches, use loglevel verbose and above.
detox[12206] ERROR: [cli.js] Error: Command failed: node_modules/.bin/jest --config e2e/config.json '--testNamePattern=^((?!:android:).)*$' --maxWorkers 1 e2e
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Exited with code exit status 1
