13

I have a bunch of tests that I don't want them running during CRAN checks or Travis CI builds. They are either long-running, or they could cause transaction/concurrency conflicts writing to a networked database. What approach of separating them (from the R CMD check tests) works best with testthat?

Should I put those tests in a separate folder? Should I tag their filename and use a regex? (eg Using filter argument in test_package to skip tests by @Jeroen)

http://cran.r-project.org/web/packages/policies.html:

Long-running tests and vignette code can be made optional for checking, but do ensure that the checks that are left do exercise all the features of the package.

Community
  • 1
  • 1
wibeasley
  • 5,000
  • 3
  • 34
  • 62
  • 1
    If you put them in another directory within tests, then you can still test them manually with `test_dir()`, but they won't be running with `test()` or `R CMD check`. E.g. R6 has some manual tests: https://github.com/wch/R6/tree/master/tests – Gabor Csardi Sep 01 '14 at 02:09
  • I like that @GaborCsardi, and how the tests are contained together, but distinguished by `test()` and `test_dir()`. – wibeasley Sep 02 '14 at 02:21
  • @GaborCsardi that was exactly what I was looking for, and it [worked well](https://github.com/OuhscBbmc/REDCapR/commit/a93bf9e95c259068ba740596679f3943e7b521fa) for my package's needs. If you change your comment to an answer, I'd like to give your response credit. – wibeasley Sep 04 '14 at 03:13

2 Answers2

8

FYI: testthat 0.9 now comes with a skip() function.

However, I can't seem to figure out how/where exactly to use it. Putting inside my test_that() function, the test runs anyway. If you put it before testthat(), then skip() throws an error.

Rappster
  • 12,762
  • 7
  • 71
  • 120
  • 1
    Thanks for mentioning that --I saw the release announcement a few days ago. Immediately inside the test functions, I've added the like `testthat::skip_on_cran()`. It's a good fit for some of my tests, because the main reason I wanted some manual tests are so that it wouldn't get rejected from CRAN if the server wasn't peppy right then. I don't really care if the Travis-CI or R-Forge checks fail, because it's easy to rerun. – wibeasley Sep 29 '14 at 03:36
  • 5
    Could you tell me, how exactly you are using `skip()` and/or `skip_on_cran()`? Where do I need to put it in order to make a specific test skip when running `test_package()` or `CRTL+SHFT+T` in RStudio? – Rappster Sep 29 '14 at 14:57
  • 2
    Here is a link to the [test code](https://github.com/OuhscBbmc/REDCapR/blob/80e9fad0eb8a42375d1291360032c4ee5ec12b97/tests/testthat/test-read_oneshot.R#L10). To run it, I either check the package with `CTRL+SHIFT+E` or run just the tests with [`test_results_checked <- devtools::test()`](https://github.com/OuhscBbmc/REDCapR/blob/80e9fad0eb8a42375d1291360032c4ee5ec12b97/utility_scripts/refresh.R#L12). – wibeasley Sep 29 '14 at 21:10
  • Thanks man. That's how I did it and it didn't work for some reason but now it does ;-) Must have been something else – Rappster Oct 01 '14 at 10:25
  • 3
    Note that skip (At least in current dev version) requires a message argument. E.g. inside `test_that(` use `if (is.friday(today())) skip("Not today.")` – jan-glx Oct 04 '15 at 18:09
7

If you put them in another directory within tests, then you can still test them manually with test_dir(), but they won't be running with test() or R CMD check.

E.g. R6 has some manual tests: https://github.com/wch/R6/tree/master/tests

Gabor Csardi
  • 10,705
  • 1
  • 36
  • 53