14

Is it possible to use the cargo command to run library tests (i.e.., cargo test --lib) and documentation tests without running any integration tests (i.e., the tests in the crate's top-level tests directory)? Bonus points are awarded for compiling the integration tests without running them.

Here's the bigger picture. My crate is a client library for a web service, and the HTTP server is not part of the crate. I've organized my crate into:

  1. Library tests, which do not depend on the HTTP server,
  2. Documentation tests, which do not depend on the HTTP server, and
  3. Integration tests, which require the HTTP server to be running on the localhost.

As such, it's sometimes unfeasible to have the HTTP server running on the machine building the crate—e.g., a Travis CI build agent. In these situations I would like to build all tests but exclude all integration tests from running because every integration test will fail.

Craig M. Brandenburg
  • 3,354
  • 5
  • 25
  • 37
  • 1
    Are you aware of the `cfg` attribute to test features `#[cfg(feature = "")]`? – Matthieu M. Jan 14 '16 at 16:14
  • @MatthieuM. I hadn't thought of that. I suppose the idea is to put all the integration tests into their own feature. E.g., `cargo test --features integration_tests --no-run && cargo test` to build all tests and run only the library tests and documentation tests. However, it's not quite as convenient as what I would like—something like `cargo test --build-all --lib --doc`, which wouldn't require marking integration tests as being conditional on a feature. – Craig M. Brandenburg Jan 14 '16 at 18:00
  • There's also [ignore](https://doc.rust-lang.org/book/testing.html#the-ignore-attribute) attribute. – ArtemGr Jan 15 '16 at 09:51

1 Answers1

11

Looking at cargo help test (as you probably have):

  • Running only tests in the library: cargo test --lib
  • Running only doc-tests: cargo test --doc
  • Building tests in tests/ without running them: cargo test --no-run --test NAME, but you need to enumerate them yourself. Again it probably makes sense to add something to Cargo here.

In the mean time, integration tests are really separate crates that use your library as a dependency. You could make them explicit with Cargo.toml files and [dependencies] foo = {path = "…"} so that cargo test without arguments on your main crate doesn’t run them.

Lukas Kalbertodt
  • 79,749
  • 26
  • 255
  • 305
Simon Sapin
  • 9,790
  • 3
  • 35
  • 44
  • Good ideas, thanks! There's already an [open Cargo issue for running just the doc tests](https://github.com/rust-lang/cargo/issues/1789). Also, I tried the sub-crate idea, but it introduces too much friction when I'm developing. So until Cargo gets that new option to run just the doc tests, my Travis CI build runs only the `--lib` tests and skips both the integration tests and the documentation tests. In short, this problem reduces to Cargo getting that new option. – Craig M. Brandenburg Jan 16 '16 at 13:17