6

I have a Haskell package I've installed from Hackage, using cabal and would like to run the tests suites that are part of the package, but it isn't clear to me from the cabal documentation how to do this.

I've tried:

cabal install --reinstall --enable-tests --run-tests the-package

and its various combinations and permutations, but no tests seem to run: I get no report about the test running, and none of the output that I know the test should produce is generated.

How do I run the tests that are part of an installed cabal package, or a package that I'm in the process of installing?

recursion.ninja
  • 5,377
  • 7
  • 46
  • 78
orome
  • 45,163
  • 57
  • 202
  • 418
  • 1
    Maybe `cabal test the-package`? – recursion.ninja Oct 09 '15 at 14:29
  • @recursion.ninja: I tried that too but that seemed way off the mark: `Package has never been configured. Configuring with default flags. If this fails, please run configure manually. cabal: No cabal file found. Please create a package description file .cabal` – orome Oct 09 '15 at 14:31
  • You're right, it's not that simple. See my answer for a more robust solution. – recursion.ninja Oct 09 '15 at 14:40
  • 1
    There may be [a known bug](https://github.com/gelisam/hawk/issues/141). – orome Oct 09 '15 at 15:36

2 Answers2

6

The --run-tests flag does not appear to be working in the current version of cabal. The --enable-tests flag no longer runs tests as a new feature of cabal. Until the issue is resolved you can manually verify that a package passes it's test suite by doing the following:

  1. Use cabal to download the package source
  2. Use cabal to build the package in a sandbox
  3. Use cabal to run the tests in the sandbox

Use this series of cabal commands to run the test for the-package:

cabal get the-package
cd the-package*
cabal sandbox init
cabal install --dependencies-only
cabal configure --enable-tests
cabal build
cabal test
cd ../
rm -r the-package*

Or use this equivalent one-liner:

cabal get the-package && cd the-package* && cabal sandbox init && cabal install --dependencies-only && cabal configure --enable-tests && cabal build && cabal test && cd ../ && rm -r the-package*
recursion.ninja
  • 5,377
  • 7
  • 46
  • 78
  • Yes, that part I get. But is there no way to simply run the tests on an installed package *in situ*? What do `install --enable-tests` and `install --run-tests` even do — nothing? – orome Oct 09 '15 at 14:40
  • @raxacoricofallapatorius Are you *sure* the package has a test suite in its `.cabal` file? I `cabal install --run-tests` on a package (`tasty`) which I know has no test suite associated with it and it did not output an failure/warning message that there were no test to run. It just silently finished. – recursion.ninja Oct 09 '15 at 14:46
  • Yes, [it's there](https://hackage.haskell.org/package/crypto-enigma-0.0.2.0/src/). – orome Oct 09 '15 at 14:48
  • I just tried with `cabal install megaparsec --run-tests` [which also has a test suite](https://github.com/mrkkrp/megaparsec/tree/master/tests) and it *also* finished without running tests. Very strange, maybe a `cabal` bug? – recursion.ninja Oct 09 '15 at 14:52
  • Yeah, maybe. Certainly nothing even remotely like what's described in the documentation is happening. – orome Oct 09 '15 at 14:55
  • My answer should give you a method to run the tests independently and verify that they are all passing, but according to [this GitHub issue conversation](https://github.com/haskell/cabal/issues/1647) the `--run-test` flag should make the independent build & run unnecessary. – recursion.ninja Oct 09 '15 at 15:00
  • Can I do the with no side effects. Just delete the directory after? – orome Oct 09 '15 at 15:08
  • Yup, just add a `cd ../ && rm -r the-package*` command at the end. The sandbox is in the directory. Delete the directory and the sandbox install is gone! – recursion.ninja Oct 09 '15 at 15:16
  • I had saw that related issue. I hope this is resolved in the next version of `cabal`. They should really prioritize this fix because *verifying an installation* is a pretty desirable function of *any* package installer. – recursion.ninja Oct 09 '15 at 15:44
  • Agreed! It's too bad, really. I'm just learning Haskell, and there are a lot of things about it and its ecosystem that are pretty (surprisingly) much in disarray (compared with, say, Python). – orome Oct 09 '15 at 15:48
  • This answer is outdated since `sandbox`es have disappeared from `cabal`. I don't know a good answer myself, see https://github.com/haskell/cabal/issues/7267. – Andreas Abel Aug 27 '21 at 08:06
0

The answer below no long works as noted in its comments. The following works for me with cabal 3.6.2:

cabal get the-package
cd the-package*
cabal v2-test
George Co
  • 961
  • 6
  • 9