2

It struck me that I do not really know of a way to black box test an executable packaged with Cabal.

With npm, for instance, I can run arbitrary shell commands, and I surely can wire it so that the necessary sources are transpiled and executed, and their side effects inspected.

Stack (as said here) builds the executables and publishes them in $PATH for the test suite, so I can easily run them.

But with Cabal, a test suite apparently cannot even depend on an executable, so there is no way to force the latter to be built. (Am I wrong about this?) And even then, I would have to know the path to the compiled binary.

How do I approach this problem?

The particulars of my situation are that an executable must extensively analyze the state of the system and branch accordingly, and I want to integration test that it does not forget to do so.

Note also that I am not at peace with running the relevant IO functions directly because I find it not integrative enough. Or, rather, I would like it to be possible to run the individual IO functions and also run the program as a whole. In my case, there are testing shell scripts in place already, but I would really like to "bake them in".

Ignat Insarov
  • 4,660
  • 18
  • 37
  • I don't think cabal natively supports this: https://github.com/haskell/cabal/issues/5415. However, you can easily move all of your code (including `main`) to an internal `library` stanza (https://github.com/haskell/cabal/pull/3022) and then manually run the same `main` function as the executable runs. You could also just add all of the sources for the executable to `hs-source-dirs`, and then everything would be built twice---once for the actual executable and once for the test---but importable from the test suite. I would recommend the convenience library approach. – Lucy Maya Menon Jul 07 '18 at 17:12
  • @PeterAmidon It so happens that it is I who has filed this ticket just a few hours ago. I guess I should clarify the question. – Ignat Insarov Jul 07 '18 at 17:14
  • Oh, my apologies. I think that what you say in the ticket is correct, I just used it a as a convenient summary. Are you just looking for a more fleshed out version of one of those general approaches which should work for your use case? – Lucy Maya Menon Jul 07 '18 at 17:16
  • @PeterAmidon No apologies needed at all. I added a note to the question that, hopefully, clarifies this aspect. Moreover, that you approve of the information I gathered about testing the `executable` part of a package is heartwarming. My question is primarily whether it is possible to run the complete, built executable, so as to be completely confident that it does what it says on the box. If not, I will simply file another issue to Cabal. – Ignat Insarov Jul 07 '18 at 17:23
  • Aha, I understand now. I think there may be a way to do this if you don't mind a second (mostly dummy) cabal project, but am looking into it. – Lucy Maya Menon Jul 07 '18 at 17:38

1 Answers1

4

It turns out that there is a (slightly hacky) way to do this, at least for now, using the new(ish) build-tool-depends Cabal field. There has been some discussion (https://github.com/haskell/cabal/issues/5411, https://github.com/haskell/cabal/pull/4104#issuecomment-266838873) of build-tool-depends only being available at build-time, and having a separate field for executables that should be available when running a component. However, this separate run-time tool depends field doesn't exist yet. Luckily, it seems like Cabal (at least 2.1 and 2.2) completely doesn't draw this distinction: executables listed in build-tool-depends are actually available when cabal new-test runs a test suite. This means that you can use a pkg.cabal file that looks like this:

name: pkg
executable exe
  ...
test-suite test
  ...
  build-tool-depends: pkg:exe

And when you run the test suite, the executable will be built & on the path.

Lucy Maya Menon
  • 1,560
  • 8
  • 15