4

I want to perform integration tests on a command line program that is supported on multiple platforms.

For example, if I am writing a mercurial plugin and want to test it from the top level, how could I create a test suite which will run on Windows, Linux, and OS X?

Alternatively, should I just write tests that use the highest level functions in the code and just perform basic sanity checks on the final program?

Philip
  • 4,128
  • 5
  • 31
  • 49

2 Answers2

3

We write tests like these in Groovy or Python. Since the Python and Groovy runtimes are largely platform independent you should be able to exec commands on all three OS's. A few parameters might have to change per platform, but you can do an OS check and set those at the start of the script. There are frameworks that simplify the running of the tests like JUnit and Spock for Groovy and Robot for Python, but they just abstract the normal frameworks. I'd start simple. It's Agile to try the simplest thing that could possibly work.

As to you second question I might do both. First I would start writing tests for everything. Then, if it became expensive to run all the tests (let's say more than a couple minutes. I would separate them into Smoke (sanity) and Functional (everything) Tests and run the Functional Tests less often.

jeremyjjbrown
  • 7,772
  • 5
  • 43
  • 55
  • Thanks! Using a scripting language and exec makes sense. Are there any test frameworks designed for spawning processes and testing them this way or do you just use the frameworks that are already popular in the python and groovy communities? – Philip Jan 29 '14 at 04:34
2

Mercurial's own test suite is cross platform and already tests many Mercurial extensions. Maybe, just add more tests to it.

Ry4an Brase
  • 78,112
  • 7
  • 148
  • 169
  • Thanks for the suggestion. Mercurial uses the [cram](https://pypi.python.org/pypi/cram) testing framework, which is available standalone at the link. Also, mercurial's [Windows test plan](http://mercurial.selenic.com/wiki/WindowsTestingPlan) indicates that it is not fully cross platform, but most tests work. – Philip Jan 30 '14 at 04:53
  • 1
    Yeah, I think the only tests that don't run are filesystem specific trickery like symlinks, which don't really exist on windows, and hardlinks, which are fraught w/ problems on windows if used on network drives, etc. – Ry4an Brase Jan 30 '14 at 14:39