21

I'm relatively new to unit testing, and I've discovered that while many sources say that you should write unit tests, few of them give any indication where to put them in your project.

The only suggestions that I've seen are to put them alongside the production code or to build a directory structure that mirrors your production code. The problem I see with the first approach is that it would become difficult to extract those tests when you make a full build of the system; for the second, how would you test functionality that is only visible at the package level?

Is there a decent way to organize unit tests so that you can easily automate the build process and still access everything that needs to be tested?

EDIT: For those wondering about a specific language, I'm mostly working in Java. However, I'd like to find a solution that's relatively language-agnostic so I can apply it regardless of what platform I'm working on.

derekerdmann
  • 17,696
  • 11
  • 76
  • 110
  • I store tests in their own directory, and have a `make test` target in my Makefile. But from your question I gather your project management practices aren't similar to mine. – jmz Aug 04 '10 at 13:06

6 Answers6

6

You don't indicate the platform and tools you're using and that has impacts on the options. I'm answering from a .NET and Visual Studio perspective:

My team (working on a quite large project) has successfully used the approach of placing code and tests together. For a given (sub)namespace in the solution, there is a "_Test" folder inside that (sub)namespace, ("_" being a hack to get Visual Studio to display it above everything else in that namespace in the solution browser).

I was introducing TDD on the team and felt that minimum friction in going between tests and code was crucial to the success of adopting the practice. Hence, the tests are always "physically near" the code it tests; you don't want to seek back and forwards in a huge solution to jump between code and tests.

Which makes sense when you think about it. Code and test belong together like yin and yang. That also means only 1 assembly needs to rebuilt when modifying types in 1 assembly (we have well over 30 projects in our solution so this enables us to safely get away with a Build Current Project only when evolving a class via TDD, way faster than a full Solution build).

Having code and tests together also solves the issue of being able to test internal classes. However, on the .NET platform you might also use the InternalsVisibleTo Assembly-level attribute to grant access to a separate test assembly if you want to keep things separate.

The only reason for having tests in separate assembly that I can see is to avoid shipping that code. However, that aspect is trivially solved in another way, if you're using build automation. As Visual Studio's project files are XML, we simply have the build server pre-process the project files, using a small XSLT, to strip everying in a */_Test/* folder from compilation in Release builds.

Cumbayah
  • 4,415
  • 1
  • 25
  • 32
  • Another way to remove test code from the release build is to add a Conditional attribute to the test methods. `[Conditional("TEST")]` – Jeffrey L Whitledge Aug 04 '10 at 14:19
  • 1
    True, or an ugly #if preprocessor construct around the entire test fixture, but that would have to be done manually for each test. We have around 4000 of those; if that would have to be done manually, someone would surely forget it somewhere. By having a convention (the Test subfolder in our case), it is trivial to remove the human factor and automate here. Automation is good. :) – Cumbayah Aug 05 '10 at 11:46
5

The second approach is best. Create a parallel source tree for your tests. Set up your package structure the same. You don't say what language you're using, but if it's Java then code in the test source tree will be able to access code in the main source tree just fine. The fact the the packages have some code from one source tree and some from another is irrelevant - the production code and the test code are still in the same package.

dty
  • 18,795
  • 6
  • 56
  • 82
4

The problem I see with the first approach is that it would become difficult to extract those tests when you make a full build of the system

You could always filter classes based on some naming convention like a special suffix or prefix when packaging (e.g. don't include **/*Test.class with Java). I don't really like this approach though, I find it messy and fragile.

for the second, how would you test functionality that is only visible at the package level?

Unless I'm missing something, I don't see the issue. You CAN have classes in the same package and have them living in different trees, e.g.:

  • src/main/java for application sources
    • src/main/java/foo/Bar.java
  • src/test/java for tests sources
    • src/test/java/foo/BarTest.java

Both Bar.java and BarTest.java are in the same foo package but in different directories. This is the approach I use.

Is there a decent way to organize unit tests so that you can easily automate the build process and still access everything that needs to be tested?

Well, as hinted, my test sources and application sources live in the same project but in distinct directory trees and this works really well for me.

This is actually the default layout suggested by Maven (see the Introduction to the Standard Directory Layout). Whatever language you use, this might give you some ideas.

Pascal Thivent
  • 562,542
  • 136
  • 1,062
  • 1,124
1

What is a Test, and what is an application?

Lets think about how a test fits into the picture...

  • A test is not directly applicable to anything the public is concerned about -- they just want it to work
  • A test is for developers and usually a business
  • Tests can be on the same scale as modules, but are in dichotomic disposition
  • If test files can exist side-by-side with its module in a given encapsulating structure, then logically -- in a fractal architecture -- a fraction of a test must also be distributed at the object scale within a given module. (Think, directory|_file|_module|_object -- where system == component && component == system && (system == system || system == subsystem ))

That last one is to convey that tests are intrinsically discrete from modules and logically do not complement each other on every scale -- to really understand this, I urge you to study fractals & self-symmetry, Type Theory and Category Theory.

However, application-specifics can drastically alter what options you have...

Coming from a JavaScript perspective, an agnostic one, and even with using an advanced architectural scheme -- it seems that including your tests within the /src has a potential to complicate your build process. Furthermore, you're creating a less readable logical-bound for other developers when there are multiple, much less lots of, tests within your src directory. You do want to protect and encapsulate you tests, privatizing them to your team members and not anyone interacting with your app publicly.

That said, it still may be worth it to go down this path based upon app specifics -- IF -- you can ensure that it is secure & doesn't create unnecessary complexity, or stifle automation.

Some advanced architectures utilize this approach for purposes of comprehensive encapsulation on discrete scales, but it still may come at a cost to security, automation, and productivity (hence to the business).

As a rule of thumb, use the industry-standard scaffolding:

/app
    |-lib
    |-src
    |-test

.. you probably can't go wrong with this.

Community
  • 1
  • 1
Cody
  • 9,785
  • 4
  • 61
  • 46
1

I suggest you have a separate project (or projects) for your unit tests.

It is also a good idea to have a folder on your project tree just for the test projects. This way if you need to build your system without the test you should have a build configuration without building the tests although it is a good idea to build the tests and run them as part of your CI (continuous integration) and than only package the production code in an installer or copy it to a specific folder.

Testing functionality that is only visible on package level is possible depending on the technology used - of example .NET have the ability to make internal methods and classes visible by using InternalVisibleTo attribute. You should ask yourself if you need to test that functionality, it is not advisable to test private methods - because they tend to change.

Dror Helper
  • 30,292
  • 15
  • 80
  • 129
  • What would be the advantage of using a separate project over separate directory trees within the same project? – derekerdmann Aug 04 '10 at 19:33
  • The benefit is that you do not have unit tests as part of your production code. – Dror Helper Aug 05 '10 at 07:29
  • A valid concern which, however, can be trivially solved by other means, cf. my answer. In my experience, having separate projects causes friction when working in the TDD cycle, always going between test and code. – Cumbayah Aug 05 '10 at 11:49
1

It probably depends on what you are now using. With visual studio / .net, common practice seems to be a test dll for each production dll, and a vague mirroring of classes and test-classes within the dlls. For ruby/python etc, a seperate test directory, and within that a hierarchy that mirrors the things being tested.

What do you mean by "extracting" those tests? For you does a full build involve compiling and running the tests or not?

Visibility at the package level could be a concern. I suppose if your tests are in separate packages, you only need tests for the things directly visible but that will indirectly test the internal stuff anyway.

Kurt
  • 4,477
  • 2
  • 26
  • 34
  • By "extracting," I mean that the code for the unit tests doesn't need to be included when building the final project. Of course they would be run when the build is made; I just don't want them in the build itself. – derekerdmann Aug 04 '10 at 13:05