0

I am building a CD pipeline. I am planning about the automated testing piece of it. I plan to do UI, WebService, Security, Perf testing. I have a question about the code structure. So what I plan is to have the tests in the same repo as the code and then have separate repos for the core test frameworks e.g.

Repo Product

  • Product Code (Project)
  • Integration Tests (Project)
  • Functional/e2e Tests (Project)
    • UI Tests (Package)
    • WebSvc Tests (Package)
    • Perf Tests (Package)
    • Sec.Tests (Package)

Repo Test Core

  • UI Test Framework Code (Project)
  • WebSvc Test Framework Code (Project)
  • Perf Test Framework Code (Project)
  • Sec Test Framework Code (Project)

Does anyone see any problem with this structure? Any other ideas? Also I am little fuzzy about the what goes in the integration tests vs in the functional tests projects (e.g. WebSvc tests can be a part of both). And where does acceptance tests go (Functional or integration) ? It will be great if someone can point to some example repos or articles on this.

Thanks

user2666282
  • 381
  • 1
  • 2
  • 15
  • Side note: I would assume that *acceptance tests* come from your business people; thus you probably want to make their life as easy as possible. Besides ... I am not so sure if SO is a good place for such a broad question. Especially the question for off-site resources is definitely "off topic" for stack overflow. So don't be surprised if downvotes come in. – GhostCat May 12 '16 at 11:24
  • Thanks - can you explain what you mean when you say make their life easy? Do you mean have their own project? – user2666282 May 12 '16 at 11:34
  • I mean: if you asking non IT-people to make a significant contribution to an IT project (and that is what "customer acceptance tests" are essentially about) ... then simply try to understand their skills and needs. Ideally, the deal with one platform only, and all they need to their job is there. No complicated setup required ... stuff like that. – GhostCat May 12 '16 at 11:48
  • Sure - but those tests need to run somewhere - correct? And I guess the business people will only care about the result reports. – user2666282 May 12 '16 at 12:01

1 Answers1

3

I find this structure somewhat irritating.

From the proposed structure I deduct that you want to build your own test frameworks. That sounds fishy to me especially when you want to write 4 of them.

On the other hand you put them all in the same repository, so they seem to be closely related. Again: not necessarily bad/wrong, but really unexpected.

Since apart from the structure I can't find a hint in your question, that gives a good reason to have separate repositories, so I would recommend to use just one, assuming your "testframeworks" are just utilities to test your main project.

The basic rule is things that change together should go together (in one repository). Everything else makes development really cumbersome: change A, install, change B, run, debug, repeat instead of change, run, debug, repeat

Since you mentioned you are not completely clear, what will go where, I recommend the following:

Start with a single project. Write ALL tests in the test directory of that project. Observe if you experience problems. If so adapt. Things you might experience, that trigger extraction of projects:

  • Tests run slow and you want to run them separately
  • Tests need a deployed application, so they should run after building and installing everything else
  • Tests in different modules need access to code that should not live in the main projects so it might end up in a module with test supporting code
Jens Schauder
  • 77,657
  • 34
  • 181
  • 348
  • Thanks Jens. Firstly why do you think its fishy to write 4 frameworks (given they test different things)? Also what I get from your suggested code structure is each of the 4 frameworks should go in their own repos - correct? But then when you say one repo I am a little confused as I want to keep the frameworks separate (because they will be needed to test other projects too). – user2666282 May 12 '16 at 11:22
  • 4 testframeworks sound fishy to me, because a) there are hundreds of frameworks out there. Why write a new one? b) writing 1 framework is a lot of work. Are you sure you can handle 4? If you're going to write 4 independent frameworks, they should have each have their own repo. Except when they are closely related. – Jens Schauder May 12 '16 at 11:35
  • Got it - Apologies if I communicated it wrong, by frameworks I don't mean Selenium (say for UI tests) - it would mean the customizations that we need, on top of it, to make it work for our projects. It will be a layer on top of the existing frameworks. Please let me know if you've any other suggestions. – user2666282 May 12 '16 at 11:41