0

I'm looking at porting an old junit4 extension to junit5 by creating an extension. One feature that I'm thus far failing on is to fail any test method that writes on System.out or System.err, and have the failure point to that specific test.

I tried using TestExecutionListener#reportingEntryPublished together with junit.jupiter.extensions.autodetection.enabled=true, and I get to that method, but it is AFAICT too late to actually change the status of the test to FAILED at this point. Or can I?

I started playing with BeforeAllCallback and AfterAllCallback and do redirects of out/err and assert on a string buffer instead. I could possibly be able to get that to work, but that seems like something that I would be likely to screw up and and up finding bugs years from now :) Our runner for junit4 does something like this, but it is really overcomplicated and I wouldn't wish it on anyone to try to maintain something similar.

Any way to do this nicely in junit5?

chris_s
  • 1
  • 1
  • Have a look at https://github.com/stefanbirkner/system-lambda. And if you want to reimplement part of its functionality yourself, look at the code. – johanneslink Oct 04 '21 at 11:46
  • Thanks. That looks to do the same type of functionality, but if I read it correctly, then I have to add that to each test method. I want it to automatically assert all tests it runs without having to add code to each of the 6000+ testcases. Or can I accomplish this with system lambda? – chris_s Oct 04 '21 at 13:54
  • You could use global extension registration as described here: https://junit.org/junit5/docs/current/user-guide/#extensions-registration-automatic – johanneslink Oct 05 '21 at 05:54
  • Thanks again for the answer. I am a bit slow on the uptake here, are you saying I should make a BeforeEachCallback/AfterEachCallback-class that calls methods on SystemLambda? That is, I have managed to get my own extension to auto-register, so I have that already, I simply didn't figure out how to use system-lambda in that one yet. – chris_s Oct 05 '21 at 06:23
  • I think I have it working, but I wasn't able to reuse system-lambda, unfortunately. I'd accept your answer if I knew how :) – chris_s Oct 05 '21 at 08:13
  • I don’t know if there’s a straightforward way to reuse SystemLambda in your own extension; it’s probably not been conceived for that. Maybe you create an answer yourself with your extension‘s code and description how to register it globally. – johanneslink Oct 05 '21 at 08:28

1 Answers1

0

What I ended up doing was to create a class implementing BeforeEachCallback, AfterEachCallback and redirecting each of stderr/out during beforeEach to a custom printstream each that stores all the output received. Then during afterEach, I ask the printstream if it has received anything. If it has, then I throw AssertionError that the custom stream has generated based on a modified stacktrace from when the first print-call was made. Then resets out/err to the original streams. This allows for some tricks to allow the output to be printed to the console without throwing an error and still flag the testcase as failed with a custom message about disallowing output.

There are some potential pitfalls in this solution as e.g. an error in one of my other extension classes that tries to call Logger will create some fishy cases. For our use case, it should be sufficient.

Due to my employer having a lot of hoops to jump through to be allowed to share code externally, I'm unfortunately unable to do so for anyone else needing to do the same thing, but should be able to figure out how to do it decently based on the above.

chris_s
  • 1
  • 1