0

we have a lot of regression tests, which are running on each nightly product build. can anybody advise a system to store regression results?

in ideal we want simple tool with API and ORM to store simple records like:

test_case1 | build_#1 | ok | timestamp1<br>
test_case2 | build_#1 | failed | timestamp2<br>

and so on or may be something similar.

seems that there is main problem for test teams and so, I believe, that there is already created tool to store regression testing results and ability to generate reports using stored data. Thanks!

Suraj Singh
  • 4,041
  • 1
  • 21
  • 36
user3057662
  • 1
  • have you considered working with Jenkins? – ekeren Dec 02 '13 at 14:10
  • not all of our tests are running from Jenkins. we use Jenkins to run test suites but we want to store results of test methods.. so testng will report test method result to any 'storage' using API. – user3057662 Dec 02 '13 at 14:15
  • I see, and for Jenkins tests... are you satisfied with how it is stored? and How data is viewable for you? – ekeren Dec 02 '13 at 14:18
  • we don't use stored data in Jenkins.. Jenkins is only launcher and it's right to use it only in this way.. store data should another system – user3057662 Dec 02 '13 at 14:27

1 Answers1

0

This depends on in which environment you are running the tests, I am not aware of any pre-built framework that does this for you. Currently at work we are logging the test results to a Hive table in Hadoop, which works very well since we are already in a Hadoop environment when running the tests. It allows us to go back and do SQL queries to see how the build failures have changed over time and to see how long the tests are running for etc.

enter image description here

TLDR; I can recommend the practice of logging test results, it is useful. I am not aware of any framework designed just for this purpose, but you can easily implement it yourself. Which storage system you should use depends entirely on what is most convenient in your particular test environment.

Limmen
  • 1,407
  • 11
  • 17