Let's say I wanted to compare the levels of test coverage for projects before and after implementing an automation framework.
I collected metrics for all projects in regards to test coverage.
Would it just be avg. test coverage for projects before vs. avg. test coverage for projects after implementing the automation framework?
Would I need to weight it?
How would this be best achieved?
I'll assume you mean line coverage.
Line coverage for a project is the number of covered lines divided by the number of coverable lines.
The most sensible measure of coverage for a set of projects is the same: the total number of covered lines in all projects divided by the total number of coverable lines in all projects. That weights the coverage for each project by the number of lines in the project. (Averaging coverage for each project would ignore the number of lines in each project.) This makes sense given that the number of bugs per uncovered line is probably roughly constant across projects.
You could do that.
I think you would be better off to focus on test coverage of new implemented features. The old existing code presumably works "well enough" or you wouldn't already be shipping it.
What you should do is to identify the code that has changed from the old version of the application to the new, and measure test coverage on that.
That will be quite difficult enough to get started. But it will force people to test the new features.
You can ease in test coverage over the whole application over time, but expect it to be very disappointing in terms of coverage percentage when you start.
