Questions tagged [mutation-testing]

A method of software testing that involves modifying the source code or compiled intermediate code (Java: byte code, .NET: IL). Test suites that do not fail under each of these "mutations" is considered to be inadequate.

Mutation testing is a method of testing the adequacy of test suites. By applying mutations to the code under test, the effectiveness of a test suite can be judged. If the test suite fails under this mutation, it is effective against it.

Different mutation types are used: Ma, Offutt and Kwon describe intra-method, inter-method, intra-class and inter-class mutations. The simplest to implement is intra-method mutation, for which some of the mutations that may be used are:

  • Statement deletion.
  • Replace each boolean subexpression with true and false.
  • Replace each arithmetic operation with another one, e.g. + with *, - and /.
  • Replace each boolean relation with another one, e.g. > with >=, == and <=.
  • Replace each variable with another variable declared in the same scope (variable types should be the same). (source: Wikipedia)
191 questions
3
votes
1 answer

Is it good practice to perform mutation testing on user interfaces?

My teammate and I are having an hard time understanding if mutation testing can / should be done on JUnit tests referring to Swing classes. More precisely, we are currently using Junit 4 and Assertj Swing to test the interface for an assignment on…
leoScomme
  • 109
  • 1
  • 1
  • 11
3
votes
1 answer

How to refactor survived mutations away for try-resource and an equivalent mutant?

I am experimenting with junit5 and pitest. My code under test looks like: // [...] InputStream istream = this.getClass().getResourceAsStream("/" + file.getName()); if (istream == null) // 1. negated condition -> suvived { istream =…
PowerStat
  • 3,757
  • 8
  • 32
  • 57
3
votes
1 answer

Generate mutant for python program

Existing mutation testing tools for python like Cosmic Ray or Mutpy seem not to store generated mutants. I want to find a tool which can generate and store the mutants for python program because I need the mutants. Does this kind of tool exist?
3
votes
1 answer

Finding bugs in code using mutation testing

I have some trouble understanding how to find a bug using mutants. So, there is the original code, I make mutants, and check for reachability, infection and propagation, find tests which kill mutants (if they exist), and than what? How should that…
wdc
  • 2,623
  • 1
  • 28
  • 41
3
votes
1 answer

Excluding certain lines of code in PITest

I am using the excellent PITest framework. I was wondering if there is something equivalent to Sonars "// NOSONAR" in PITest, whereby certain lines just get excluded from PITest coverage (so it's not red on the report)? I know methods and classes…
Nos
  • 1,024
  • 1
  • 8
  • 14
3
votes
1 answer

How do we activate and use non default deactivated mutators in Pitclipse?

I was using PITclipse on Eclipse to run PIT mutation tests on code. I noticed in the PIT documentation where it stated that the some of the mutators are not activated by default. Even in PITclipse these mutators do not seem to be used. I wanted to…
nitinkaveriappa
  • 91
  • 2
  • 10
3
votes
1 answer

Can I get a quality gate to fail if mutation coverage is not high enough?

We are using the default 'SonarQube way' quality gate, which has a 'Mutations Coverage' setting, however this does not seem to cause the gate to fail. For example, I have a small test project that does not have enough mutation coverage, and reports…
John Davis
  • 51
  • 2
3
votes
1 answer

How can I perform mutation tests on an F# codebase?

What options are available for me to execute mutation tests on an F# codebase? For example, are there any frameworks available?
Scott Nimrod
  • 11,206
  • 11
  • 54
  • 118
3
votes
5 answers

JUnit report to show test functionality, not coverage

One of the problems of a team lead is that people on the team (sometimes even including myself) often create JUnit tests without any testing functionality. It's easily done since the developers use their JUnit test as a harness to launch the part…
Adam
  • 5,215
  • 5
  • 51
  • 90
3
votes
1 answer

Is there any test framework for Drools mutation testing

I'm looking for test framework which will perform mutation testing for code written in Drools, i.e. it should check if there're tests that fails when one of the rules is removed from KnowledgeBase. It's needed for confidence that every rule is…
3
votes
5 answers

Mutation Testing

Can mutation testing be successfully used to improve TDD from a defect reduction perspective?
user228584
  • 31
  • 1
3
votes
2 answers

Mutation testing in cpp without modifying/recompiling the code

Currently I have some tests that are blamed to not catch bugs very well. I want to do mutation testing in order to detect them (and prevent from adding new useless ones), but without the time-inefficient loop: change the code -> recompile -> run…
Adam
  • 316
  • 2
  • 11
2
votes
1 answer

Possible class loader problems with java jumble mutations testing framework

This question is regarding the jumble mutation testing framework (http://jumble.sourceforge.net/) i am trying to get jumble into the projects i work on my job. i've run jumble on the classes of one of this projects and it performs the mutations on…
2
votes
0 answers

How to exclude specific methods and classes from pit mutations (intelliJ)

I have a little java project with Test. I see two options for PIT tests: 1.) Pittest all classes in modulo 2.) Pittest one single file Now I have two problems, firstly, when I run the pittest, it also mutates the toString() Method, which is for…
capoaira
  • 89
  • 1
  • 1
  • 7
2
votes
0 answers

npm run stryker fails Error: Cannot find module 'mutation-testing-report-schema/dist/src/api'

Stryker started failing because of unloaded module 'mutation-testing-report-schema/dist/src/api'. I tried deleting node_modules and npm clean install. Here is my dependency list: "@stryker-mutator/core": "^1.2.0", "@stryker-mutator/html-reporter":…