Questions tagged [mutation-testing]

A method of software testing that involves modifying the source code or compiled intermediate code (Java: byte code, .NET: IL). Test suites that do not fail under each of these "mutations" is considered to be inadequate.

Mutation testing is a method of testing the adequacy of test suites. By applying mutations to the code under test, the effectiveness of a test suite can be judged. If the test suite fails under this mutation, it is effective against it.

Different mutation types are used: Ma, Offutt and Kwon describe intra-method, inter-method, intra-class and inter-class mutations. The simplest to implement is intra-method mutation, for which some of the mutations that may be used are:

  • Statement deletion.
  • Replace each boolean subexpression with true and false.
  • Replace each arithmetic operation with another one, e.g. + with *, - and /.
  • Replace each boolean relation with another one, e.g. > with >=, == and <=.
  • Replace each variable with another variable declared in the same scope (variable types should be the same). (source: Wikipedia)
191 questions
2
votes
1 answer

What is coupling effect in mutation testing?

While I am learning about mutation testing, I've read in Wikipedia: The coupling effect asserts that simple faults can cascade or couple to form other emergent faults. Subtle and important faults are also revealed by higher-order mutants, which…
Hamza Belmellouki
  • 2,516
  • 1
  • 18
  • 39
2
votes
1 answer

Compile another Maven project using Maven Compiler API (CompilerMojo)

I'm working on a Maven project that injects and runs some tests in another Maven projects. I already managed to inject them, however, I'm struggling to programmatically compile the other Maven project given its path. I tried Java Compiler API and…
2
votes
2 answers

Ignore log entries in Stryker mutation test

I'm adding Stryker.net to my C# project. I see that it mutates on all log entries. Is there any way I can ignore these in the config?
runnerpaul
  • 5,942
  • 8
  • 49
  • 118
2
votes
1 answer

Configure multiple project files in Stryker.Net test

I have a Stryker test with this stryker-config.json: { "stryker-config": { "reporters": [ "progress", "html", "json" ], "log-level": "info", "log-file": true, "timeout-ms": 10000, "project-file":…
runnerpaul
  • 5,942
  • 8
  • 49
  • 118
2
votes
0 answers

`mutmut` Stops Running After About 100 Mutants

I'm posting this here to help out mutmut's dev solve this issue, because he doesn't know how to fix it yet. The issue has already been logged here. Basically, what happens is that when you run mutmut and tell it to run Mutation Tests for your…
2
votes
3 answers

What does removed call to "com.some.Filename::someMethodName" --> SURVIVED mean in pitest?

What does removed call to "com.some.Filename::someMethodName" --> SURVIVED mean in pitest. Does it mean that if that method call is removed, the code will still work properly?
Shruti Joshi
  • 301
  • 1
  • 4
  • 16
2
votes
0 answers

Pitest report-aggregate goal complains about transitive dependencies

I integrated Pitest according to https://pitest.org/aggregating_tests_across_modules/ in a Maven module project (Maven version 3.5.4). When running mvn clean package the log shows the Pitest execution for the target modules (i.e. the modules for…
Mark Schäfer
  • 925
  • 1
  • 12
  • 23
2
votes
2 answers

C# Stryker Mutation Framework

I am new to testing and writing unit tests. I came upon this mutation framework called Stryker. I liked it as I am writing better test cases with it which helps me cover all use cases. During some test cases mutation I come across Equality Mutation…
supernerd
  • 379
  • 4
  • 13
2
votes
1 answer

Is it possible to extract mutation testing results for every test method with Pit Mutation Test

I know that PIT Mutation Test framework can export mutation coverage information based on the test suite or the test class. However, I was wondering if there is an option to extract or export mutation coverage information based on the test case…
Ekin
  • 407
  • 1
  • 6
  • 17
2
votes
1 answer

Can I disable string mutation in mutmut?

I used mutmut yesterday to evaluate the unit test quality of some of my projects. I tried cosmic-ray before, but it was unusable (bugs, performance). I also tried mut.py, but I also couldn't get that one startd. One issue when using mutmut I found…
Martin Thoma
  • 124,992
  • 159
  • 614
  • 958
2
votes
2 answers

How compatible are rspec and heckle?

I'm currently using test/unit, and I'm considering using rspec. However, I've noticed that rspec currently doesn't support heckle in ruby 1.9.1, and doesn't support passing any parameters to heckle apart from the target module/class/method. Are…
Andrew Grimm
  • 78,473
  • 57
  • 200
  • 338
2
votes
2 answers

Mutation Testing for programs with an unknown output

Since the eventual aim of mutation testing is to detect faults with programs by finding test cases that would reveal the faults (by changing parts of the program and then verifying the output), it seems that only if the output is different for the…
rahs
  • 1,759
  • 2
  • 15
  • 31
2
votes
2 answers

Automatically recognize which mutants killed by which test cases in Pitest

I am using Pitest for mutation testing and I need lots of mutants for a project (for example 500 mutants). I need a matrix which shows which mutants that the Pitest has created, killed by which test cases. I can do it manually, but it takes a long…
2
votes
0 answers

IntelliJ PitTest plugin has a missing dependency?

I have installed PIT Mutation testing Idea plugin for IntelliJ. I have a small demo project in Java 8, it runs the mutation tests perfectly. Our larger project, which is still Java 7 consists of a parent with a lot of Maven modules. I'm not sure if…
JohannisK
  • 535
  • 2
  • 10
2
votes
1 answer

Issues running a custom PIT Mutation Engine Plugin

I'm writing a custom Mutation Engine for PIT, but I'm having troubles getting PIT to find it. My guess is that I'm failing to put the package with my own engine in the right classpath when running PIT using a project. To test my engine I'm using…