Questions tagged [mutation-testing]

A method of software testing that involves modifying the source code or compiled intermediate code (Java: byte code, .NET: IL). Test suites that do not fail under each of these "mutations" is considered to be inadequate.

Mutation testing is a method of testing the adequacy of test suites. By applying mutations to the code under test, the effectiveness of a test suite can be judged. If the test suite fails under this mutation, it is effective against it.

Different mutation types are used: Ma, Offutt and Kwon describe intra-method, inter-method, intra-class and inter-class mutations. The simplest to implement is intra-method mutation, for which some of the mutations that may be used are:

  • Statement deletion.
  • Replace each boolean subexpression with true and false.
  • Replace each arithmetic operation with another one, e.g. + with *, - and /.
  • Replace each boolean relation with another one, e.g. > with >=, == and <=.
  • Replace each variable with another variable declared in the same scope (variable types should be the same). (source: Wikipedia)
191 questions
8
votes
4 answers

Off By One errors and Mutation Testing

In the process of writing an "Off By One" mutation tester for my favourite mutation testing framework (NinjaTurtles), I wrote the following code to provide an opportunity to check the correctness of my implementation: public int SumTo(int max) { …
pms1969
  • 3,354
  • 1
  • 25
  • 34
7
votes
7 answers

How can I perform mutation testing of my Java program?

Can anyone provide suggestions of tools that can be used to make mutations within a Java program at a source-code (not byte-code) level? I need to seed my source code with faults. I would prefer an application with a GUI, if one exists.
Konstantin Milyutin
  • 11,946
  • 11
  • 59
  • 85
7
votes
1 answer

Run Pitest from the command line

According to Pitest's documentation, it seems that this should be simple, but it is giving me some trouble. I should be able to have java -cp \ org.pitest.mutationtest.commandline.MutationCoverageReport \ --reportDir…
DeadEli
  • 983
  • 2
  • 13
  • 28
6
votes
3 answers

Making unit tests fail quickly for mutation testing

One problem encountered with mutation testing is that it's slow, because by default you do a full test run (either a test file, or a suite of test files) for each mutation generated. One way to make mutation testing faster would be to stop the test…
Andrew Grimm
  • 78,473
  • 57
  • 200
  • 338
6
votes
1 answer

Mutation Test tools in C#

I am starting a new project in c# and I am looking for a tool that helps me do mutation tests. I have previously done mutation testing in java using pitest and liked it very much. Now I am looking for such a tool for C# and the only ones i can find…
ziraak
  • 131
  • 3
  • 14
6
votes
2 answers

Finding useless unit tests with PIT

Assume we have a code we'd like to test: class C { int doSmth() { return 1; } } Now assume we have 2 unit tests placed within a single class. The 1st one "test everything" while the 2nd one "does…
Bass
  • 4,977
  • 2
  • 36
  • 82
6
votes
1 answer

Sonar Pitest Plugin

I want to integrate some mutation testing to ensure the quality of my junit tests. I want to have the results in the sonar dashboard of my project. The sonar pitest plugin seems to do what I want, but there are some issues with maven 3 and it is…
gontard
  • 28,720
  • 11
  • 94
  • 117
5
votes
2 answers

pit mutation - if ( x !=null ) return null else throw new RuntimeException

I have a method that returns a custom object public MyObject getTheObject(){ ... return muObject; } its unit test checks that the object returned by getTheObject() method is not null @Test public void testGetTheObject(){ ... …
Toseef Zafar
  • 1,601
  • 4
  • 28
  • 46
5
votes
1 answer

Proper setup for pitest-maven report-aggregate goal

folks! I've tried to use the pitest-maven plugin in my Maven / Java project and it is apparently failing to generate an aggregated report (taking into consideration that I have a multi-module project). I gather some information from the official…
Fernando Barbeiro
  • 762
  • 4
  • 15
  • 33
5
votes
1 answer

How to exclude tests from PIT mutation analysis?

I'm in a drawn-out situation now with a difficult project where a significant part was refactored and checked in with a significant number of failing tests. I run PIT in maven but I haven't been able to for a long time because the tests have to be…
Adam
  • 5,215
  • 5
  • 51
  • 90
5
votes
3 answers

What programming languages can support mutation testing?

Is it harder (or impossible) to implement mutation testing in some languages than others? For example, is it possible to implement mutation testing in functional programming languages?
Andrew Grimm
  • 78,473
  • 57
  • 200
  • 338
5
votes
2 answers

Open source mutation testing c++

I need an open source tool (even a relatively primitive one will do) which performs Mutation Testing on C++ code. I require it to be open source as I require to modify it in a proof of concept experiment. I tried Googling it but did not come up with…
cachiama
  • 590
  • 3
  • 19
4
votes
5 answers

What's the difference between regression testing and mutation testing?

Just wonder what the difference is. I need some concise explanation.The wikipedia is a bit too verbose for me.
Hongxu Chen
  • 5,240
  • 2
  • 45
  • 85
4
votes
0 answers

PIT test says Minion exited abnormally due to TIMED_OUT

I have added mutation test plugin to Springboot app's pom. When I do mvn clean install, build passes but it is taking too long to complete install. Below are the logs and configuration for pit tests. I have same config for other apps and I don't…
4
votes
1 answer

Mutation Testing - Negated conditional on a for-each loop?

we are just playing around a bit with mutation testing and there is one thing I don't understand. Why is it always trying to apply the "Negated Conditional Mutator" on my for-each loops like this one: for (final Order order : orders) If I take a…
Core_F
  • 3,372
  • 3
  • 30
  • 53
1
2
3
12 13