Questions tagged [mutation-testing]

A method of software testing that involves modifying the source code or compiled intermediate code (Java: byte code, .NET: IL). Test suites that do not fail under each of these "mutations" is considered to be inadequate.

Mutation testing is a method of testing the adequacy of test suites. By applying mutations to the code under test, the effectiveness of a test suite can be judged. If the test suite fails under this mutation, it is effective against it.

Different mutation types are used: Ma, Offutt and Kwon describe intra-method, inter-method, intra-class and inter-class mutations. The simplest to implement is intra-method mutation, for which some of the mutations that may be used are:

  • Statement deletion.
  • Replace each boolean subexpression with true and false.
  • Replace each arithmetic operation with another one, e.g. + with *, - and /.
  • Replace each boolean relation with another one, e.g. > with >=, == and <=.
  • Replace each variable with another variable declared in the same scope (variable types should be the same). (source: Wikipedia)
191 questions
4
votes
0 answers

How bad is it to leak AppDomain?

I am using AppDomain to isolate assembly loading and in some cases, the AppDomain.Unload(customDomain) never return (I suspect this is due to some thread usage). I plan to put the AppDomain.Unload call in a separate Task, time it, and if nothing…
shorty_ponton
  • 454
  • 4
  • 14
4
votes
5 answers

Pitest can't detect class of test

i have problem with my configuration of maven and pitest. Pitest generation mutation is ok but he can't see my class of test .. if you have any solution :D I have main source like that /src/main/java/com.plugin..... .java I have test source…
Modship
  • 57
  • 1
  • 6
4
votes
0 answers

PIT mutation testing: Coverage generation minion exited abnormally

I use pitest with Maven in Jenkins. In my pom.xml I have org.pitest pitest-maven 1.1.8
Amedee Van Gasse
  • 7,280
  • 5
  • 55
  • 101
4
votes
1 answer

Understanding Mutant Failures

I have the following ActiveRecord model class method: def self.find_by_shortlink(shortlink) find_by!(shortlink: shortlink) end When I run Mutant against this method, I'm told there were 17 mutations and 16 are still "alive" after the test has…
pdoherty926
  • 9,895
  • 4
  • 37
  • 68
4
votes
1 answer

PIT Mutation testing on apache commons math shows line coverage and mutation coverage as 0%

I am trying to use PIT Mutation testing with maven for apache common math tests. org.pitest pitest-maven 0.29
Biparite
  • 203
  • 1
  • 2
  • 13
4
votes
1 answer

Mutation testing: does anybody know mutant?

Yesterday a stumble in this article on Mutation Testing and immediatly had a try with elasticrepo a lib I'm writing to index github on elasticsearch. Unfurtunatly it seems I'm missing something, because I don't get any so called "Killed: rspec"…
Luca G. Soave
  • 12,271
  • 12
  • 58
  • 109
4
votes
1 answer

Mutation testing ActiveRecord models with the mutant gem

I've been trying to get started with mutation testing for our rails app, but haven't been able to get it working - has anyone managed to set this up? Here's what I've tried so far: mutant -I app/models -r foo "Foo#bar"…
Mikey
  • 2,942
  • 33
  • 37
4
votes
1 answer

Creating an IL instruction with an inline argument using Mono.Cecil

I'm playing with mutation testing at the moment. One of the mutations I'm looking at involves swapping parameters, for which I may need to swap, for example Ldarg.0 and Ldarg_S with an operand indicating the index. The operand type for this is an…
David M
  • 71,481
  • 13
  • 158
  • 186
3
votes
1 answer

C# Reflection: Replace a referenced assembly

I am currently writing a framework for MutationTesting. The code is almost complete, but there is a tiny bit which (after spending half a day on it) I cannot figure out: via reflection I would like to execute a method "TestMethod" which is inside…
Christian
  • 41
  • 5
3
votes
2 answers

Stryker.NET support for SLN files from the root of the project

Getting different mutation scores each time I run Stryker from the root of the project which has multiple test projects. Does stryker support running '.sln' files from the root of the project? Command used from the root of the project: dotnet…
Itachi
  • 41
  • 3
3
votes
2 answers

PIT Coverage generation minion exited abnormally

i see the following stacktrace when running the pitest gradle task in my project after adding all the required configurations in build.gradle. Can you please help me on this ? I'm using 1.5.1 version of the plugin. The changes have been done in…
3
votes
1 answer

Can pitest be configured to mutation test code that exists a maven dependent jar file and not the maven module itself?

We are currently introducing parallel builds into our maven project to reduce build times. We have 4 modules in our maven project. main-app main-app-mutation-test main-app-integration-test main-app-acceptance-test As part of the parallel build, we…
Nos
  • 1,024
  • 1
  • 8
  • 14
3
votes
2 answers

Is there a way to run instrumented Android tests under mutation with gradle and pitest?

I have a set of instrumented Android tests which run on an emulated device. I can run them with gradle using gradlew connectedDebugAndroidTest, and I've set up gradle-pitest-plugin like so: buildscript { repositories { google() …
Jim J
  • 546
  • 3
  • 11
3
votes
1 answer

invalid question: boolean replaced return of integer sized value with (x == 0 ? 1 : 0) → SURVIVED

Update: The mutation coverage failure went away since this question was posted, probably due to a bugfix in pit? So this question is invalid now. I don't know what to do in this case. Delete the question? Leave it here? I am using pit for mutation…
Árpád Magosányi
  • 1,394
  • 2
  • 19
  • 35
3
votes
2 answers

How Stryker (mutation testing framework) generate mutants?

I am looking into mutation testing and trying to integrate Stryker into my code base. My application is written in React, Nodejs and currently using Jest for client-side testing and Mocha for server-side testing. I am having a few questions…
Hannah
  • 31
  • 2
1 2
3
12 13