Mutation Testing

Purpose

Code coverage measures which lines ran during testing — not whether the tests verified anything meaningful about them. Mutation testing checks that directly: it introduces small, automated bugs (“mutants”) into the code and reruns the test suite against each one.

Example

// Original
function isAdult(age) {
  return age >= 18;
}

// A mutant: >= changed to >
function isAdult(age) {
  return age > 18;
}
// If no test checks age === 18 specifically, this mutant "survives" —
// every test still passes against the broken code.

Behavior notes

• A killed mutant means some test failed against the mutated code — the suite genuinely caught that specific class of bug.
• A survived mutant means every test still passed despite the introduced bug — a real gap in the suite’s actual verification, invisible to a plain coverage report.
• A high mutation-kill rate is a meaningfully stronger signal of test quality than coverage percentage alone, since coverage says nothing about whether assertions would catch a real defect.
• Mutation testing is computationally expensive (rerunning the full suite per mutant) — typically run less frequently than the test suite itself, such as periodically or before a release.

Common mistakes

• Treating high code coverage as proof of strong tests — coverage only proves lines executed, not that meaningful assertions ran against their results.
• Chasing a 100% mutation-kill rate mechanically, including for mutants that represent behaviorally equivalent code (a mutation that happens not to change actual observable behavior) — some survived mutants are false positives worth recognizing, not fixing.

Runtime notes

A testing technique implemented by dedicated tools (Stryker for JavaScript, similar tools elsewhere) — applicable regardless of runtime, though execution cost scales with test suite size.