binkley's BLOG: JaCoCo, Gradle, and exclusions

Tuesday, May 01, 2018

JaCoCo, Gradle, and exclusions

The setup

My team is working on a Java server, as part of a larger project project, using Gradle to build and JaCoCo to measure testing code coverage. The build fails if coverage drops below fixed limits (branch, instruction, and line)—"verification" in JaCoCo-speak.

We follow the strategy of The Ratchet: as dev pairs push commits into the project, code coverage may not drop without group agreement, and if coverage rises, the verification limits rise to match. This ensures we have ever-rising coverage, and avoid new code which lacks adequate testing.

The problem

At a work project, we're struggling to get JaCoCo to ignore some new, configuration-only Java classes. These classes have no "real" implementation code to test, are used to setup communication with an external resource, yet are high line-count (static configuration via code). So they drag down our code coverage limits, and there is no effective way to unit test them sensibly. (They are best tested as system tests within our CI pipeline using live programs and remote resources.)

JaCoCo has what seems at first blush a sensible way to exclude these configuration classes from testing:

jacocoTestVerificationCoverage {
    violationRules {
        rule {
            excludes ['hm.binkley.labs.saml.SomeConfig']
            limit {
                counter = 'LINE'
                minimum = 0.90
            }
        }
    }
}

Unfortunately, this does nothing. There is no warning or error, and coverage continues to include the whole code base.

A solution

After a lot of experimenting and StackOverflow research, this answer from Juan Vimberg worked exactly as we needed. Following his approach:

final def excludedClasses = ['hm.binkley.labs.saml.SomeConfig']

jacocoTestVerificationCoverage {
    violationRules {
        rule {
            limit {
                counter = 'LINE'
                minimum = 0.90
            }
        }
    }

    afterEvaluate {
        classDirectories = files(classDirectories.files.collect {
            fileTree(dir: it, excludes: excludedClasses.collect {
                it.replace('.', '/') + '.class'
            })
        })
    }
}

The list of excluded classes is extracted so the same trick can be used in the generated reports:

jacocoTestReport {
    executionData test, databaseTest
    reports {
        html.enabled = true
        xml.enabled = true
        csv.enabled = false
    }
    afterEvaluate {
        classDirectories = files(classDirectories.files.collect {
            fileTree(dir: it, excludes: excludedClasses.collect {
                it.replace('.', '/') + '.class'
            })
        })
    }
}

Something to consider: using wildcards (hm.binkley.labs.saml.*) may take additional work.

Why?

Why does this work, and the "obvious" way does not?

JaCoCo has more than one notion of scoping. The clearest one is the counters: branches, classes, instructions, lines, and methods.

Not as well documented is the scope of checks: bundles, classes, methods, packages, and source files. These are not mix-and-match. For example, exclusions apply to classes. Lyudmil Latinov has the best hints I've found on how this works.