Showing posts with label gradle. Show all posts
Showing posts with label gradle. Show all posts

Sunday, September 23, 2018

Removing Joda from Spring Boot

Automated Acceptance Criteria

Removing Joda from Spring Boot

The problem

We recently migrated a medium-sized Java project to Spring Boot 2 from version 1. One of the challenges was migrating to the JDK date-time library from Joda. It turns out that Spring Boot 2 has excellent native support for JDK date-times, as does Jackson (JSON) and Hibernate (database), the default technologies offered by Spring Boot 2 for these features.

The migration itself went smoothly, which is unsurprising given the fantastic work of Stephen Colebourne in designing JDK date-time support based on his authorship of Joda.

So we looked at disabling Joda completely in our Gradle build. The most concise approach we found was:

configurations {
    compile.exclude group: 'joda-time'
}

This removed Joda completely from configurations (classpaths) related to Java. However, this had unintended side effects:

  • During tests, we needed Joda in the runtime classpath for a 3rd-party library, OpenSAML
  • During boot run (running the app), we needed Joda in the classpath for another 3rd-party library, SpringFox

We easily found a workaround for SpringFox, but not for OpenSAML.

(If you're curious, yes, we do intent to migrate from OpenSAML 2 (desupported in 2016) to OpenSAML 3; however, we would like spring-security-saml2-core support first.)

A solution in progress

The exclusion in the compile configuration does exactly what we need: Joda disappears! But what to do about SpringFox and OpenSAML?

For the Spring Boot runtime classpath, there is another concise solution, though finding it was rather troublesome, and it is not well-documented by Pivotal or in Stack Overflow.

First, we setup another classpath of our own making named bootRuntime:

configurations {
    // Other parts of "configurations", including the Joda exclusion from above

    bootRuntime // Synthetic configuration for deps needed *only* to launch app
}

Then we added Joda to that synthetic classpath relying on Spring Boot plugin's definition for the version of Joda to use:

dependencies {
    // Other parts of "dependencies"

    bootRuntime 'joda-time:joda-time'
}

Lastly, we taught Spring Boot to include this synthetic classpath when launching our app (this was the trickiest part):

bootJar {
    bootInf {
        from configurations.bootRuntime
        into 'lib'
    }
}

bootRun {
    classpath += configurations.bootRuntime
}

This adds Joda to the runtime classpath for both the single "fat jar" built by Spring (bootJar), and when launching the app on the command line with gradle (bootRun).

Unless you are a heavy Gradle user, from ... into ... syntax may be unfamiliar: this copies the jars in the synthetic configuration into the fat jar at the location Boot expects to find them. The "'lib'" is literally a directory location within the jar. Useful magic, but a bit obtuse. The outcome:

$ jar tf build/libs/the-project.jar | grep joda-time
BOOT-INF/lib/joda-time-2.9.9.jar

As a matter of fact, Joda is the very last file in the boot jar, a suggestion that it was added by our bootInf section after the Boot plugin built the jar.

(Our workaround is intentionally small. If we're unable to make it work, we'll switch to brute-force library exclusions in our dependencies lists. The goal is to prevent accidental import from Joda, for example, of LocalDate.)

Remaining work

For running the boot app, this solution is great: it is small, readable, easy to maintain, and it works. However, for tests which exercise our user authentication with OpenSAML, it fails. Joda is not in the test classpath, and we cannot use or mock OpenSAML methods which use Joda types.

Barring another magical solution like bootRuntime, we'll fall back on manually excluding Joda from each dependency, and adding it back in to the test classpath. A pity given how pithy the solution is with exclusion from the compile configuration.

Tuesday, May 01, 2018

JaCoCo, Gradle, and exclusions

The setup

My team is working on a Java server, as part of a larger project project, using Gradle to build and JaCoCo to measure testing code coverage. The build fails if coverage drops below fixed limits (branch, instruction, and line)—"verification" in JaCoCo-speak.

We follow the strategy of The Ratchet: as dev pairs push commits into the project, code coverage may not drop without group agreement, and if coverage rises, the verification limits rise to match. This ensures we have ever-rising coverage, and avoid new code which lacks adequate testing.

The problem

At a work project, we're struggling to get JaCoCo to ignore some new, configuration-only Java classes. These classes have no "real" implementation code to test, are used to setup communication with an external resource, yet are high line-count (static configuration via code). So they drag down our code coverage limits, and there is no effective way to unit test them sensibly. (They are best tested as system tests within our CI pipeline using live programs and remote resources.)

JaCoCo has what seems at first blush a sensible way to exclude these configuration classes from testing:

jacocoTestVerificationCoverage {
    violationRules {
        rule {
            excludes ['hm.binkley.labs.saml.SomeConfig']
            limit {
                counter = 'LINE'
                minimum = 0.90
            }
        }
    }
}

Unfortunately, this does nothing. There is no warning or error, and coverage continues to include the whole code base.

A solution

After a lot of experimenting and StackOverflow research, this answer from Juan Vimberg worked exactly as we needed. Following his approach:

final def excludedClasses = ['hm.binkley.labs.saml.SomeConfig']

jacocoTestVerificationCoverage {
    violationRules {
        rule {
            limit {
                counter = 'LINE'
                minimum = 0.90
            }
        }
    }

    afterEvaluate {
        classDirectories = files(classDirectories.files.collect {
            fileTree(dir: it, excludes: excludedClasses.collect {
                it.replace('.', '/') + '.class'
            })
        })
    }
}

The list of excluded classes is extracted so the same trick can be used in the generated reports:

jacocoTestReport {
    executionData test, databaseTest
    reports {
        html.enabled = true
        xml.enabled = true
        csv.enabled = false
    }
    afterEvaluate {
        classDirectories = files(classDirectories.files.collect {
            fileTree(dir: it, excludes: excludedClasses.collect {
                it.replace('.', '/') + '.class'
            })
        })
    }
}

Something to consider: using wildcards (hm.binkley.labs.saml.*) may take additional work.

Why?

Why does this work, and the "obvious" way does not?

JaCoCo has more than one notion of scoping. The clearest one is the counters: branches, classes, instructions, lines, and methods.

Not as well documented is the scope of checks: bundles, classes, methods, packages, and source files. These are not mix-and-match. For example, exclusions apply to classes. Lyudmil Latinov has the best hints I've found on how this works.

Friday, August 11, 2017

How to write clean Java

I cannot tell you how to write good Java, but I can help you write clean Java. As usual, automation is key. It's all about the tooling.

The best thing about good tooling is that they work together: each covers a different area, does not impeded another tool, and fixing one complaint a tool reveals often fixes complaints from other tools.

This advice applies to any programming language, not just Java. Java, having the most mature ecosystem, is instructive.

The tools

Use good source control
Git is your best choice. Use either trunk-based development with feature toggles (feature flags), or gitflow patterns, depending on your needs and organization. Require full builds and testing before pushing to a shared repository; hooks can help together with your build tool.
Use good build automation
Maven or Gradle are your best choices; either is excellent. Teach the tool to be strict and fail builds if anything is not just right. Treat your build configuration as code, part of the same repository and held to the same hygiene standards. Keep your local build fast.
Use a good editor
IntelliJ is the best choice. Use inspections, intentions, and reformatting obsessively: code diffs should only show real changes, not whitespace or formatting. Use plugins to ease integrating IntelliJ with other tooling.
Keep your style consistent
Checkstyle is your best choice; other choices are needed for non-Java JVM languages. Fail your build if checkstyle complains. Keep your IntelliJ formatting and Checkstyle rules consistent. Generally, choose either Sun or Google coding standards, and be leary of deviating from them; if unsure, pick Sun.
Fully test your code
JaCoCo works well. Fail your build if coverage drops below a threshhold; rachet up the threshhold as coverage improves. Start low and aim for a 95%+ threshhold. Follow the Test Pyramid; save your end-to-end tests for CI, or run locally only once before pushing commits.
Be zealous in looking for problems
FindBugs, PMD, or Error Prone are your best choices (pick one); other choices may work better for non-Java JVM languages. Fail your build if your choice complains. Be judicious in disabling complaints (for example, FindBugs "experimental" checks, should likely be disabled).
Use code generation
Lombok is your first choice. Add others as needed (or build domain-specific code generators). Generated code does not need test coverage, style checks, or bug detection if the generator is clean and well-tested: trust it.

Update

Dan Wallach reminded me of Error Prone, added above.