Generic logback.xml

I've started using a generic logback.xml file for my projects which use logging:

<?xml version="1.0"?>
<configuration debug="${log.debug:=false}">
  <property resource="logback-style.properties"/>
  <appender name="console"
      class="ch.qos.logback.core.ConsoleAppender">
    <encoder
        class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
      <pattern>${${log.style:-standard}}</pattern>
    </encoder>
  </appender>
  <root level="${log.level:-warn}">
    <appender-ref ref="console"/>
  </root>
  <include resource="logback-included.xml" optional="true"/>
  <contextListener
      class="ch.qos.logback.classic.jul.LevelChangePropagator"/>
  <jmxConfigurator/>
</configuration>

Some explanation:

• I make use of the substitution with default feature to make use of system properties:

  log.debug

  Setting this to "true" gets logback to log during configuration, default is "false"

  log.style

  See below

  log.level

  This adjusts the global log level, default is "warn" so only errors and warnings log

• Reading variables from a property file as a resource lets me externalize them to the classpath, in particular, the log style (see the next item).
• The name nesting for substitution feature is neat: it lets you use the value of a variable as the name of another variable. For log.style I read a log format string from a properties file, using another property for the key name.
• File inclusion let me keep the boilerplate in this example, and the application-specific bits in a separate logback XML file.
• Lastly I add a context listener to hook java.util.logging into logback and turn on JMX support. I do these last for some small performance gain.

Here is one example log-style.properties:

standard=%date{yyyy-MM-dd'T'HH:mm:ss.SSS'Z',UTC} %level [%thread] %logger - %message%n

Update:

A handy trick to automatically lower logging to "debug" when "logback.debug" is true, best placed near the top:

<if condition='property("logback.debug").equals("true")'>
    <then>
        <variable name="log.level" value="debug"/>
    </then>
</if>

Cautionary tale

Toyota is a cautionary tale for following good practice at software.

Friends don't let friends smash their own stack.

Explaining LISP right

Are you a C++ programmer? Sergey Mozgovoy explains LISP to you. Are you not a C++ programmer? This is still a good post. He discusses LISP well and concisely.

Doing Java file I/O right

Rick Hightower writes I wrote up a small example that shows how to use the FileSystem, URL, Reader, Pattern, Objects, InputStream, URL, URI, etc., really chock full of good practice tidbits. His post is focused on file I/O but the recommendations are good anywhere.

Signing maven releases with gpg while keeping your passphrase secure

On the road towards putting sample code into Maven Central I stumbled at GPG signatures of my artifacts. The problem was how to handle my passphrase.

The recommended solutions (one source of many) all had shortcomings:

• Prompting for your passphrase is often unuseful and is broken when running maven in Cygwin.
• Putting your passphrase on the command-line is visible to ps and is saved in history.
• Putting your passphrase into {{settings.xml}} leaves it on disk.
• Putting your passphrase on removable media and linking to it in {{settings.xml}} is awkward.
• Using a gpg agent would be best, but I did get everything hooked up right; I should investigate further.
• I will not even entertain removing the passphrase from my secrets just to satisfy maven.

A little think and some trial and error led me to:

1. In pom.xml enable maven-pgp-plugin and configure maven-release-plugin:

   <plugin>
       <artifactId>maven-release-plugin</artifactId>
       <configuration>
           <arguments>-Dgpg.passphrase=${gpg.passphrase}</arguments>
       </configuration>
   </plugin>

2. In settings.xml add a profile:

   <profile>
       <id>gpg-sign</id>
       <activation>
           <activeByDefault>true</activeByDefault>
       </activation>
       <properties>
           <gpg.passphrase>${env.GPG_PASSPHRASE}</gpg.passphrase>
       </properties>
   </profile>

3. Write a small helper shell script:

   #!/usr/bin/bash
   
   read -er -p 'Passphrase: ' gpg_passphrase
   echo # Get back newline
   GPG_PASSPHRASE="$gpg_passphrase" exec "$@"

The key observation is that environment variables are private to the process, in memory and transient. I invoke thus:

$ ./rls mvn clean verify
Passphrase:
[INFO] Scanning for projects...

Chaining exceptions in Java 7

I used to write code like this when transmogrifying an exception:

try {
    // Something broken
} catch (final RandomUninformativeException e) {
    final Bug x = new Bug(format("Sprocket sprung a leak: %s", e));
    x.setStackTrace(e.getStackTrace());
    throw x;
}

The intent was to turn a less meaningful exception into a more meaningful one. This was especially useful for log grepping. Most times I could chain the caught exception in the constructor of the thrown one (new BlahException(message, e)) but some exceptions do not have this constructor (e.g., NullPointerException).

With Java 7 I have a more expressive way:

final Bug x = new Bug(format("Sprocket sprung a leak: %s", e));
x.addSuppressed(e);
throw x

You may add as many suppressed exceptions as you like, they all show up nicely (output from some test code):

java.lang.NullPointerException: Outer!
 at scratch.ExceptionMain.main(ExceptionMain.java:11)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:606)
 at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
 Suppressed: java.lang.NullPointerException: Inner!
  at scratch.ExceptionMain.main(ExceptionMain.java:12)
  ... 5 more
 Suppressed: java.lang.NullPointerException: Second!
  at scratch.ExceptionMain.main(ExceptionMain.java:13)
  ... 5 more

State of the art Java lock-free queues

Nitsan Wakart writes on optimizing SPSC lock-free queues (single producer, single consumer) with lots of code samples, performance data and refernces. Even the comments are interesting. An excellent read to understand state of the art in Java.

A random blub:

There are a few subtleties explored here:

• I'm using Unsafe for array access, this is nothing new and is a cut and paste job from the AtomicReferenceArray JDK class. This means I've opted out of the array bound checks we get when we use arrays in Java, but in this case it's fine since the ring buffer wrapping logic already assures correctness there. This is necessary in order to gain access to getVolatile/putOrdered.
• I switched Pressanas original field padding method with mine, mostly for consistency but also it's a more stable method of padding fields (read more on memory layout here).
• I doubled the padding to protect against pre-fetch caused false sharing (padding is omitted above, but have a look at the code).
• I replaced the POW final field with a constant (ELEMENT_SHIFT). This proved surprisingly significant in this case. Final fields are not optimized as aggressively as constants, partly due to the exploited backdoor in Java allowing the modification of final fields (here's Cliff Click's rant on the topic). ELEMENT_SHIFT is the shift for the sparse data and the shift for elements in the array (inferred from Unsafe.arrayIndexScale(Object[].class)) combined.

Better Java properties handling

Luigi R. Viggiano provides OWNER API, a clever, better way to handle Java properties.

Essentially he maps properties in a general sense onto Java interfaces, using annotations to guide sourcing and processing. This is going directly into my projects.

I wrote a similar library based on interfaces to access properties, but using inheritance, implementation and generics with a set of helper base classes. His approach is superior.

@Sources({ "file:~/.myapp.config", 
           "file:/etc/myapp.config", 
           "classpath:foo/bar/baz.properties" })
public interface ServerConfig extends Config {
    @Key("server.http.port")
    int port();

    @Key("server.host.name")
    String hostname();

    @Key("server.max.threads");
    @DefaultValue("42")
    int maxThreads();
}

Interfaces to access properties is right. Among benefits you get strong typing of configuration and your IDE provides code completion.

HTML5 quoting

Great explanation in Quoting and citing with <blockquote>, <q>, <cite>, and the cite attribute.

PIP 446

From the difficulties of advancing a well-established code base:

We are aware of the code breakage this is likely to cause, and doing it anyway for the good of mankind. (Details in the section "Backward Compatibility" below.)

— PEP 446 -- Make newly created file descriptors non-inheritable

Quality

Nikita Salnikov-Tarnovski explains why his company moved to IntelliJ from Eclipse. Amen, brother.

JDK8 preview

Get it while it's hot! JDK8 preview.

Bye, bye PermGen, I won't miss you.

Overwhelmed: git flow, github, maven and central

I decided to publish code to Maven central providing extended version of this blog's sample code. And I have become fascinated with gitflow. So I am stirring the pot, mixing git, github, maven and the central repo. Wish me luck. I'll post a "howto" when it works well enough.

Quick tour of Java 8

Bienvenido David III provides a quick tour of Java 8 in Everything About Java 8.

Rebooting Jigsaw redux

Alex Blewitt has interesting observations on the next reboot of Jigsaw, the project to modularize the JDK, in Jigsaw, Second Cut. Amusing bit:

There has been a module system available for Java from the early days, known as JSR 8 and more recently as OSGi and JSR 291, that is used by all major Java EE application servers, and many other systems and applications in wide use today (JIRA, Eclipse) as well as specific vertical markets (embedded systems, home automation). Jigsaw follows in the footsteps of design successes such as java.util.Date, Java IO NIO NIO2 NIO2.2 and the java.logging package, and aims to ignore best practices to developer an inferior system for use by the JDK itself.

Ultimately he hopes for OSGi-lite.

Netflix contributes RxJava

Somehow I missed this earlier: Functional Reactive in the Netflix API with RxJava by Ben Christensen. It begins:

Our recent post on optimizing the Netflix API introduced how our web service endpoints are implemented using a "functional reactive programming" (FRP) model for composition of asynchronous callbacks from our service layer.

This post takes a closer look at how and why we use the FRP model and introduces our open source project RxJava – a Java implementation of Rx (Reactive Extensions).

Open recursion

Excellent explanation of open recursion from Bob Nystrom.

Providing failure policy with generics and annotations

Java generics with annotations provides a simple technique for specifying failure policies in an API. The exception in a throws clause may be a generic parameter. The @Nullable and @Nonnull annotations express intent.

In the example, Base takes an Exception as a generic parameter. Three implementations demonstrate the choices:

1. ReturnsNull is for "finder" type APIs, where a null return is in good taste.
2. ThrowsUnchecked is when failure is fatal, and should be handled higher in the stack.
3. ThrowsChecked is when failure is transient, and can be handled by the caller (e.g., network failures).

Because the exception is defined at a class level, this technique is best for SAM-style APIs or when a common exception is shared by a few calls.

static void main(final String... args) {
    new ReturnsNull().returnSomething();
    System.out.println("Returned null");
    try {
        new ThrowsUnchecked().returnSomething();
    } catch (final RuntimeException ignored) {
        System.out.println("Threw unchecked");
    }
    try {
        new ThrowsChecked().returnSomething();
    } catch (final Exception ignored) {
        System.out.println("Threw checked");
    }
}

interface Base<E extends Exception> {
    Object returnSomething() throws E;
}

final class ReturnsNull implements Base<RuntimeException> {
    @Nullable @Override
    public String returnSomething() {
        return null;
    }
}

final class ThrowsUnchecked implements Base<RuntimeException> {
    @Nonnull @Override
    public String returnSomething() {
        throw new RuntimeException();
    }
}

final class ThrowsChecked implements Base<Exception> {
    @Nonnull @Override
    public String returnSomething() throws Exception {
        throw new Exception();
    }
}

Extending String in Java with enums

Without extension methods in Java 8 (or without changing JVM language) one cannot extend String. Or can one?

public final class ExtendStringWithEnum {
    public static void main(final String... args) {
        out.println(format("%s: %s", APPLE, toCrAzYcAsE(APPLE)));
        out.println(format("%s: %s", BANANA, BANANA.toCrAzYcAsE()));
    }

    public enum EnumConstants {
        APPLE("apple"), BANANA("banana");
        private final String value;

        EnumConstants(final String value) {
            this.value = value;
        }

        public String toCrAzYcAsE() {
            return ExtendStringWithEnum.toCrAzYcAsE(value);
        }

        @Override
        public String toString() {
            return value;
        }
    }

    public static final class StringConstants {
        public static final String APPLE = "apple";
        public static final String BANANA = "banana";
    }

    public static String toCrAzYcAsE(final String value) {
        final StringBuilder builder = new StringBuilder(value.length());
        for (int i = 0, x = value.length(); i != x; ++i)
            builder.append(
                    0 == i % 2 ? toUpperCase(value.charAt(i))
                            : toLowerCase(value.charAt(i)));
        return builder.toString();
    }
}

Given suitable static imports, the code in "main" for APPLE is how I write it using string constants; the code for BANANA is how I write it for enum constants wrapping strings, and how I would like to write it for strings as well using the static "toCrAzYcAsE" methods. Maybe Java 9.

OOPs

John D. Cook notes in Too many objects the condition of OOP gone mad:

For example, I had a discussion with a colleague once on how to represent depth in an oil well for software we were writing. I said “Let’s just use a number.”

double depth;

My suggestion was laughed off as hopelessly crude. We need to create depth objects! And not just C++ objects, COM objects! We couldn’t send a double out into the world without first wrapping it in the overhead of an object.

Martin Thompson on GC

Another great post from Martin Thompson, Java Garbage Collection Distilled. In spite of the unassuming title, this is a must-read for advanced Java developers. Introduction:

Serial, Parallel, Concurrent, CMS, G1, Young Gen, New Gen, Old Gen, Perm Gen, Eden, Tenured, Survivor Spaces, Safepoints, and the hundreds of JVM startup flags. Does this all baffle you when trying to tune the garbage collector while trying to get the required throughput and latency from your Java application? If it does then do not worry, you are not alone. Documentation describing garbage collection feels like man pages for an aircraft. Every knob and dial is detailed and explained but nowhere can you find a guide on how to fly. This article will attempt to explain the tradeoffs when choosing and tuning garbage collection algorithms for a particular workload.

The focus will be on Oracle Hotspot JVM and OpenJDK collectors as those are the ones in most common usage. Towards the end other commercial JVMs will be discussed to illustrate alternatives.

I especially like his diagram for G1:

Enjoy!

Example RMI code

Syed Muhammad Humayun posts Java Remote Method Invocation (RMI) with Secure Socket Layer (SSL), a succinct, clear example of using SSL with vanilla RMI in Java. RMI never had good examples that I saw, Sayed's contribution goes into toolbox of tricks.

G1GC Overview

Richard Warburton posts on the G1 GC algorithm in Java 6+, part of a series on JVM collectors.

Key takeaway:

The Land of DevOps

Aslan Brooke ties up recent devops writing by Niek Bartholomeus and Gene Kim.