12 Maven tips from Johathan Wolter

Repurposing test code for production

For various reasons I often wrap the well-known logback logging library; more precisely, I wrap the SLF4J API it implements.
However, this has a strong drawback: filename and line numbers in the log which should tell me the origin of the log calls instead show me my wrapper class. From logback's perspective the wrapper is the caller; from my perspective the wrapper should be invisible.
The culprit is CallerData.isDirectlyInvokingClass(String, String). It tells logback if a given stack frame is internal to logback or from the calling application. It is static and logback has no hooks for changing its behavior. What to do?
The hack:

public class Wrapper {
    static {
        new MockUp() {
            CallerData it;
 
            @Mock(reentrant = true)
            public boolean isDirectlyInvokingClass(
                    final String currentClass,
                    final String fqnOfInvokingClass) {
                return it.isDirectlyInvokingClass(
                        currentClass, fqnOfInvokingClass)
                    || currentClass.equals(Wrapper.class.getName());
            }
        };
    }

    // Rest of class
}

Enter the clever jmockit library sitting at the high end of the testing mock library pile up (or deep end, if you prefer). Jmockit is not the easiest library to use, but is has more code-fu per line than most other swiss-army knives.
I replace the static method in logback with my own when my wrapper class is loaded. I even get a handle (it) to the original method so I may forward appropriately.
This is not my first try at a solution to wrapping logback. Nor my second. Nor my third. But it is the only one so far which:

• Works
• Looks like Java
• Is compact and limited to the wrapper class
• Can be explained to others (mostly)
• Can be maintained (by some)

I have penance to pay for pulling a testing library into production code, but I get some colleague credit for cleverness. It might even be a wash.
Do watch out for:

• Running the application right in the presence of jmockit
• The obligatory AOP warning for the "it" trick

Guicing your jars

Through the magic of ServiceLoader you can automate the wiring of your classpath components:

public static void main(final String... args) {
    final Injector injector = Guice.createInjector(
            ServiceLoader.load(Module.class).iterator());
    // Use injector here, etc.
}

For each jar or classpath component, include a META-INF/services/com.google.inject.Module file containing the class name of a Guice module which can configure the jar, e.g., hm.binkley.rice.MobModule.

This makes your jars auto-configuring. Merely including them in the classpath is sufficient to be injected into your application. You will find multibindings useful for each jar to provide a well-known list of services without them colliding.

Raw types make my head hurt

This compiles:

public static void main(final String... args) {
    final Fred fred = new Fred();
    final Integer i = fred.get(Integer.class);
}

public interface Bob {
    <T> T get(final Class<T> type);
}

public static class Fred implements Bob {
    @Override
    public <T> T get(final Class<T> type) {
        return type.cast(null);
    }
}

This does not:

public static void main(final String... args) {
    final Fred fred = new Fred();
    final Integer i = fred.get(Integer.class);
}

public interface Bob {
    <T> T get(final Class<T> type);
}

public static class Fred<Q> implements Bob {
    @Override
    public <T> T get(final Class<T> type) {
        return type.cast(null);
    }
}

But this does:

public static void main(final String... args) {
    final Fred<?> fred = new Fred();
    final Integer i = fred.get(Integer.class);
}

public interface Bob {
    <T> T get(final Class<T> type);
}

public static class Fred<Q> implements Bob {
    @Override
    public <T> T get(final Class<T> type) {
        return type.cast(null);
    }
}

And this does:

public static void main(final String... args) {
    final Fred fred = new Fred();
    final Integer i = ((Bob) fred).get(Integer.class);
}

public interface Bob {
    <T> T get(final Class<T> type);
}

public static class Fred<Q> implements Bob {
    @Override
    public <T> T get(final Class<T> type) {
        return type.cast(null);
    }
}

What is going on here?

UPDATE: Thanks to Bob Lee in the comments, I see what is going on. Using a class-level raw type results in the methods of that class also being treated as raw types, even though the type parameters of the class and method are separate. (NB — this does not apply to static methods, only instance methods.)

readlink for better scripting

Fritz Thomas solves a common shell scripting problem for me: finding where your script lives. This script saved as ~/bin/x:

#!/bin/bash
echo "$0"
readlink -f "$0"

With ~/bin in PATH produces:

$ cd ~/bin; ./x
./x
/home/where/the/heart/is/bin/x
$ cd; x
/home/where/the/heart/is/bin/x
/home/where/the/heart/is/bin/x

Just what I need! (Yes, quite an odd home directory.)

Do note the caveats for readlink(1).

JPMorgan hiring in Houston

JPMorgan is hiring in Houston for experienced core Java hands. Please drop me a line if you are interested.

No time for Windows

I'm timing code on Windows and I notice calls take either 0ms or 15ms. Odd. A little research reveals the culprit: Windows. (Although apparently fixed in Windows 7.) How to address this?

If I am simply timing code I'll use System.nanoTime(), but in my case I am relying on a software clock buried deeply in a library which calls System.currentTimeMillis(). But I can fix that code!

I need to continue using wall-clock millis but I'd like actual millisecond granularity. My solution:

class ActualMillis {
    private final long millis = System.currentTimeMillis();
    private final long nanos = System.nanoTime();

    public long currentTimeMillis() {
        return millis + (System.nanoTime() - nanos) / 1000000;
    }
}

The idea here is to record a snapshot of "now" at construction, and then use offsets from the snapshot on query. This addresses two issues at once:

1. Millis have wall-clock accuracy but 15ms precision.
2. Nanos have no accuracy but better than 1ms precision.

But still some things bother me:

• Will this drift over long periods of time? I ran a series of tests over a few minutes or less, and there was no drift, but I have not tried full day or multi-day runs.
• The contract for System.nanoTime() is difficult. I believe dividing by 1,000,000 obliterates problems like backwards time, but I cannot prove it (although Java 7 seems to address this).
• What happens on different hardware than mine, on different JVMs (1.6.0_18), on different versions of Windows or perhaps if I just hold my breath wrong? I really cannot say.
• The voodoo factor. This code is non-obvious and needs explication. I do not trust explications.

Other solutions very welcome.

JetBrains and the Meta-Programming System

I'm embarrassed to have missed this when version 1.0 came out: JetBrains MPS 1.1: Performance Improvements and Easier Debugging. Or, from the firehose.

Upgrading read lock to write lock in Java

Setup

I ran into a great question on stackoverflow while researching a concurrency question: how to you upgrade a read lock to a write lock?

The answer is easy: you can't. Trying to do so gives you deadlock.

The discussion covers a DAG data structure, but I wanted to try a simpler case.

Simplest case

Say you have an idempotent "resource" (intentionally vague):

public interface Resource {
    void something() throws IOException;
}

The resource can spoil, and you want try another instance automatically. The obvious thing:

public class RetryResourceFacade
        implements Resource {
    private Resource resource;

    public RetryResourceFacade() {
        refreshResource();
    }

    public void something() {
        while (true) {
            try {
                resource.something();
                return;
            } catch (final IOException e) {
                refreshResource();
            }
        }
    }

    private void refreshResource() {
        while (true) {
            try {
                resource = newResource();
                return;
            } catch (final IOException e) {
                // try another one
            }
        }
    }

    public static Resource newResource()
            throws IOException {
        return null; // unlikely in real life
    }
}

Essentially when a resource spoils, you try another one until one works. But this is horrible for concurrent code: how to fix it?

Thread-safe

public class RetryResourceFacade
        implements Resource {
    private final Object lock = new Object();

    private volatile Resource resource;

    public RetryResourceFacade() {
        refreshResource();
    }

    public void something() {
        synchronized(lock) {
            while (true)
                try {
                    resource.something();
                    return;
                } catch (final IOException e) {
                    refreshResource();
                }
            }
        }
    }

    private void refreshResource() {
        while (true) {
            try {
                resource = newResource();
                return;
            } catch (final IOException e) {
                // try another one
            }
        }
    }

    public static Resource newResource()
            throws IOException {
        return null; // unlikely in real life
    }
}

Well, better, but still—do I really want to single-thread calls to something()? I can do yet better still:

More throughput

public class RetryResourceFacade
        implements Resource {
    private final Lock readLock;
    private final Lock writeLock;

    private volatile Resource resource;

    public RetryResourceFacade() {
        final ReentrantReadWriteLock lock = new ReentrantReadWriteLock();
        readLock = lock.readLock();
        writeLock = lock.writeLock();

        refreshResource();
    }

    public void something() {
        while (true) {
            readLock.lock();
            try {
                resource.something();
                readLock.unlock();
                return;
            } catch (final IOException e) {
                readLock.unlock();
                refreshResource();
            }
        }
    }

    private void refreshResource() {
        writeLock.lock();
        while (true) {
            try {
                resource = newResource();
                writeLock.unlock();
                return;
            } catch (final IOException e) {
                // try another one
            }
        }
    }

    public static Resource newResource()
            throws IOException {
        return null; // unlikely in real life
    }
}

By introducing read/write locks, I can let many callers use something() but single-thread refreshing the resource (I do not want several callers refreshing resource at once).

Best of all

But there is one final problem. What happens when two callers are both in something() and the same resource spoils? They both race to refreshResource(), one of them wins and gets a valid resource, but the other then throws that good one away and gets another new resource. Wasteful and embarrassing. The final solution:

public class RetryResourceFacade
        implements Resource {
    private final Lock readLock;
    private final Lock writeLock;

    private volatile Resource resource;
    private volatile boolean valid;

    public RetryResourceFacade() {
        final ReentrantReadWriteLock lock = new ReentrantReadWriteLock();
        readLock = lock.readLock();
        writeLock = lock.writeLock();

        refreshResource();
    }

    public void something() {
        while (true) {
            readLock.lock();
            try {
                resource.something();
                readLock.unlock();
                return;
            } catch (final IOException e) {
                valid = false;
                readLock.unlock();
                refreshResource();
            }
        }
    }

    private void refreshResource() {
        writeLock.lock();
        if (valid) {
            // Another caller already fixed resource
            writeLock.unlock();
            return;
        }

        while (true) {
            try {
                resource = newResource();
                valid = true;
                writeLock.unlock();
                return;
            } catch (final IOException e) {
                // try another one
            }
        }
    }

    public static Resource newResource()
            throws IOException {
        return null; // unlikely in real life
    }
}

Only refresh the resource if it is invalid. This is a little bit like handling spurious wakeup in that several threads compete for the resource, but only the first one gets to actually do something with it.

Key points

• Back to the setup: avoid deadlock. You cannot obtain the write lock while holding the read lock, so release the read lock first, then obtain the write lock.
• Use a status flag to ensure the write lock section is not executed needlessly.
• You must mark resource and valid as volatile to ensure "publish" semantics (Jeremy Mason has a great post on this), otherwise anything could happen (really).

UPDATE: Thanks to Darren Accardo for key ideas in this post; his threading is stronger than mine!

Neil Bartlett on OSGi history

I like to brag about my ex-coworkers at ThoughtWorks, they are some pretty amazing folks. But I should not overlook my coworkers at JPMorgan, who are also pretty amazing folks.

Take this great post on OSGi history from Neil Bartlett who was at JPMorgan in London. I think I've been pretty lucky at picking employers and coworkers (with one huge exception).

(JPMorgan is the kind of company where its easy to connect to others across the globe. It is unlucky I never worked with Neil.)

How to pick pairs

A beautiful post from Michael Norton explaining how to optimize agile pairing, though he does not so in those words.

The answer? Follow the (simple) math: pair your weaker teammates with the solid middle to strengthen them, and let your high-fliers continue to soar.

Explaining agile to management: a good top 10 list

Alberto Gutierrez has a great list of development principles any agilist should take to heart. I like that the list is organized in such a way I can easily present it to management.

I also enjoy his post formatting and layout — it's like reading a good newspaper with modern typography. Into my reader feed goes Alberto.

Using java.lang.Process

Kyle Cartmell has a year-old post entitled Five Common java.lang.Process Pitfalls that recently turned up as "new" on DZone. The post is really top flight and will set you straight if you need to fork processes from inside Java.

Massive agile

James Shore writes about Large-scale Agile. Half a decade ago I was at ThoughtWorks and heard tale of massive agile undertakings in the UK by TW under the aegis, "distributed agile". I never experienced it first hand (but I can read about one such project) and I'd love to hear from any other TWers or alums who worked firsthand on these.

(Thanks to Rod Coffin for pointing out Shore's post to me.)

Clever deployment: blue v. green

Martin Fowler writes about blue-green deployment, a technique I will try out for my next production project. With posts like these, Fowler shows why ThoughtWorks continues to turn out brilliant agilists.

C# envy

Tim MacFarlane posts on continuations in C# (be sure to continue). Java 7 cannot come quickly enough.

I require a ... subtle Scala

C++ is complex in part because it is multi-paradigm. That is, C++ supports many styles of programming. It is a rabbit hole, but a necessary one—no one likes to hear "sorry, you can't do that".

Scala is so exciting a follow-on to Java in part because it too is multi-paradigm. Witness Jesse Eichar on self-annotation:

1. class Base {
2.   def magic = "bibbity bobbity boo!!"
3. }
5. trait Extender extends Base {
6.   def myMethod = "I can "+magic
7. }
9. trait SelfTyper {
10.   self : Base => 
11.   
12.   def myMethod = "I can "+magic
13. }

But the two are completely different. Extender can be mixed in with any class and adds both the "magic" and "myMethod" to the class it is mixed with. SelfType can only be mixed in with a class that extends Base and SelfTyper only adds the method "myMethod" NOT "magic".

Why is the "self annotations" useful? Because it allows several provides a way of declaring dependencies. One can think of the self annotation declaration as the phrase "I am useable with" or "I require a".

Java is my daily bread and butter (and puts food on the table). There is no easy way in Java to express this subtle point of inheritance—delegation is the closest you come.

In C++ the template system lets you parameterize your base class which you cannot of course do in Java:

template<class T>
class template_base_class : public T
{
};

Billions and billions

Lambda the Ultimate brings me A Few Billion Lines of Code Later: Using Static Analysis to Find Bugs in the Real World. These fellows have a wicked dry sense of humor. Typically:

We thought this approach was bulletproof. Unfortunately, as the astute reader has noted, it requires a command prompt. Soon after implementing it we went to a large company, so large it had a hyperspecialized build engineer, who engaged in the following dialogue:

"How do I run your tool?"

"Oh, it's easy. Just type 'cov-build' before your build command."

"Build command? I just push this [GUI] button..."

Hilarity ensues. Unfortunately the article generated only one comment:

I hugely enjoyed this article. I can't think when I last read an article that made so many important points. Thanks for not sugar-coating the description of real-world problems and real solutions. And thanks for not drowning those points in jargon.

   — Bjarne Stroustrup, January 29, 2010

Happy President's Day.

Scala is Java, or how to grow to love the bomb

Twitter's Nick Kallen posts Why I love everything you hate about Java, a satisfying read about Scala, scaling and design patterns in the real world. More authors (including myself) should post like this.

That he dispenses with explanation why a Java article is really about Scala evidences this fact: Scala is Java, or what Java could have become in a better universe.

UPDATE: Why I love the blogosphere, also entitled Why I hate everything you love about Java courtesy of Roman Roelofsen.

Snakes on the backplane

This presentation is so delightful, I weep that I was not present. And when I thought it could not get better, I learned the term duck-punching. Where have I been?

Scala on Cygwin: Help from Andrew Beacock

Andrew Beacock posts exactly the advice I needed: how to get the scala prompt working on Cygwin. Thanks!

Google has the best Java engineers

Are you reading Jeremy Manson? Why not? (Hint: yet another reason finalizers are wrong even if Gosling won't say it. Alternatives here.)

Against magic

A wonderful post against magic by Stephan Schmidt.

Remember: If you don't understand it, don't use it.

The Rosetta stone, what a paper

I am really enjoying this paper by John Baez (yes, that John Baez) and Mike Stay. My background is in physics (and music) and the crossing of physics, topology, logic and computation is too much to keep to myself. I hope you enjoy it, too.