JPMorgan hiring in Houston

JPMorgan is hiring in Houston for experienced core Java hands. Please drop me a line if you are interested.

No time for Windows

I'm timing code on Windows and I notice calls take either 0ms or 15ms. Odd. A little research reveals the culprit: Windows. (Although apparently fixed in Windows 7.) How to address this?

If I am simply timing code I'll use System.nanoTime(), but in my case I am relying on a software clock buried deeply in a library which calls System.currentTimeMillis(). But I can fix that code!

I need to continue using wall-clock millis but I'd like actual millisecond granularity. My solution:

class ActualMillis {
    private final long millis = System.currentTimeMillis();
    private final long nanos = System.nanoTime();

    public long currentTimeMillis() {
        return millis + (System.nanoTime() - nanos) / 1000000;
    }
}

The idea here is to record a snapshot of "now" at construction, and then use offsets from the snapshot on query. This addresses two issues at once:

1. Millis have wall-clock accuracy but 15ms precision.
2. Nanos have no accuracy but better than 1ms precision.

But still some things bother me:

• Will this drift over long periods of time? I ran a series of tests over a few minutes or less, and there was no drift, but I have not tried full day or multi-day runs.
• The contract for System.nanoTime() is difficult. I believe dividing by 1,000,000 obliterates problems like backwards time, but I cannot prove it (although Java 7 seems to address this).
• What happens on different hardware than mine, on different JVMs (1.6.0_18), on different versions of Windows or perhaps if I just hold my breath wrong? I really cannot say.
• The voodoo factor. This code is non-obvious and needs explication. I do not trust explications.

Other solutions very welcome.

JetBrains and the Meta-Programming System

I'm embarrassed to have missed this when version 1.0 came out: JetBrains MPS 1.1: Performance Improvements and Easier Debugging. Or, from the firehose.

Upgrading read lock to write lock in Java

Setup

I ran into a great question on stackoverflow while researching a concurrency question: how to you upgrade a read lock to a write lock?

The answer is easy: you can't. Trying to do so gives you deadlock.

The discussion covers a DAG data structure, but I wanted to try a simpler case.

Simplest case

Say you have an idempotent "resource" (intentionally vague):

public interface Resource {
    void something() throws IOException;
}

The resource can spoil, and you want try another instance automatically. The obvious thing:

public class RetryResourceFacade
        implements Resource {
    private Resource resource;

    public RetryResourceFacade() {
        refreshResource();
    }

    public void something() {
        while (true) {
            try {
                resource.something();
                return;
            } catch (final IOException e) {
                refreshResource();
            }
        }
    }

    private void refreshResource() {
        while (true) {
            try {
                resource = newResource();
                return;
            } catch (final IOException e) {
                // try another one
            }
        }
    }

    public static Resource newResource()
            throws IOException {
        return null; // unlikely in real life
    }
}

Essentially when a resource spoils, you try another one until one works. But this is horrible for concurrent code: how to fix it?

Thread-safe

public class RetryResourceFacade
        implements Resource {
    private final Object lock = new Object();

    private volatile Resource resource;

    public RetryResourceFacade() {
        refreshResource();
    }

    public void something() {
        synchronized(lock) {
            while (true)
                try {
                    resource.something();
                    return;
                } catch (final IOException e) {
                    refreshResource();
                }
            }
        }
    }

    private void refreshResource() {
        while (true) {
            try {
                resource = newResource();
                return;
            } catch (final IOException e) {
                // try another one
            }
        }
    }

    public static Resource newResource()
            throws IOException {
        return null; // unlikely in real life
    }
}

Well, better, but still—do I really want to single-thread calls to something()? I can do yet better still:

More throughput

public class RetryResourceFacade
        implements Resource {
    private final Lock readLock;
    private final Lock writeLock;

    private volatile Resource resource;

    public RetryResourceFacade() {
        final ReentrantReadWriteLock lock = new ReentrantReadWriteLock();
        readLock = lock.readLock();
        writeLock = lock.writeLock();

        refreshResource();
    }

    public void something() {
        while (true) {
            readLock.lock();
            try {
                resource.something();
                readLock.unlock();
                return;
            } catch (final IOException e) {
                readLock.unlock();
                refreshResource();
            }
        }
    }

    private void refreshResource() {
        writeLock.lock();
        while (true) {
            try {
                resource = newResource();
                writeLock.unlock();
                return;
            } catch (final IOException e) {
                // try another one
            }
        }
    }

    public static Resource newResource()
            throws IOException {
        return null; // unlikely in real life
    }
}

By introducing read/write locks, I can let many callers use something() but single-thread refreshing the resource (I do not want several callers refreshing resource at once).

Best of all

But there is one final problem. What happens when two callers are both in something() and the same resource spoils? They both race to refreshResource(), one of them wins and gets a valid resource, but the other then throws that good one away and gets another new resource. Wasteful and embarrassing. The final solution:

public class RetryResourceFacade
        implements Resource {
    private final Lock readLock;
    private final Lock writeLock;

    private volatile Resource resource;
    private volatile boolean valid;

    public RetryResourceFacade() {
        final ReentrantReadWriteLock lock = new ReentrantReadWriteLock();
        readLock = lock.readLock();
        writeLock = lock.writeLock();

        refreshResource();
    }

    public void something() {
        while (true) {
            readLock.lock();
            try {
                resource.something();
                readLock.unlock();
                return;
            } catch (final IOException e) {
                valid = false;
                readLock.unlock();
                refreshResource();
            }
        }
    }

    private void refreshResource() {
        writeLock.lock();
        if (valid) {
            // Another caller already fixed resource
            writeLock.unlock();
            return;
        }

        while (true) {
            try {
                resource = newResource();
                valid = true;
                writeLock.unlock();
                return;
            } catch (final IOException e) {
                // try another one
            }
        }
    }

    public static Resource newResource()
            throws IOException {
        return null; // unlikely in real life
    }
}

Only refresh the resource if it is invalid. This is a little bit like handling spurious wakeup in that several threads compete for the resource, but only the first one gets to actually do something with it.

Key points

• Back to the setup: avoid deadlock. You cannot obtain the write lock while holding the read lock, so release the read lock first, then obtain the write lock.
• Use a status flag to ensure the write lock section is not executed needlessly.
• You must mark resource and valid as volatile to ensure "publish" semantics (Jeremy Mason has a great post on this), otherwise anything could happen (really).

UPDATE: Thanks to Darren Accardo for key ideas in this post; his threading is stronger than mine!

Neil Bartlett on OSGi history

I like to brag about my ex-coworkers at ThoughtWorks, they are some pretty amazing folks. But I should not overlook my coworkers at JPMorgan, who are also pretty amazing folks.

Take this great post on OSGi history from Neil Bartlett who was at JPMorgan in London. I think I've been pretty lucky at picking employers and coworkers (with one huge exception).

(JPMorgan is the kind of company where its easy to connect to others across the globe. It is unlucky I never worked with Neil.)

How to pick pairs

A beautiful post from Michael Norton explaining how to optimize agile pairing, though he does not so in those words.

The answer? Follow the (simple) math: pair your weaker teammates with the solid middle to strengthen them, and let your high-fliers continue to soar.

Explaining agile to management: a good top 10 list

Alberto Gutierrez has a great list of development principles any agilist should take to heart. I like that the list is organized in such a way I can easily present it to management.

I also enjoy his post formatting and layout — it's like reading a good newspaper with modern typography. Into my reader feed goes Alberto.

Using java.lang.Process

Kyle Cartmell has a year-old post entitled Five Common java.lang.Process Pitfalls that recently turned up as "new" on DZone. The post is really top flight and will set you straight if you need to fork processes from inside Java.

Massive agile

James Shore writes about Large-scale Agile. Half a decade ago I was at ThoughtWorks and heard tale of massive agile undertakings in the UK by TW under the aegis, "distributed agile". I never experienced it first hand (but I can read about one such project) and I'd love to hear from any other TWers or alums who worked firsthand on these.

(Thanks to Rod Coffin for pointing out Shore's post to me.)

Clever deployment: blue v. green

Martin Fowler writes about blue-green deployment, a technique I will try out for my next production project. With posts like these, Fowler shows why ThoughtWorks continues to turn out brilliant agilists.