ThoughtWorks technology radar for 2011

The ThoughtWorks Technology Radar for 2011 (h/t Martin Fowler, previous versions).

Great problem report on JDK7 bug

A great problem report from Alex Blewitt on the JDK7 loop bug, which turns out is avoidable and not limited to JDK7.

Exposure for Kotlin

The early signs of traction for JetBrain's Kotlin: inclusion in discussions of JVM languages.

Java v. C++, redux redux redux

Yet another Java/C++ comparison, this time for high-frequency trading, a sexy industrial programming area.

I found this quote most trenchant:

If you have a typical Java programmer and typical C++ programmer, each with a few years experience writing a typical Object Oriented Program, and you give them the same amount of time, the Java programmer is likely to have a working program earlier and will have more time to tweak the application. In this situation it is likely the Java application will be faster. IMHO.

In my experience, Java performs better at C++ at detecting code which doesn't need to be done. esp micro-benchmarks which don't do anything useful. ;) If you tune Java and C++ as far as they can go given any amount of expertise and time, the C++ program will be faster. However, given limited resources and in changing environment a dynamic language will out perform. i.e. in real world applications.

You decide.

About time

Mark Reinhold announces the JDK7 release.

UPDATE: And the release notes.

UPDATE: Oops.. I'll wait until the next patch release.

Straight from Darcy: JDK7

Nice keynote deck from Joseph Darcy on JDK7.

JMockit and static methods

Static methods are painful in Java when mocking but JMockit makes some impossible testing possible, though not easy:

public final class MockitEg {
    public static final int SHOE_SIZE = 13;

    public static int shoeSize() {
        System.out.println("MockitEg.shoeSize");
        return SHOE_SIZE;
    }

    private MockitEg() {
    }
}

public class MockitTest {
    private static final int MOCK_SHOE_SIZE = SHOE_SIZE + 29;

    @Test
    public void shouldMockStaticMethod() {
        new NonStrictExpectations() {
            final MockitEg mock = null;

            {
                MockitEg.shoeSize();
                result = new MockitEgDelegate();
            }
        };

        assertThat(MockitEg.shoeSize(),
                is(equalTo(MOCK_SHOE_SIZE)));
    }

    private static final class MockitEgDelegate
            implements Delegate {
        public static int shoeSize() {
            return MOCK_SHOE_SIZE;
        }
    }
}

The test passes.

Two web page nuggets

Two great web page nuggest from colleagues today:

• The C10K problem
• A Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux

(Kudos to Gorlak on a clever domain name.)

Farwell LinkedBlockingQueue

More astonishing figures for the LMAX disruptor. More results like this and I feel a new JDK concurrency framework coming soon.

UPDATE:Trisha posts more explanation of the interesting technical tricks in Disruptor: magic cache line padding.

Java Zipper

A colleague moving between Python and Java asked me if there were an implementation of zip. Handling arbitrary tuples is challenging in Java, but the simple 2-tuple is straight-forward enough:

public final class Pair<T, U> {
    public final T first;
    public final U second;

    public static <T, U> Pair<T, U> pair(final T first, final U second) {
        return new Pair<T, U>(first, second);
    }

    private Pair(final T first, final U second) {
        this.first = first;
        this.second = second;
    }
}

public final class Zipper {
    public static <T, U> Iterable<Pair<T, U>> zip(
            final T[] first, final Iterable<U> second) {
        return zip(asList(first), second);
    }

    public static <T, U> Iterable<Pair<T, U>> zip(
            final Iterable<T> first, final U[] second) {
        return zip(first, asList(second));
    }

    public static <T, U> Iterable<Pair<T, U>> zip(
            final T[] first, final U[] second) {
        return zip(asList(first), asList(second));
    }

    public static <T, U> Iterable<Pair<T, U>> zip(
            final Iterable<T> first, final Iterable<U> second) {
        return new Iterable<Pair<T, U>>() {
            @Override
            public Iterator<Pair<T, U>> iterator() {
                return new Iterator<Pair<T, U>>() {
                    private final Iterator<T> fit
                            = first.iterator();
                    private final Iterator<U> sit
                            = second.iterator();

                    @Override
                    public boolean hasNext() {
                        return fit.hasNext() && sit.hasNext();
                    }

                    @Override
                    public Pair<T, U> next() {
                        return pair(fit.next(), sit.next());
                    }

                    @Override
                    public void remove() {
                        throw new UnsupportedOperationException();
                    }
                };
            }
        };
    }

    private Zipper() {
    }
}

I found several functional Java libraries but none as simple as my colleague wanted.

Kotlin: less is more

JetBrains announced a new JVM language underway: Kotlin. Thanks to Cedric Beust for pointing this out.

My first impression of Kotlin is a simpler Scala retaining the core principle: let the compiler do more, the programmer less. Heavy amounts of inference where it is useful, much shared syntax.

Good luck, JetBrains!

UPDATE: I missed this wiki page my first pass through: Comparison to Scala. The page has interesting comments as well.

UPDATE: Stephen Colebourne expresses a lot of my thoughts better than I do on Kotlin.

A subtle shuffling bug

Mike James explains why random shuffling might not be so random. Little surprise here: Knuth has the right solution.

Closure: Google Javascript optimizer and more

Chris Alexander cites Closure as one of the technologies behind Google+. The FAQ contrasts Closure with GWT:

Closure is geared toward developers who want to work with JavaScript and have a strong understanding of the language, while GWT allows developers to work primarily in Java (although they can work in JavaScript, too) without worrying about the underlying JavaScript.

I wonder what my Javascript friends think?

A reminder why make is a beast

At work today was a vivid reminder why make is a beast.
I'm integrating protobuf into a custom build system based on GNU make, call it yamake ("yet another make") and a proprietary IDL-like language, call it SDL ("sort-of definition language"). SDL is firmly rooted in the culture and systems, so it is non-negotiable. Over the course of the past few days, learned the steps:

1. Write a backend for the Python compiler script which translates SDL to your target language, in my case protobuf.
2. Hand off protobuf backend to guru colleague who munges it to better match the overall systems, and patches it into yamake.
3. Work on makefile to actually build usable jar of compiled protobuf for Java.
4. Watch colleague redo munging to something more permanent, less hacktastic.
5. Rework on makefile to actually build usable jar of compiled protobuf for Java.
6. Roll fake maven local repo holding new jar, of course yamake knows nothing about deploying jars.
7. Build new CORBA IDL jar for Java including new interfaces to access protobuf in other systems.

All this, and finally I can actually do useful work. Tomorrow.
But this is not the point of my post.
Yamake is a wrapper around venerable GNU make, so I am editing makefiles. Ultimately I have this dependency chain: SDL to protobuf to Java to classes to jar. But watch:

$ sdlc -t protobuf foobar.sdl -o proto/foobar.proto
$ protoc --java_out=java proto/foobar.proto
$ javac -sourcepath java java/my/package/FoobarProtos.java
$ jar cf foobar.jar -C java my/package/FoobarProtos*.class

It's actually more Rube-Goldbergesque than this, as the C++ versions of the protobuf play games with namespaces requiring various flags at various stages along with directory changes. So I made Java keep up. You come up with dependency rules that cope with option java_package = "something.random" and option java_outer_classname="SomethingProtos" and mismatched output directories!
I opted for using helper shell scripts for parsing protobuf for proper Java packages with class names and generating dependency makefiles to include in my actual makefile.
Maven, remind me to quit poking on you for your ugly XML and ridiculously noisy output. I forgot all the love you give me.

New Shimmer for Java Concurrency

At work we use this Java idiom to coordinate start up:

public class LatchExecutorService
        implements ExecutorService {
    private final ExecutorService threadPool
            = newSingleThreadExecutor();
    private final CountDownLatch latch;

    public LatchExecutorService(final int count) {
        latch = new CountDownLatch(count);
        threadPool.submit(new Runnable() {
            @Override
            public void run() {
                try {
                    latch.await();
                    threadPool.shutdown();
                } catch (final InterruptedException e) {
                    threadPool.shutdownNow();
                }
            }
        });
    }

    public void countDown() {
        latch.countDown();
    }

    public int getCount() {
        return latch.getCount();
    }

    public <T> Future<T> submit(final Callable<T> task) {
        try {
            return threadPool.submit(task);
        } catch (final RejectedExecutionException e) {
            task.run();
        }
    }

    public <T> Future<T> submit(final Runnable task,
            final T result) {
        try {
            return threadPool.submit(task, result);
        } catch (final RejectedExecutionException e) {
            task.run();
        }
    }

    public Future<?> submit(final Runnable task) {
        try {
            return threadPool.submit(task);
        } catch (final RejectedExecutionException e) {
            task.run();
        }
    }

    // Delegate remaining executor service methods
    // Cut for brevity
}

This class is like a count down latch but with the extra "runnable" feature of a cyclic barrier. When the latch fully counts down, it executes any registered runnables in order of submission. And it is also an executor service.

Some explanation of the unusual submit semantics. The intent of the class is use at start up, to delay actions that depend on some common initialization but guarantee their execution without coordination of parties.

Say there are four worker threads at start up performing essential work. The rest of the program works in a reduced state until these four complete. Rather than carry four independent signals, I collapsed all into LatchExecutorService.

As tasks are submitted, LatchExecutorService delays their execution until its latch fully counts down. With a blocking task at the head of the single-threaded inner executor service, all other tasks must wait on the latch. When the latch triggers, the blocking task clears and all waiting tasks execute on the inner executor service thread.

Once the latch triggers, I do not need the inner executor service any longer except to clear out waiting tasks so I call shutdown() and free up that thread.

I want any new tasks submitted to run immediately. Rather than have them fail or need to coordinate between slow submitters and fast count downers, I run submitted tasks in their caller's thread on rejection. This guarantees new tasks eventually execute without any coordination, either on the original inner executor service or directly at submission.

You can also view LatchExecutorService as an inside-out cyclic barrier. Rather than one runnable registered at construction to execute when all parties signal, callers can provide as many runnables as they like at any time.

It's a count down latch; it's a cyclic barrier; it's an executor service. It's New Shimmer.

Beautiful illustration of feature branching with git

Vincent Driessen posted a while back a rather beautiful illustration of feature branching in git. I wish I had not missed it when first posted (2010). And just what I needed — he includes git command line instructions for executing the model. Posts like this are what make the Internet so helpful in my day-to-day work life.

Gosling and JoSQL

Ran into James Gosling promoting hash maps and RAM as an alternative to SQL. My team has as positive experience with this approach. Coupling it with JoSQL for a SQL-like syntax we can query our object collections. Adding a ZQL ANSI SQL parser and some simple AST editing, we can take pure SQL where clauses and find matching objects in our maps, performantly too. Custom SQL functions are a bonus.

Why Carnegie-Mellon is one of the best undergraduate computer science educations in the world today

I usually link to individual blog posts that catch my imagination — this blog was started as a way for me to recall and find such posts again. Today is an exception.

Robert Harper is a professor in Computer Science at Carnegie-Mellon. His blog, Of Course ML Has Monads! « Existential Type, chronicles his freshman introductory course in functional programming among other topics. Lately I have been looking into Haskell as an alternative to Java at work for some kinds of projects. Harper's posts on ML is opening my horizons further.

The main problem with Haskell or ML is the lack of a good JVM port. Clojure is the next best thing, they all being moral equivalents of LISP even if quite different on the surface.

Back to point. Though the blog is young, I have enjoyed each of Harper's posts and hope he continues. And I with each post I learned something. He is a good teacher outside the classroom. Alas, I was a music major.

Moving Java to Haskell

I am enjoying Tim Carstens' post that translates Java to Haskell focused around monads. It is clean, complete and just how a guide for the novitiate should look.

On the fact that Donald Knuth is not Human

Weiqi Gao reminds me that Donald Knuth is not made of the same stuff as the rest of us. Or he is a very good leg-puller.

I won!

To my utter surprise, I won a Google notebook. Perhaps it is my wife who won. :-)

Thank you, Google!

Agile teams are self-organizing

At Agile Advice a clear example that agile teams are self-organizing: The BIG Difference Between Agile Teams and Project Teamd.

Teach Spring to use per-listener JMS destination queue names, not hard-coded ones

This took some experimenting.

My problem: I am using Spring Framework 3.0 JMS support to wire up a JMS listener container to listener beans. The XML syntax looks like:

<jms:listener-container>
    <jms:listener
            destination="hard-coded-queue-name"
            ref="listener"/>
</jms:listener-container>

This XML is distributed with each program instance. Not a single single server or client instance but a server cluster or client farm: each one of these needs a unique destination so JMS routes correctly.

My first try fixed the uniqueness problem but was less than fully usable:

<jms:listener-container>
    <jms:listener
            destination="#{T(java.util.UUID).randomUUID().toString()}"
            ref="listener"/>
</jms:listener-container>

This suffers excess cleverness. Each listener gets a random UUID for its destination. But, how does the listener refer to this queue name when filling in a JMS reply-to field, or logging or other purposes?

The real answer is to ask the listener for a destination, no produce one externally:

<jms:listener-container>
    <jms:listener
            destination="#{listener.inbox}"
            ref="listener"/>
</jms:listener-container>

And in the listener:

private final String inbox = UUID.randomUUID().toString();

public String getInbox() {
    return inbox;
}

JacORB in Jetty

There is a nasty JVM bug with class verification when you use JacORB in a webapp in Jetty. The issue only occurs when you use a custom classloader (as Jetty does) and load the CORBA in JacORB which replace the default implementation in the JDK. The characteristic message is:

java.lang.VerifyError: (class: org/jacorb/orb/Delegate, method: getReference signature: (Lorg/jacorb/poa/POA;)Lorg/omg/CORBA/portable/ObjectImpl;) Incompatible object argument for function call

The problem has been ongoing since at least 2004 that I can tell from googling without much progress. It happens to other containers than Jetty.

The solution is to ensure the JacORB classes are loaded with the system classloader, not the custom classloader in the webapp container. The only way this can happen is

1. The jacorb jar and its dependencies are not in WEB-INF/lib so Jetty cannot find them there with its classloader, forcing delegation to the system classloader.
2. The system classloader has the jacorb jar and its dependencies prepended to the boot classpath so they can take precedence over the default CORBA implementation in the JDK.

As I use maven, this means my POM for building the war includes:

<dependency>
    <groupId>jacorb</groupId>
    <artifactId>jacorb</artifactId>
    <optional>true</optional>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <optional>true</optional>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-classic</artifactId>
    <optional>true</optional>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-core</artifactId>
    <version>${ch.qos.logback.version}</version>
    <optional>true</optional>
    <scope>provided</scope>
</dependency>

(I am relying on a parent pom to define versions for these)

And my command line to run Jetty from maven with the excellent jetty-maven-plugin (I am on Windows for this project):

$ MAVEN_OPTS='-Xbootclasspath/p:jacorb-2.3.1jboss.patch01-brew.jar\;slf4j-api-1.6.1.jar\;logback-classic-0.9.26.jar\;logback-core-0.9.26.jar -Dorg.omg.CORBA.ORBClass=org.jacorb.orb.ORB -Dorg.omg.CORBA.ORBSingletonClass=org.jacorb.orb.ORBSingleton' mvn jetty:run-war

I am glad this is finally behind me; a day and a half I could have better spent playing elsewhere.