What is maven doing now?

Pragmatic Java give a first rate explanation of Maven lifecycles and where and why you use hooks into them.

Agile seating for the perplexed

Great post from Mike Caspar Where should I sit?.

I am a bit late but more marvel from Trisha of LMAX, Dissecting the Disruptor: Demystifying Memory Barriers. How can one not love this writing?

Compilers and CPUs can re-order instructions, provided the end result is the same, to try and optimise performance. Inserting a memory barrier tells the CPU and the compiler that what happened before that command needs to stay before that command, and what happens after needs to stay after. All similarities to a trip to Vegas are entirely in your own mind.

And with diagrams.

Working with Google Protobuf DynamicMessage

Google Protobuf is an stand out project with mixed quality documentation. A corporate cynic points working with DynamicMessage, an interesting Protobuf feature lacking official documentation beyond skeletal javadoc and a short blurb.

The key trick condenses down to:

public static Descriptor descriptorFor(
        final InputStream in, final String name)
        throws IOException, DescriptorValidationException {
    final FileDescriptorSet fileDescriptorSet
            = FileDescriptorSet.parseFrom(in);
    final FileDescriptorProto fileDescriptorProto
            = fileDescriptorSet.getFile(0);
    final FileDescriptor fileDescriptor = FileDescriptor.
            buildFrom(fileDescriptorProto, new FileDescriptor[0]);
    return fileDescriptor.findMessageTypeByName(name);
}

// Elsewhere

final InputStream in = Class.class.
        getResourceAsStream("... your descriptors from protoc ...");
final Descriptor descriptor = descriptorFor(in,
        Outer.getDescriptor().getName());
final DynamicMessage dynamicMessage = DynamicMessage.
        parseFrom(descriptor, outer.toByteArray());
System.out.println("dynamicMessage = " + dynamicMessage);

Note the baked-in assumption of "getFile(0)", etc. The cynic's example is more forthcoming. Caveat coder.

Avoid feature branches

Another explanation, with typically excellent illustrations, from Martin Fowler on why you should avoid feature branches.

A rewrite success story

I avoid rewriting when possible, but greenfield work is ideal: a rewrite success story from the NowJS team.

Why Kotlin?

Dmitri Jemerov answers "Why Kotlin?".

UPDATE: Maria Arias adds his two cents on this question.

ThoughtWorks technology radar for 2011

The ThoughtWorks Technology Radar for 2011 (h/t Martin Fowler, previous versions).

Great problem report on JDK7 bug

A great problem report from Alex Blewitt on the JDK7 loop bug, which turns out is avoidable and not limited to JDK7.

Exposure for Kotlin

The early signs of traction for JetBrain's Kotlin: inclusion in discussions of JVM languages.

Java v. C++, redux redux redux

Yet another Java/C++ comparison, this time for high-frequency trading, a sexy industrial programming area.

I found this quote most trenchant:

If you have a typical Java programmer and typical C++ programmer, each with a few years experience writing a typical Object Oriented Program, and you give them the same amount of time, the Java programmer is likely to have a working program earlier and will have more time to tweak the application. In this situation it is likely the Java application will be faster. IMHO.

In my experience, Java performs better at C++ at detecting code which doesn't need to be done. esp micro-benchmarks which don't do anything useful. ;) If you tune Java and C++ as far as they can go given any amount of expertise and time, the C++ program will be faster. However, given limited resources and in changing environment a dynamic language will out perform. i.e. in real world applications.

You decide.

About time

Mark Reinhold announces the JDK7 release.

UPDATE: And the release notes.

UPDATE: Oops.. I'll wait until the next patch release.

Straight from Darcy: JDK7

Nice keynote deck from Joseph Darcy on JDK7.

JMockit and static methods

Static methods are painful in Java when mocking but JMockit makes some impossible testing possible, though not easy:

public final class MockitEg {
    public static final int SHOE_SIZE = 13;

    public static int shoeSize() {
        System.out.println("MockitEg.shoeSize");
        return SHOE_SIZE;
    }

    private MockitEg() {
    }
}

public class MockitTest {
    private static final int MOCK_SHOE_SIZE = SHOE_SIZE + 29;

    @Test
    public void shouldMockStaticMethod() {
        new NonStrictExpectations() {
            final MockitEg mock = null;

            {
                MockitEg.shoeSize();
                result = new MockitEgDelegate();
            }
        };

        assertThat(MockitEg.shoeSize(),
                is(equalTo(MOCK_SHOE_SIZE)));
    }

    private static final class MockitEgDelegate
            implements Delegate {
        public static int shoeSize() {
            return MOCK_SHOE_SIZE;
        }
    }
}

The test passes.

Two web page nuggets

Two great web page nuggest from colleagues today:

• The C10K problem
• A Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux

(Kudos to Gorlak on a clever domain name.)

Farwell LinkedBlockingQueue

More astonishing figures for the LMAX disruptor. More results like this and I feel a new JDK concurrency framework coming soon.

UPDATE:Trisha posts more explanation of the interesting technical tricks in Disruptor: magic cache line padding.

Java Zipper

A colleague moving between Python and Java asked me if there were an implementation of zip. Handling arbitrary tuples is challenging in Java, but the simple 2-tuple is straight-forward enough:

public final class Pair<T, U> {
    public final T first;
    public final U second;

    public static <T, U> Pair<T, U> pair(final T first, final U second) {
        return new Pair<T, U>(first, second);
    }

    private Pair(final T first, final U second) {
        this.first = first;
        this.second = second;
    }
}

public final class Zipper {
    public static <T, U> Iterable<Pair<T, U>> zip(
            final T[] first, final Iterable<U> second) {
        return zip(asList(first), second);
    }

    public static <T, U> Iterable<Pair<T, U>> zip(
            final Iterable<T> first, final U[] second) {
        return zip(first, asList(second));
    }

    public static <T, U> Iterable<Pair<T, U>> zip(
            final T[] first, final U[] second) {
        return zip(asList(first), asList(second));
    }

    public static <T, U> Iterable<Pair<T, U>> zip(
            final Iterable<T> first, final Iterable<U> second) {
        return new Iterable<Pair<T, U>>() {
            @Override
            public Iterator<Pair<T, U>> iterator() {
                return new Iterator<Pair<T, U>>() {
                    private final Iterator<T> fit
                            = first.iterator();
                    private final Iterator<U> sit
                            = second.iterator();

                    @Override
                    public boolean hasNext() {
                        return fit.hasNext() && sit.hasNext();
                    }

                    @Override
                    public Pair<T, U> next() {
                        return pair(fit.next(), sit.next());
                    }

                    @Override
                    public void remove() {
                        throw new UnsupportedOperationException();
                    }
                };
            }
        };
    }

    private Zipper() {
    }
}

I found several functional Java libraries but none as simple as my colleague wanted.

Kotlin: less is more

JetBrains announced a new JVM language underway: Kotlin. Thanks to Cedric Beust for pointing this out.

My first impression of Kotlin is a simpler Scala retaining the core principle: let the compiler do more, the programmer less. Heavy amounts of inference where it is useful, much shared syntax.

Good luck, JetBrains!

UPDATE: I missed this wiki page my first pass through: Comparison to Scala. The page has interesting comments as well.

UPDATE: Stephen Colebourne expresses a lot of my thoughts better than I do on Kotlin.

A subtle shuffling bug

Mike James explains why random shuffling might not be so random. Little surprise here: Knuth has the right solution.

Closure: Google Javascript optimizer and more

Chris Alexander cites Closure as one of the technologies behind Google+. The FAQ contrasts Closure with GWT:

Closure is geared toward developers who want to work with JavaScript and have a strong understanding of the language, while GWT allows developers to work primarily in Java (although they can work in JavaScript, too) without worrying about the underlying JavaScript.

I wonder what my Javascript friends think?

A reminder why make is a beast

At work today was a vivid reminder why make is a beast.
I'm integrating protobuf into a custom build system based on GNU make, call it yamake ("yet another make") and a proprietary IDL-like language, call it SDL ("sort-of definition language"). SDL is firmly rooted in the culture and systems, so it is non-negotiable. Over the course of the past few days, learned the steps:

1. Write a backend for the Python compiler script which translates SDL to your target language, in my case protobuf.
2. Hand off protobuf backend to guru colleague who munges it to better match the overall systems, and patches it into yamake.
3. Work on makefile to actually build usable jar of compiled protobuf for Java.
4. Watch colleague redo munging to something more permanent, less hacktastic.
5. Rework on makefile to actually build usable jar of compiled protobuf for Java.
6. Roll fake maven local repo holding new jar, of course yamake knows nothing about deploying jars.
7. Build new CORBA IDL jar for Java including new interfaces to access protobuf in other systems.

All this, and finally I can actually do useful work. Tomorrow.
But this is not the point of my post.
Yamake is a wrapper around venerable GNU make, so I am editing makefiles. Ultimately I have this dependency chain: SDL to protobuf to Java to classes to jar. But watch:

$ sdlc -t protobuf foobar.sdl -o proto/foobar.proto
$ protoc --java_out=java proto/foobar.proto
$ javac -sourcepath java java/my/package/FoobarProtos.java
$ jar cf foobar.jar -C java my/package/FoobarProtos*.class

It's actually more Rube-Goldbergesque than this, as the C++ versions of the protobuf play games with namespaces requiring various flags at various stages along with directory changes. So I made Java keep up. You come up with dependency rules that cope with option java_package = "something.random" and option java_outer_classname="SomethingProtos" and mismatched output directories!
I opted for using helper shell scripts for parsing protobuf for proper Java packages with class names and generating dependency makefiles to include in my actual makefile.
Maven, remind me to quit poking on you for your ugly XML and ridiculously noisy output. I forgot all the love you give me.

New Shimmer for Java Concurrency

At work we use this Java idiom to coordinate start up:

public class LatchExecutorService
        implements ExecutorService {
    private final ExecutorService threadPool
            = newSingleThreadExecutor();
    private final CountDownLatch latch;

    public LatchExecutorService(final int count) {
        latch = new CountDownLatch(count);
        threadPool.submit(new Runnable() {
            @Override
            public void run() {
                try {
                    latch.await();
                    threadPool.shutdown();
                } catch (final InterruptedException e) {
                    threadPool.shutdownNow();
                }
            }
        });
    }

    public void countDown() {
        latch.countDown();
    }

    public int getCount() {
        return latch.getCount();
    }

    public <T> Future<T> submit(final Callable<T> task) {
        try {
            return threadPool.submit(task);
        } catch (final RejectedExecutionException e) {
            task.run();
        }
    }

    public <T> Future<T> submit(final Runnable task,
            final T result) {
        try {
            return threadPool.submit(task, result);
        } catch (final RejectedExecutionException e) {
            task.run();
        }
    }

    public Future<?> submit(final Runnable task) {
        try {
            return threadPool.submit(task);
        } catch (final RejectedExecutionException e) {
            task.run();
        }
    }

    // Delegate remaining executor service methods
    // Cut for brevity
}

This class is like a count down latch but with the extra "runnable" feature of a cyclic barrier. When the latch fully counts down, it executes any registered runnables in order of submission. And it is also an executor service.

Some explanation of the unusual submit semantics. The intent of the class is use at start up, to delay actions that depend on some common initialization but guarantee their execution without coordination of parties.

Say there are four worker threads at start up performing essential work. The rest of the program works in a reduced state until these four complete. Rather than carry four independent signals, I collapsed all into LatchExecutorService.

As tasks are submitted, LatchExecutorService delays their execution until its latch fully counts down. With a blocking task at the head of the single-threaded inner executor service, all other tasks must wait on the latch. When the latch triggers, the blocking task clears and all waiting tasks execute on the inner executor service thread.

Once the latch triggers, I do not need the inner executor service any longer except to clear out waiting tasks so I call shutdown() and free up that thread.

I want any new tasks submitted to run immediately. Rather than have them fail or need to coordinate between slow submitters and fast count downers, I run submitted tasks in their caller's thread on rejection. This guarantees new tasks eventually execute without any coordination, either on the original inner executor service or directly at submission.

You can also view LatchExecutorService as an inside-out cyclic barrier. Rather than one runnable registered at construction to execute when all parties signal, callers can provide as many runnables as they like at any time.

It's a count down latch; it's a cyclic barrier; it's an executor service. It's New Shimmer.

Beautiful illustration of feature branching with git

Vincent Driessen posted a while back a rather beautiful illustration of feature branching in git. I wish I had not missed it when first posted (2010). And just what I needed — he includes git command line instructions for executing the model. Posts like this are what make the Internet so helpful in my day-to-day work life.

Gosling and JoSQL

Ran into James Gosling promoting hash maps and RAM as an alternative to SQL. My team has as positive experience with this approach. Coupling it with JoSQL for a SQL-like syntax we can query our object collections. Adding a ZQL ANSI SQL parser and some simple AST editing, we can take pure SQL where clauses and find matching objects in our maps, performantly too. Custom SQL functions are a bonus.