Loading the dice

A thoughtful algorithm/data structure post from Keith Schwartz on dice rolling, applicable to a broad range of simulations, using the Alias Method.

Linking profile to Google+

I've linked my Blogger profile to my Google+ account. Please comment if you have any thoughts on this, or if you see anything odd or missing as a result. Thanks!

Java server hiccups

Gil Tene at Azul has great insight into Java server performance gaffs. He calls them hiccups and provides tooling to find them.

Monad in Java

The shortest, clearest description of monads in Java I have read:

public class OptionalMonad {
    public static <T> Optional<T> unit(T value) {
        return Optional.of(value);
    }

    public static <T, U> Optional<U> bind(
            Optional<T> value,
            Function<T, Optional<U>> function) {
        if (value.isPresent()) return function.apply(value.get());
        else return Optional.absent();
    }
}

Thanks to François Sarradin in a very nice post.

Netflix circuit breaker

Ben Schmaus of the Netflix team posts a lovely description of the degraded service features of their API. The real life use case is compelling. The dashboard is just brilliant.

The death of HyperCard, a whodunit

Stanislav Datskovskiy posts his theory on the murder of HyperCard. It wasn't the butler.

Baker's Treadmill and Disruptor

Flying Frog Consultancy in Cambridge, England posts a nice comparison between Disruptor and low-latency GC algorithms:

The core of [Disruptor's] idea is to accomplish all of this message passing using a single shared data structure, the disruptor, rather than using several separate concurrent queues.

Baker's Treadmill is a low-latency garbage collection algorithm. Allocated blocks in the heap are linked together to form a cyclic doubly-linked list that is divided into four segments using four iterators that chase each other around the ring as heap blocks are allocated, traced and freed.

Cleverly observed with a good diagram. As they say, read the whole thing.

Microbenchmarking and JVM magic

Martin Thompson posts on what I call microbenchmarking and JVM magic, Biased Locking, OSR, and Benchmarking Fun. It is a good praticum to Cliff Click's What the heck is OSR and why is it Bad (or Good)?. OSR is the kind of JVM magic that both surprises and frightens, but makes perfect sense. I walked a colleague through OSR and he agreed with a shrug: the thing you are glad someone else wrote, excess hair.

Colebourne on Scala: EJB2 redux?

Stephen Colebourne posts a lengthy critique of Scala, Scala feels like EJB2. Scala attracts my inner physicist but brings concern to my practical programmer. Colebourne makes these implicit thoughts explicit.

Defeating SSL certificate validation in Java

This post is a hack. You have been warned.

My employer swaps out SSL certificates traversing the corporate firewall. It is a financial firm with regulatory responsibilities that require monitoring all traffic leaving the company, including encrypted traffic.

One side effect is trouble with well-meaning programs which validate SSL certificates before trusting an encrypted connection. These validations fail using the self-signed replacement certificate.

Maven is such a tool:

[WARNING] Could not transfer metadata com.devspan.vendor.envjs:envjs-rhino/maven-metadata.xml from/to sonatype-oss (https://oss.sonatype.org/content/groups/public): Error transferring file: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

The remark about "PKIX path building failed" is characteristic of the probelm.

There is lots of good advice on fixing this correctly, say, here, here or here. But none of them work just now; I cannot edit the locked-down trust store for Java, it does not already contain the aforementioned replacement certificate, and I do not want to build a custom Maven to work around the problem.

Enter the hack.

$ mvn -Djavax.net.ssl.trustStore=/dev/null package

Martin Thompson on lock cost

Anything Martin Thompson says about performance is usually worth a good study. No exception is Locks & Condition Variables - Latency Impact.

His takeaway points:

1. The one-way latency to signal a change is pretty much the same as what is considered current state of the art for network hops between nodes via a switch.
2. The impact is clear when letting the OS choose what CPUs the threads get scheduled on rather than pinning them manually.

Azul's approach for low-latency Java servers

I no longer work in the low-latency space. Still Azul's completion of a non-VM "pause-less" JVM for Linux is exciting. (Also running a JVM on a VM was too recursive.) May it be a benefit to those who need low-latency java servers.

Whither JDK8?

Stephen Colebourne posts on the future of JDK8 based on current JEPs. Among the possiblities, I'm looking forward to:

• Getting the PID of the current process
• Multi-style collections
• Reflection for parameter names
• Significant help for annotation writers (the compiler model stinks for usability)

Punk rock languages

I love Chris Adamson's description of Punk Rock Languages. His introduction should get your blood flowing:

That C has won the end-user practicality battle is obvious to everyone except developers.

The year is 1978, and the first wave of punk rock is reaching its nihilistic peak with infamous U.K. band the Sex Pistols touring the United States and promptly breaking up by the time they reach the West Coast. Elsewhere, Brian Kernighan and Dennis Ritchie are putting the finishing touches on their book The C Programming Language, which will become the de facto standardization of the language for years. While totally unrelated, these two events share a common bond: the ethos of both punk rock and C have lasted for decades, longer than anyone in 1978 could possibly have imagined.

And in many important ways, C is the programmer’s punk rock: it’s fast, messy, dangerous, and perfectly willing to kick your ass, but it’s also an ideal antidote to the pretensions and vanities that plague so many new programming languages. In an era of virtual machines and managed environments, C is the original Punk Rock Language.

Peregrine: Help for programming concurrency

Good news from Columbia University. Peregrine is automated software to aid in programming concurrency. More in the ACM: Major Breakthrough Improves Software Reliability and Security:

"Our main finding in developing Peregrine is that we can make threads deterministic in an efficient and stable way: Peregrine can compute a plan for allowing when and where a thread can 'change lanes' and can then place barriers between the lanes, allowing threads to change lanes only at fixed locations, following a fixed order," says Columbia professor Junfeng Yang. "Once Peregrine computes a good plan without collisions for one group of threads, it can reuse the plan on subsequent groups to avoid the cost of computing a new plan for each new group."

Herb Sutter: Elements of Modern C++ Style

The always readable Herb Sutter begins Elements of Modern C++ Style with:

The C++11 standard offers many useful new features. This page focuses specifically and only on those features that make C++11 really feel like a new language compared to C++98, because:

• They change the styles and idioms you’ll use when writing C++ code, often including the way you’ll design C++ libraries. For example, you’ll see more smart pointer parameters and return values, and functions that return big objects by value.
• They will be used so pervasively that you’ll probably see them in most code examples. For example, virtually every five-line modern C++ code example will say “auto” somewhere.

Use the other great C++11 features too. But get used to these ones first, because these are the pervasive ones that show why C++11 code is clean, safe, and fast – just as clean and safe as code written in any other modern mainstream language, and with C++’s traditional to-the-metal performance as strong as ever.

This sounds good.

Focus, focus, focus

Steve Jobs, role-model for Google CEO.

The 300

A collation of the top 300 algorithms. Please contribute implementations.

Why does blogger republish my posts so often?

Watching the ThoughtWorks Alumni aggregator feed, I see my own posts reappear frequently in batches. I'll see posts last edited days ago suddenly repost. Any suggestion on stopping this misbehavior would be appreciated.

Cheers,
--binkley

When it rains, it pours

More on one of my favorite themes: Romain Prieto provides Expressive code: combining Guava, LambdaJ and Hamcrest.

Kotlin and delegation

I have harped on Java for some time to add language support for delegation. Well, Java is not headed that direction anytime soon. But Kotlin delivers!

Saying:

class Derived(b : Base) : Base by b

forwards method calls on "Derived" to "Base" through the instance of Base named "b". Nifty!.

Kotlin is so full of good ideas it's hard to know where to start.

Neal Ford on functional programming

ThoughtWork's Neal Ford posts a great discussion of object-oriented vs. functional style for coupling and composition. Functional is where it's at. Read Neal to see why.

Amazon and SOA

Steve Yegge's rant really describes how to make SOA work at full-on scales. This is pure gold:

[M]onitoring and QA are the same thing. You'd never think so until you try doing a big SOA. But when your service says "oh yes, I'm fine", it may well be the case that the only thing still functioning in the server is the little component that knows how to say "I'm fine, roger roger, over and out" in a cheery droid voice. In order to tell whether the service is actually responding, you have to make individual calls. The problem continues recursively until your monitoring is doing comprehensive semantics checking of your entire range of services and data, at which point it's indistinguishable from automated QA. So they're a continuum.

Memory for nothing and chips for free

As a long time Linux user, this quote from MSDN nearly spilled my coffee (emphasis mine):

Fundamentals such as memory usage represent a key engineering tenet of Windows 8. In building Windows 8 we set out to significantly reduce the overall runtime memory requirements of the core system. This is always good for everyone and especially in a world where people want to run more and more apps at the same time or run on systems with only 1 or 2GB of memory.

I'm happy to read this post and see engineering has a seat at the table for Windows 8. Funny world where memory is next to free.

If you say "world's best", you should be right

Peteris Krumins publishes the World's best introduction to sed. And he's right.

The power of laziness

On the power of laziness from Australian CIO Magazine:

'Progress isn't made by early risers. It's made by lazy men trying to find easier ways to do something.' Robert Heinlein (1907-1988)

And Seth lived a hundred and five years, and begat Enos

Well, Java has not been around 105 years, but it feels like that sometimes.

David Polak explains why Scala is too hard to replace Java. I laughed at this true description of JEE programming:

Junior developers who are asked to maintain code written by senior developers have to understand the idioms and patterns in the code. While Scala makes putting business logic at the forefront of the code (rather than distributed through a bunch of for loops and complex if statements), depending on the idioms used, decoding that logic is non-obvious. This is a variant of the lack of idioms issue, but at the end of the day, you need a team with a mind-meld to grok some Scala code. … In Java, the patterns are vomited out of the IDE, so developers grow up being able to spot the patterns. That's not true of Scala where the idioms are diverse and team/framework specific.

How to know if Scala is right for you?

• Your company has speakers at JavaOne, OSCON, Strangle Loop, QCon: Scala will be easy
• Lunch-time discussions involve the criteria for moving from a developer to a senior developer: Scala will be hard
• Your developers can write code in NotePad if they have to: Easy
• Your developers stare blankly or say 3 "Hail Marys" when they hear the name "Zed Shaw": Scala == Hard
• Developers all follow Dean Wampler on Twitter: Scala Easy
• Your developers come in at 9:15 and leave before 6 and don't check work email at night: Hard

Given Dave's solid criteria in the posting, my organization will find Scala hard to adopt. I am saddened, because Scala looks fun, productive and intelligent. But my foremost job is to help my organization succeed.

Which brings me to Kotlin (emphasis in the original):

"Project Kotlin" is the codename for a statically-typed JVM-targeted programming language developed by JetBrains and intended for industrial use.

On the Java to Scala scale Kotlin lies solidly between, more like Java in some areas, more like Scala in others, unlike either in yet others. Overall Kotlin looks like a winner for organizations looking to bail from Java but find Scala and friends daunting.

Kotlin may prove a winner.

Architecture gone wild

Tony Arcieri rants on bad suggestions from Node.js folks. His catch phrase, "It gets worse". I love his conclusion:

Clearly the next logical steps are to store previously computed Fibonacci numbers in MongoDB. Or you could build a system with Socket.IO which caches previously computed Fibonacci numbers client-side using a distributed hash table. Then you'd be well on your way to a multimillion dollar cloud-based Fibonacci number service. Eventually you'll probably want to build in an OpenMP C++ native extension to calculate Fibonacci numbers across multiple CPU cores. That way you'll have a truly roflscale Fibonacci web service.

I run into this kind of thinking occassionally at work, but it is quickly tamped down before real money gets burned.

OCaml in real life

Yaron Minsky of the quant prop shop, Jane Street, promotes OCaml in the ACM Queue with OCaml for the Masses. His writing is exact, his code example illuminating and short.

JetBrains IntelliJ IDEA on github

Cool beans: IntelliJ IDEA Community Edition now on GitHub.

Dream on

Big words from Oracle:

Looking ahead to Java 9 and beyond, he explained, Oracle is already identifying key areas of development: a self-tuning JVM, improved native integration and big-data support, reification, adding tail calls and continuations, a new meta-object protocol to improve cross language support, multi-tenancy, resource management for cloud applications, and the building of heterogeneous compute models.

If they deliever this will help poor Java. 60s & 70s technologies 30 years later is better than a poke in the eye with a sharp stick.

My first github fork - maven-protoc-plugin

I just finished with my first github fork, the maven-protoc-plugin. David Trott's original works great, but does not provide all the options available to the protoc program. I added --descriptor_set_out and --include_imports. Enjoy!

Maven, the ideal vs. the reality

Rob Williams describes his maven woes. His solution is nexus, which is great software. But why is this Rube Goldberg set up required? Again, maven.

Web application scaling punch list

Sean Hull posts a punch list of do-nots for web application scalability:

1. Object Relational Mappers
2. Synchronous, Serial, Coupled or Locking Processes
3. One Copy of Your Database
4. Having No Metrics
5. Lack of Feature Flags

A good introduction for intermediate web developers looking to broaden their perspective: read the whole thing.

Java code injection

Jakub Holý posts a first-rate introduction to code injection in Java. As Jakub points out, The coolest thing is that it enables you to modify third party, closed-source classes and actually even JVM classes.

My personal experience with AOP and code injection is mixed, I generally prefer code generation and classpath fiddling, but Jakub is right that one should use the best tool for the job. If a tool is unfamiliar, it's an opportunity.

Non-portable

Elliott Hughes quips:

Oh, yeah... The "_np" suffix means "non-portable". The pthread_getattr_np(3) function is available on glibc and bionic, but not (afaik) on Mac OS. But let's face it; any app that genuinely needs to query stack addresses and sizes is probably going to contain an #ifdef or two anyway...

Generic database records in Java

I haven't posted code in too long. In part I am enjoying work so much these days I don't need this blog as an outlet for my creativity. Still, I feel a touch of guilt.

In a trivial context this came up: How to represent a database record efficiently and generically in Java? By efficiently I mean close to the efficiency of standard Java beans. By generically I mean without the custom writing of standard Java beans. The traditional map representation is certainly generic and simple to write but is not efficient in time or space.

With one condition a nice solution arises: generically applies only to compile-time; that is, I know at compile-time the types and labels of columns read from the database record. That solution: EnumMap.

Some code:

interface Field<R, X extends Exception> {
    <T> T get(final R set) throws X;
}

class EnumRecord<R, X extends Exception,
                E extends Enum<E> & Field<R, X>>
            implements Field<E, RuntimeException> {
    private Map<E, Object> fields;

    EnumRecord(Class<E> enumType, R set) throws X {
        this(enumType, getKeyUniverse(enumType), set);
    }

    EnumRecord(Class<E> enumType, E[] values, R set)
            throws X {
        fields = new EnumMap<E, Object>(enumType);

        for (E field : values)
            fields.put(field, field.get(set));
    }

    EnumRecord(Class<E> enumType, Iterable<E> values, R set)
            throws X {
        fields = new EnumMap<E, Object>(enumType);

        for (E field : values)
            fields.put(field, field.get(set));
    }

    @Override
    public final <T> T get(E value) {
        return (T) fields.get(value);
    }

    @Override
    public boolean equals(Object o) {
        if (this == o)
            return true;
        if (null == o || getClass() != o.getClass())
            return false;

        EnumRecord detail = (EnumRecord) o;

        return fields.equals(detail.fields);
    }

    @Override
    public int hashCode() {
        return fields.hashCode();
    }

    private static <K extends Enum<K>>
            K[] getKeyUniverse(Class<K> enumType) {
        // Copied shamelessly from EnumMap
        return SharedSecrets.getJavaLangAccess()
                .getEnumConstantsShared(enumType);
    }
}

class ResultSetRecord<E extends Enum<E>
            & Field<ResultSet, SQLException>>
        extends EnumRecord<ResultSet, SQLException, E> {
    ResultSetRecord(Class<E> enumType, ResultSet set)
            throws SQLException {
        super(enumType, set);
    }

    ResultSetRecord(Class<E> enumType, E[] values, ResultSet set)
            throws SQLException {
        super(enumType, values, set);
    }

    ResultSetRecord(Class<E> enumType, Iterable<E> values,
                ResultSet set) throws SQLException {
        super(enumType, values, set);
    }
}

enum SampleSQLEnum
        implements Field<ResultSet, SQLException> {
    nick_name() {
        @Override
        public String get(ResultSet set) throws SQLException {
            return getString(set);
        }
    },
    lucky_number() {
        @Override
        public Integer get(final ResultSet set)
                throws SQLException {
            return getInteger(set);
        }
    };

    @Override
    public abstract <T> T get(final ResultSet set)
            throws SQLException;

    protected String getString(ResultSet set)
            throws SQLException {
        String value = set.getString(name()).trim();
        return isBlank(value) ? null : value;
    }

    protected Integer getInteger(ResultSet set)
            throws SQLException {
        int value = set.getInt(name());
        return set.wasNull() ? null : value;
    }
}

class SampleRecord
        extends ResultSetRecord<SampleSQLEnum> {
    SampleRecord(ResultSet set) throws SQLException {
        super(SampleSQLEnum.class, SampleSQLEnum.values(), set);
    }


    String nick_name() {
        return get(SampleSQLEnum.nick_name);
    }

    Integer luck_number() {
        return get(SampleSQLEnum.lucky_number);
    }
}

public class SampleRecordMain {
    public static void main(final String... args)
            throws SQLException {
        SampleRecord record = new SampleRecord(readFromSomewhere());
        System.out.println(record.nick_name());
        System.out.println(record.luck_number());
    }

    private static ResultSet readFromSomewhere() {
        throw null;
    }
}

UPDATE: Some have had trouble importing SharedSecrets. I used it as a convenience to avoid reflection. As an alternative you could reflect over the enum to implement getKeyUniverse yourself.

Java lambdas: C# for the win

Goetz, et al have picked the C# syntax for Java lambdas.

Is state wrong?

An interesting take on state by Tony Arcieri summarized here. My favorite passage is on immuability:

In mutable state languages, performance problems can often be mitigated by mutating local (i.e. non-shared) state instead of creating new objects. To give an example from the Ruby language, combining two strings with the + operator, which creates a new string from two old ones, is significantly slower than combining two strings with the concatenating >> operator, which modifies the original string. Mutating state rather than creating new objects means there's fewer objects for the garbage collector to clean up and helps keep your program in-cache on inner loops. If you've seen Cliff Click's crash course on modern hardware, you're probably familiar with the idea that latency from cache misses is quickly becoming the dominating factor in today's software performance. Too much object creation blows the cache.

Cliff Click also covered Actors, the underpinning of Erlang's concurrency model, in his Concurrency Revolution from a Hardware Perspective talk at JavaOne. One takeaway from this is that actors should provide a safe system for mutable state, because all mutable state is confined to actors which only communicate using messages. Actors should facilitate a shared-nothing system where concurrent state mutations are impossible because no two actors share state and rely on messages for all synchronization and state exchange.

The Kilim library for Java provides a fast zero-copy messaging system for Java which still enables mutable state. In Kilim, when one actor sends a message, it loses visibility of the object it sends, and it becomes the responsibility of the recipient. If both actors need a copy of the message, the sender can make a copy of an object before it's sent to the recipient. Again, Erlang doesn't provide zero-copy (except for binaries) so Kilim's worst case is actually Erlang's best case.

The original posting from July.

Props for github

You cannot pay for an endorsement like this: Linus Torvalds posts the Linux kernel source on github rather than kernel.org, at least for now.

Paen to the state machine

Alan Skorkin writes a paen to the state machine. I've only written state machines for two classes of problems. They are indispensable for scanner-parsers, but I did not do the actual writing: a tool took my grammar and wrote the state machine for me. And for small logic problems I've written trivial state machines around the switch/case statement.

I think Skorkin (and van Bergen, whom he references) overlook the main reason few programmers write state machines: they are hard to reason about which makes them hard to write and hard to test.

What is maven doing now?

Pragmatic Java give a first rate explanation of Maven lifecycles and where and why you use hooks into them.

Agile seating for the perplexed

Great post from Mike Caspar Where should I sit?.

I am a bit late but more marvel from Trisha of LMAX, Dissecting the Disruptor: Demystifying Memory Barriers. How can one not love this writing?

Compilers and CPUs can re-order instructions, provided the end result is the same, to try and optimise performance. Inserting a memory barrier tells the CPU and the compiler that what happened before that command needs to stay before that command, and what happens after needs to stay after. All similarities to a trip to Vegas are entirely in your own mind.

And with diagrams.

Working with Google Protobuf DynamicMessage

Google Protobuf is an stand out project with mixed quality documentation. A corporate cynic points working with DynamicMessage, an interesting Protobuf feature lacking official documentation beyond skeletal javadoc and a short blurb.

The key trick condenses down to:

public static Descriptor descriptorFor(
        final InputStream in, final String name)
        throws IOException, DescriptorValidationException {
    final FileDescriptorSet fileDescriptorSet
            = FileDescriptorSet.parseFrom(in);
    final FileDescriptorProto fileDescriptorProto
            = fileDescriptorSet.getFile(0);
    final FileDescriptor fileDescriptor = FileDescriptor.
            buildFrom(fileDescriptorProto, new FileDescriptor[0]);
    return fileDescriptor.findMessageTypeByName(name);
}

// Elsewhere

final InputStream in = Class.class.
        getResourceAsStream("... your descriptors from protoc ...");
final Descriptor descriptor = descriptorFor(in,
        Outer.getDescriptor().getName());
final DynamicMessage dynamicMessage = DynamicMessage.
        parseFrom(descriptor, outer.toByteArray());
System.out.println("dynamicMessage = " + dynamicMessage);

Note the baked-in assumption of "getFile(0)", etc. The cynic's example is more forthcoming. Caveat coder.

Avoid feature branches

Another explanation, with typically excellent illustrations, from Martin Fowler on why you should avoid feature branches.

A rewrite success story

I avoid rewriting when possible, but greenfield work is ideal: a rewrite success story from the NowJS team.

Why Kotlin?

Dmitri Jemerov answers "Why Kotlin?".

UPDATE: Maria Arias adds his two cents on this question.

ThoughtWorks technology radar for 2011

The ThoughtWorks Technology Radar for 2011 (h/t Martin Fowler, previous versions).

Great problem report on JDK7 bug

A great problem report from Alex Blewitt on the JDK7 loop bug, which turns out is avoidable and not limited to JDK7.

Exposure for Kotlin

The early signs of traction for JetBrain's Kotlin: inclusion in discussions of JVM languages.

Java v. C++, redux redux redux

Yet another Java/C++ comparison, this time for high-frequency trading, a sexy industrial programming area.

I found this quote most trenchant:

If you have a typical Java programmer and typical C++ programmer, each with a few years experience writing a typical Object Oriented Program, and you give them the same amount of time, the Java programmer is likely to have a working program earlier and will have more time to tweak the application. In this situation it is likely the Java application will be faster. IMHO.

In my experience, Java performs better at C++ at detecting code which doesn't need to be done. esp micro-benchmarks which don't do anything useful. ;) If you tune Java and C++ as far as they can go given any amount of expertise and time, the C++ program will be faster. However, given limited resources and in changing environment a dynamic language will out perform. i.e. in real world applications.

You decide.

About time

Mark Reinhold announces the JDK7 release.

UPDATE: And the release notes.

UPDATE: Oops.. I'll wait until the next patch release.

Straight from Darcy: JDK7

Nice keynote deck from Joseph Darcy on JDK7.

JMockit and static methods

Static methods are painful in Java when mocking but JMockit makes some impossible testing possible, though not easy:

public final class MockitEg {
    public static final int SHOE_SIZE = 13;

    public static int shoeSize() {
        System.out.println("MockitEg.shoeSize");
        return SHOE_SIZE;
    }

    private MockitEg() {
    }
}

public class MockitTest {
    private static final int MOCK_SHOE_SIZE = SHOE_SIZE + 29;

    @Test
    public void shouldMockStaticMethod() {
        new NonStrictExpectations() {
            final MockitEg mock = null;

            {
                MockitEg.shoeSize();
                result = new MockitEgDelegate();
            }
        };

        assertThat(MockitEg.shoeSize(),
                is(equalTo(MOCK_SHOE_SIZE)));
    }

    private static final class MockitEgDelegate
            implements Delegate {
        public static int shoeSize() {
            return MOCK_SHOE_SIZE;
        }
    }
}

The test passes.

Two web page nuggets

Two great web page nuggest from colleagues today:

• The C10K problem
• A Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux

(Kudos to Gorlak on a clever domain name.)

Farwell LinkedBlockingQueue

More astonishing figures for the LMAX disruptor. More results like this and I feel a new JDK concurrency framework coming soon.

UPDATE:Trisha posts more explanation of the interesting technical tricks in Disruptor: magic cache line padding.

Java Zipper

A colleague moving between Python and Java asked me if there were an implementation of zip. Handling arbitrary tuples is challenging in Java, but the simple 2-tuple is straight-forward enough:

public final class Pair<T, U> {
    public final T first;
    public final U second;

    public static <T, U> Pair<T, U> pair(final T first, final U second) {
        return new Pair<T, U>(first, second);
    }

    private Pair(final T first, final U second) {
        this.first = first;
        this.second = second;
    }
}

public final class Zipper {
    public static <T, U> Iterable<Pair<T, U>> zip(
            final T[] first, final Iterable<U> second) {
        return zip(asList(first), second);
    }

    public static <T, U> Iterable<Pair<T, U>> zip(
            final Iterable<T> first, final U[] second) {
        return zip(first, asList(second));
    }

    public static <T, U> Iterable<Pair<T, U>> zip(
            final T[] first, final U[] second) {
        return zip(asList(first), asList(second));
    }

    public static <T, U> Iterable<Pair<T, U>> zip(
            final Iterable<T> first, final Iterable<U> second) {
        return new Iterable<Pair<T, U>>() {
            @Override
            public Iterator<Pair<T, U>> iterator() {
                return new Iterator<Pair<T, U>>() {
                    private final Iterator<T> fit
                            = first.iterator();
                    private final Iterator<U> sit
                            = second.iterator();

                    @Override
                    public boolean hasNext() {
                        return fit.hasNext() && sit.hasNext();
                    }

                    @Override
                    public Pair<T, U> next() {
                        return pair(fit.next(), sit.next());
                    }

                    @Override
                    public void remove() {
                        throw new UnsupportedOperationException();
                    }
                };
            }
        };
    }

    private Zipper() {
    }
}

I found several functional Java libraries but none as simple as my colleague wanted.

Kotlin: less is more

JetBrains announced a new JVM language underway: Kotlin. Thanks to Cedric Beust for pointing this out.

My first impression of Kotlin is a simpler Scala retaining the core principle: let the compiler do more, the programmer less. Heavy amounts of inference where it is useful, much shared syntax.

Good luck, JetBrains!

UPDATE: I missed this wiki page my first pass through: Comparison to Scala. The page has interesting comments as well.

UPDATE: Stephen Colebourne expresses a lot of my thoughts better than I do on Kotlin.

A subtle shuffling bug

Mike James explains why random shuffling might not be so random. Little surprise here: Knuth has the right solution.

Closure: Google Javascript optimizer and more

Chris Alexander cites Closure as one of the technologies behind Google+. The FAQ contrasts Closure with GWT:

Closure is geared toward developers who want to work with JavaScript and have a strong understanding of the language, while GWT allows developers to work primarily in Java (although they can work in JavaScript, too) without worrying about the underlying JavaScript.

I wonder what my Javascript friends think?

A reminder why make is a beast

At work today was a vivid reminder why make is a beast.
I'm integrating protobuf into a custom build system based on GNU make, call it yamake ("yet another make") and a proprietary IDL-like language, call it SDL ("sort-of definition language"). SDL is firmly rooted in the culture and systems, so it is non-negotiable. Over the course of the past few days, learned the steps:

1. Write a backend for the Python compiler script which translates SDL to your target language, in my case protobuf.
2. Hand off protobuf backend to guru colleague who munges it to better match the overall systems, and patches it into yamake.
3. Work on makefile to actually build usable jar of compiled protobuf for Java.
4. Watch colleague redo munging to something more permanent, less hacktastic.
5. Rework on makefile to actually build usable jar of compiled protobuf for Java.
6. Roll fake maven local repo holding new jar, of course yamake knows nothing about deploying jars.
7. Build new CORBA IDL jar for Java including new interfaces to access protobuf in other systems.

All this, and finally I can actually do useful work. Tomorrow.
But this is not the point of my post.
Yamake is a wrapper around venerable GNU make, so I am editing makefiles. Ultimately I have this dependency chain: SDL to protobuf to Java to classes to jar. But watch:

$ sdlc -t protobuf foobar.sdl -o proto/foobar.proto
$ protoc --java_out=java proto/foobar.proto
$ javac -sourcepath java java/my/package/FoobarProtos.java
$ jar cf foobar.jar -C java my/package/FoobarProtos*.class

It's actually more Rube-Goldbergesque than this, as the C++ versions of the protobuf play games with namespaces requiring various flags at various stages along with directory changes. So I made Java keep up. You come up with dependency rules that cope with option java_package = "something.random" and option java_outer_classname="SomethingProtos" and mismatched output directories!
I opted for using helper shell scripts for parsing protobuf for proper Java packages with class names and generating dependency makefiles to include in my actual makefile.
Maven, remind me to quit poking on you for your ugly XML and ridiculously noisy output. I forgot all the love you give me.

New Shimmer for Java Concurrency

At work we use this Java idiom to coordinate start up:

public class LatchExecutorService
        implements ExecutorService {
    private final ExecutorService threadPool
            = newSingleThreadExecutor();
    private final CountDownLatch latch;

    public LatchExecutorService(final int count) {
        latch = new CountDownLatch(count);
        threadPool.submit(new Runnable() {
            @Override
            public void run() {
                try {
                    latch.await();
                    threadPool.shutdown();
                } catch (final InterruptedException e) {
                    threadPool.shutdownNow();
                }
            }
        });
    }

    public void countDown() {
        latch.countDown();
    }

    public int getCount() {
        return latch.getCount();
    }

    public <T> Future<T> submit(final Callable<T> task) {
        try {
            return threadPool.submit(task);
        } catch (final RejectedExecutionException e) {
            task.run();
        }
    }

    public <T> Future<T> submit(final Runnable task,
            final T result) {
        try {
            return threadPool.submit(task, result);
        } catch (final RejectedExecutionException e) {
            task.run();
        }
    }

    public Future<?> submit(final Runnable task) {
        try {
            return threadPool.submit(task);
        } catch (final RejectedExecutionException e) {
            task.run();
        }
    }

    // Delegate remaining executor service methods
    // Cut for brevity
}

This class is like a count down latch but with the extra "runnable" feature of a cyclic barrier. When the latch fully counts down, it executes any registered runnables in order of submission. And it is also an executor service.

Some explanation of the unusual submit semantics. The intent of the class is use at start up, to delay actions that depend on some common initialization but guarantee their execution without coordination of parties.

Say there are four worker threads at start up performing essential work. The rest of the program works in a reduced state until these four complete. Rather than carry four independent signals, I collapsed all into LatchExecutorService.

As tasks are submitted, LatchExecutorService delays their execution until its latch fully counts down. With a blocking task at the head of the single-threaded inner executor service, all other tasks must wait on the latch. When the latch triggers, the blocking task clears and all waiting tasks execute on the inner executor service thread.

Once the latch triggers, I do not need the inner executor service any longer except to clear out waiting tasks so I call shutdown() and free up that thread.

I want any new tasks submitted to run immediately. Rather than have them fail or need to coordinate between slow submitters and fast count downers, I run submitted tasks in their caller's thread on rejection. This guarantees new tasks eventually execute without any coordination, either on the original inner executor service or directly at submission.

You can also view LatchExecutorService as an inside-out cyclic barrier. Rather than one runnable registered at construction to execute when all parties signal, callers can provide as many runnables as they like at any time.

It's a count down latch; it's a cyclic barrier; it's an executor service. It's New Shimmer.

Beautiful illustration of feature branching with git

Vincent Driessen posted a while back a rather beautiful illustration of feature branching in git. I wish I had not missed it when first posted (2010). And just what I needed — he includes git command line instructions for executing the model. Posts like this are what make the Internet so helpful in my day-to-day work life.

Gosling and JoSQL

Ran into James Gosling promoting hash maps and RAM as an alternative to SQL. My team has as positive experience with this approach. Coupling it with JoSQL for a SQL-like syntax we can query our object collections. Adding a ZQL ANSI SQL parser and some simple AST editing, we can take pure SQL where clauses and find matching objects in our maps, performantly too. Custom SQL functions are a bonus.

Why Carnegie-Mellon is one of the best undergraduate computer science educations in the world today

I usually link to individual blog posts that catch my imagination — this blog was started as a way for me to recall and find such posts again. Today is an exception.

Robert Harper is a professor in Computer Science at Carnegie-Mellon. His blog, Of Course ML Has Monads! « Existential Type, chronicles his freshman introductory course in functional programming among other topics. Lately I have been looking into Haskell as an alternative to Java at work for some kinds of projects. Harper's posts on ML is opening my horizons further.

The main problem with Haskell or ML is the lack of a good JVM port. Clojure is the next best thing, they all being moral equivalents of LISP even if quite different on the surface.

Back to point. Though the blog is young, I have enjoyed each of Harper's posts and hope he continues. And I with each post I learned something. He is a good teacher outside the classroom. Alas, I was a music major.

Moving Java to Haskell

I am enjoying Tim Carstens' post that translates Java to Haskell focused around monads. It is clean, complete and just how a guide for the novitiate should look.

On the fact that Donald Knuth is not Human

Weiqi Gao reminds me that Donald Knuth is not made of the same stuff as the rest of us. Or he is a very good leg-puller.

I won!

To my utter surprise, I won a Google notebook. Perhaps it is my wife who won. :-)

Thank you, Google!

Agile teams are self-organizing

At Agile Advice a clear example that agile teams are self-organizing: The BIG Difference Between Agile Teams and Project Teamd.

Teach Spring to use per-listener JMS destination queue names, not hard-coded ones

This took some experimenting.

My problem: I am using Spring Framework 3.0 JMS support to wire up a JMS listener container to listener beans. The XML syntax looks like:

<jms:listener-container>
    <jms:listener
            destination="hard-coded-queue-name"
            ref="listener"/>
</jms:listener-container>

This XML is distributed with each program instance. Not a single single server or client instance but a server cluster or client farm: each one of these needs a unique destination so JMS routes correctly.

My first try fixed the uniqueness problem but was less than fully usable:

<jms:listener-container>
    <jms:listener
            destination="#{T(java.util.UUID).randomUUID().toString()}"
            ref="listener"/>
</jms:listener-container>

This suffers excess cleverness. Each listener gets a random UUID for its destination. But, how does the listener refer to this queue name when filling in a JMS reply-to field, or logging or other purposes?

The real answer is to ask the listener for a destination, no produce one externally:

<jms:listener-container>
    <jms:listener
            destination="#{listener.inbox}"
            ref="listener"/>
</jms:listener-container>

And in the listener:

private final String inbox = UUID.randomUUID().toString();

public String getInbox() {
    return inbox;
}

JacORB in Jetty

There is a nasty JVM bug with class verification when you use JacORB in a webapp in Jetty. The issue only occurs when you use a custom classloader (as Jetty does) and load the CORBA in JacORB which replace the default implementation in the JDK. The characteristic message is:

java.lang.VerifyError: (class: org/jacorb/orb/Delegate, method: getReference signature: (Lorg/jacorb/poa/POA;)Lorg/omg/CORBA/portable/ObjectImpl;) Incompatible object argument for function call

The problem has been ongoing since at least 2004 that I can tell from googling without much progress. It happens to other containers than Jetty.

The solution is to ensure the JacORB classes are loaded with the system classloader, not the custom classloader in the webapp container. The only way this can happen is

1. The jacorb jar and its dependencies are not in WEB-INF/lib so Jetty cannot find them there with its classloader, forcing delegation to the system classloader.
2. The system classloader has the jacorb jar and its dependencies prepended to the boot classpath so they can take precedence over the default CORBA implementation in the JDK.

As I use maven, this means my POM for building the war includes:

<dependency>
    <groupId>jacorb</groupId>
    <artifactId>jacorb</artifactId>
    <optional>true</optional>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <optional>true</optional>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-classic</artifactId>
    <optional>true</optional>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-core</artifactId>
    <version>${ch.qos.logback.version}</version>
    <optional>true</optional>
    <scope>provided</scope>
</dependency>

(I am relying on a parent pom to define versions for these)

And my command line to run Jetty from maven with the excellent jetty-maven-plugin (I am on Windows for this project):

$ MAVEN_OPTS='-Xbootclasspath/p:jacorb-2.3.1jboss.patch01-brew.jar\;slf4j-api-1.6.1.jar\;logback-classic-0.9.26.jar\;logback-core-0.9.26.jar -Dorg.omg.CORBA.ORBClass=org.jacorb.orb.ORB -Dorg.omg.CORBA.ORBSingletonClass=org.jacorb.orb.ORBSingleton' mvn jetty:run-war

I am glad this is finally behind me; a day and a half I could have better spent playing elsewhere.