Does Windows really need to be this hard?

While setting up a new Cygwin installation I ran cron-config and found this nugget: Switching the user context without password, Method 1: Create a token from scratch, Switching the user context without password, Method 2: LSA authentication package, and Switching the user context without password, Method 3: With password. Does Windows really need to be this hard?

Correct SLF4J logger wrapping

The problem

As much as the idea pains me, I find it necessary to wrap the well-known logger wrapping framework SLF4J. Why?

The framework author goes to lengths to support JDK1.4 in the API, a reasonable goal for a broadly used API. The result is signatures such as this:

public interface Logger {
    /* ... */

    void info(String msg);

    void info(String format, Object arg);

    void info(String format, Object arg1, Object arg2);

    void info(String format, Object[] args);

    void info(String format, Throwable t);

    /* ... */
}

That's it. You get 0-2 parameters to put into your formatted mesage. If you have more than two parameters you are out of luck. An no mixing exceptions with formatted messages. However, were SLF4J coded against JDK5 the API might become:

public interface Logger {
    /* ... */

    void info(String msg, Object... args);

    /* NB - ellipsis parameter must be last. */
    void info(Throwable t, String msg, Object... args);

    /* ... */
}

Yes, both simpler and more functional. But not a an option without maintaining two versions of the code base or losing backwards compatibility.

A solution

In the past I worked around this with some pretty wild solutions, but today I ran across Wrapping the slf4j API, posted in August.

I can fix SLF4J for JDK5 myself:

public class LoggerBase {
    private static final String FQCN = LoggerBase.class.getName();

    private final Logger logger;

    public LoggerBase(Class thisClass) {
        this.logger = getLogger(thisClass);
    }

    public LoggerBase(String loggerName) {
        this.logger = getLogger(loggerName);
    }

    /* ... */

    public void info(String format, Object... args) {
        info(null, format, args);
    }

    public void info(Throwable thrown, String format, Object... args) {
        if (!logger.isInfoEnabled())
            return;

        if (logger instanceof LocationAwareLogger)
            ((LocationAwareLogger) logger)
                    .log(null, FQCN, INFO_INT, format, args, thrown);
        else
            logger.info(arrayFormat(format, args).getMessage(), thrown);
    }

    /* ... */
}

Hopefully SLF4J presents a solution for this (perhaps a separate JDK5 jar) so I can drop my warped logger wrapper wrapping class.

Scala for the Java programmer

Dave Copeland wrote a wonderful wiki, Another Tour of Scala, around the official A Tour of Scala. In Dave's version he presents Scala topics from the perspective of the Java programmer, explaining how Scala features looks coded in Java and commenting on usefulness and scrutability.

This is the best bootstrap into Scala resource for Java programmers (like me) I have seen. Thanks, Dave.

Jumping off into Spring with Maven

I keep wasting time rediscovering the right Maven dependencies for Spring Framework. I could have just read Keith Donald's blog post.

A new leaf

Today was my last day at JPMorgan Investment Bank where I worked with a great group of people in equities trading. I will miss them.

In a week or two I start my first day at Macquarie Group here in Houston at the energy/gas trading desks and turn over a new leaf. Leaf here means a page of a book. So I am beginning a new part in the diary of my life, a book I hope never closes.

Programming when speed counts

A clever fellow at my work posted a link to Numbers everyone should know. When speed matters, you need to know what is cheap and what is dear.

Dear Oscar Wilde said "What is a cynic? A man who knows the price of everything and the value of nothing." But Alan Perlis countered, "LISP programmers know the value of everything and the cost of nothing."

Learn your costs, the value you get for them, and the price you pay.

Getting started with Ruby on Windows from Felix Raab

Felix Raab posts Ruby Development on Windows, a practical getting started guide. I've seen similar instructions elsewhere. This one is a little different in that he walks you through setting up Microsoft ConsolePowerShell instead of Cygwin. I want to give this a try.

Take that, Mike

This post is gentle ribbing at the best developer I work with at JPMorgan: Mike Doberenz. He is a big Eclipse fan, I am an IntelliJ fan. It's a little like the Emacs-v-vi flame war except that there is an actual argument in favor of Eclipse.

Martin Mikkelborg writes a helpful post on preferring IntelliJ to Eclipse for Android development.

As Mike and I often observe, folks like me feel IntelliJ is better than Eclipse. But I also believe the development model of Eclipse is likely to win out in the long run (units arbitrary):

However, JetBrains has done brilliant work lately in bringing out a free, community edition and keeping their API accessible to plugin authors. It will be interesting to watch the contest of editors over the next few years.

Here is JetBrains explaining how they succeed in an interview with Michael Hunger.

Looking forward to IntelliJ IDEA X

I'm looking forward to IntelliJ IDEA X (10), due real soon now. Typical of the incremental improvements every release has is How to Run a Single Inspection in IDEA X?.

I've been running the preview releases for a while now, and have noticed to my delight that version X is generally faster and lighter-weight than 9 or 8. Faster, better — what's not to like?

Find broken symlinks

Helpful sysadmin tip from Jason White (jasonjgw), philosopher cum Linux expert, on finding broken symlinks.

Great agile advice from Improving Enterprises

Devlin Liles of Improving Enterprises writes of his type at Tyson: Lessons Learned: A Developer Retrospective. My favorite piece of advice is the first one: Always Say Yes:

If the user wants you to write a program that will send unmanned spaceships to hundreds of planets and safely get them back. Say yes. The time, money, effort, and return on investment should be the determining factors for a request, not the developers opinion.

I'd love to build that project.

Ceci n'est pas une liste

Like a programming René Magritte, Robert Smith tells us that LISP does not have lists and it does not have parentheses. To be fair, he is right.

Java 6 GC algorithms and tuning

Sri Lankan Hiranya Jayathilaka posts on Java 6 GC algorithms and tuning. This goes in my bookmarks.

Great management example: origins of Java

I found a real gem in this e-mail exchange featuring Patrick Naughton. This part jumped out at me (written by Sean Luke):

As I remember my Java history Patrick Naughton the gentleman who got the ball rolling was about to quit Sun and join up with NeXT. He happened to be on the same intermural hockey team as Scott McNealy. Scott told him to hold off, write what he thought was wrong with Sun before he left. Patrick didn't leave and was one of the original Oak people.

This is amazing! McNealy was the right kind of manager. Naughton discussed his interests in leaving Sun with McNealy, McNealy asked for feedback and helped Naughton start a new project.

The Internet is filled with tales of the other kind of boss.

Also, I should join my boss' team.

Project inception

A great post from Jonathan Rasmusson (author of The Agile Samurai) on successful project inception. I wish I had his slide deck.

What a find! Land of Lisp!

What a find!Land of Lisp! Don't forget to scroll down. ٩(•̮̮̃•̃)۶

More on more uses for Iterables

Rúnar Bjarnason writes Scalaz Tutorial: Enumeration-Based I/O with Iteratees, an all-singing, all-dancing approach to composable iterators for I/O in Scala. This is a much fuller treatment of I/O iterators than my post, More uses for Iterables.

How to tempt good programmers to work for you

Danilo Campos shows by examples how to tempt good programmers to work for you.

Java ain't got no rhythm, but can it keep time?

I need sub-millisecond timing in Java; I want to measure events in the low microsecond range. What to do? My options seem to be:

1. Use System.currentTimeMillis() and average over long numbers of runs; many problems here.
2. Use System.nanoTime() and time individual events. Great, but the Javadoc is very scary for multi-threaded code.
3. Try out sun.misc.Perf.highResCounter() (many non-official references for this.)
4. Write our own JNI call for gettimeofday().

After months of hitting my head I wised up and looked into the JDK6 C++ source code. Surprise!

The C++ source

Drilling down points me to os::javaTimeNanos() (nanoTime), os::elapsed_counter() (highResCounter), and os::javaTimeMillis() (currentTimeMillis). On Linux these are:

System.currentTimeMillis()

jlong os::javaTimeMillis() {
  timeval time;
  int status = gettimeofday(&time, NULL);
  assert(status != -1, "linux error");
  return jlong(time.tv_sec) * 1000  +  jlong(time.tv_usec / 1000);
}

System.nanoTime()

jlong os::javaTimeNanos() {
  if (Linux::supports_monotonic_clock()) {
    struct timespec tp;
    int status = Linux::clock_gettime(CLOCK_MONOTONIC, &tp);
    assert(status == 0, "gettime error");
    jlong result = jlong(tp.tv_sec) * (1000 * 1000 * 1000) + jlong(tp.tv_nsec);
    return result;
  } else {
    timeval time;
    int status = gettimeofday(&time, NULL);
    assert(status != -1, "linux error");
    jlong usecs = jlong(time.tv_sec) * (1000 * 1000) + jlong(time.tv_usec);
    return 1000 * usecs;
  }
}

sun.misc.Perf.highResCounter()

jlong os::elapsed_counter() {
  timeval time;
  int status = gettimeofday(&time, NULL);
  return jlong(time.tv_sec) * 1000 * 1000 + jlong(time.tv_usec) - initial_time_count;
}

And:

void os::init(void) {
  char dummy;   /* used to get a guess on initial stack address */
//  first_hrtime = gethrtime();

  // With LinuxThreads the JavaMain thread pid (primordial thread)
  // is different than the pid of the java launcher thread.
  // So, on Linux, the launcher thread pid is passed to the VM
  // via the sun.java.launcher.pid property.
  // Use this property instead of getpid() if it was correctly passed.
  // See bug 6351349.
  pid_t java_launcher_pid = (pid_t) Arguments::sun_java_launcher_pid();

  _initial_pid = (java_launcher_pid > 0) ? java_launcher_pid : getpid();

  clock_tics_per_sec = sysconf(_SC_CLK_TCK);

  init_random(1234567);

  ThreadCritical::initialize();

  Linux::set_page_size(sysconf(_SC_PAGESIZE));
  if (Linux::page_size() == -1) {
    fatal1("os_linux.cpp: os::init: sysconf failed (%s)", strerror(errno));
  }
  init_page_sizes((size_t) Linux::page_size());

  Linux::initialize_system_info();

  // main_thread points to the aboriginal thread
  Linux::_main_thread = pthread_self();

  Linux::clock_init();
  initial_time_count = os::elapsed_counter();
  pthread_mutex_init(&dl_mutex, NULL);
}

(The net effect is to normalize to 0-time at JVM boot rather than the epoch.)

The solution

Just use System.nanoTime() and stop worrying so much. It is fine for microsecond timing—even across threads—just RTFM and ignore javadoc.

Elena Yatzeck promotes the agile Army Leadership Field Manual

A positive, fascinating post from Elena Yatzeck on agile leadership as taught in the U.S. Army Leadership Field Manual. The only military members of teams immediate to me are presently serving overseas. Looks like I missed out.

Larval Java

Very cool post from John Rose on larval objects in the VM. How are immutable objects really made in the JVM? How could it be improved? Read and find out.

Nice refactoring example from Bobby Johnson

Bobby Johnson walks us through a nice refactoring example. Its not rocket science or brain surgery, but good old-fashioned craftsman handling of code, great to show journeymen programmers not quite sure what code cleanup looks like.

Pay your debts

Pay your debts or they will eat your project alive. Tea-partying is for code, too.

Come hear me at the Houston TechFest

Come here my talk at the Houston TechFest! I am speaking on Hands-free Coding: Code Generation with Annotation Processors on October 9, 2010 03:45 PM - 04:45 PM at the University of Houston University Center. See you there!

UPDATE: Unfortunately, I spent today laid up at home with a bad back, and was unable to present at TechFest. This is very disappointing. However, as promised, I'll still post sources in the next day or so.

Dick Management

Some Guy's Blog captures dick management perfectly: the difference between good and bad managers. I pursue the former and eschew the latter. You should, too.

Hamlet D'arcy on Mockito

A beautiful post from Hamlet D'arcy on Mockito - Pros, Cons, and Best Practices. My team at work uses EasyMock. I want to use Mockito for my next project; this post makes it an easier sell.

Getting maven dependency sources and javadocs

I found Ted Wise posting on the problem I tackled today: how to download sources and javadocs from command-line maven:

$ mvn dependency:sources
$ mvn dependency:resolve -Dclassifier=javadocs

Unsure why the discrepancy.

The error of group-think

Idris Mootee writes at Blogging Innovation about the error of group-think. My favorite quote:

I see this all the time, one smart person can make a quick decision to solve a problem, two smart persons can frame an issue better and share perspectives and ten smart persons often have no idea of what problems they are trying to solve.

I have seen this all my work life: decision-making by committee produces the proverbial camel instead of a horse.

UPDATE: Fixed article link.

IntelliJ made me $1500 trading on my portfolio today

IntelliJ made me $1500 trading on my portfolio today! No, not really. But is there anything for Java programming it won't do? I read about a "new" (new to me) feature in the IDEA blog, Referencing marked objects in expression evaluation. The post describes cools updates for IDEA X to this already cool feature.

And a picture to save (1000 - N) words:

Agile games people play

3 great agile games from the First Agile Games Conference:

• Packing Peanuts
• The Backlog is in the Eye of the Beholder
• Making Pamphlets: Kanban Game

Thank you Improving Blogs mashup!

Manifesto for Half-Arsed Agile Software Development

Courtesy of Brian Winquist, the Manifesto for Half-Arsed Agile Software Development.

More uses for Iterables

Iterable is one of my favorite interfaces in Java. I love the for-each construct:

for (final Type item : iterableOfItems)
    doSomethingWith(item);

Notice what cannot go wrong:

1. I cannot run off the end, or forget to go far enough (no off-by-one bugs)
2. I cannot forget to call hasNext() before calling next() (iterator misuse)
3. I cannot make the wrong cast or pick the wrong container type

I could go on. It is also concise and conveys programmer intention clearly, both wonderful virtues.

How can I make more use of Iterable?

Here is an example of transforming input into Iterable. First the base class:

public abstract class ResourceIterable<T, R extends Closeable>
        implements Closeable, Iterable<T> {
    private final R resource;

    private volatile boolean closed;

    protected ResourceIterable(final R resource) {
        this.resource = resource;
    }

    protected final R resource() {
        return resource;
    }

    protected abstract T getNext()
            throws IOException;

    @Override
    public final void close()
            throws IOException {
        closed = true;
        resource.close();
    }

    @Override
    public final Iterator<T> iterator() {
        return new Iterator<T>() {
            private T next;

            @Override
            public boolean hasNext() {
                try {
                    next = getNext();
                    return null != next;
                } catch (final EOFException ignored) {
                    next = null;
                    return false;
                } catch (final IOException e) {
                    next = null;
                    if (closed)
                        return false;
                    throw new UndeclaredThrowableException(e);
                }
            }

            @Override
            public T next() {
                if (null == next)
                    throw new NoSuchElementException();
                return next;
            }

            @Override
            public void remove() {
                throw new UnsupportedOperationException();
            }
        };
    }
}

And a sample using class:

public class LineIterable
        extends ResourceIterable {
    public AmpsResponseIterable(final BufferedReader reader) {
        super(reader);
    }

    @Override
    protected String getNext()
            throws IOException {
        return resource().readLine();
    }
}

The example is trivial (my actual uses are hiding sophisticated readers of data structures from network connections) but clear:

for (final String line : new LineIterable(someBufferedReader))
    process(line);

However, all use of Iterable is similarly trivial: that's the point if it. The complexity is in extracting objects from the input (though not in this example).

I prefer hard-to-write-easy-to-use code when I make libraries; easy-to-write-hard-to-use just feels lazy (the wrong kind of lazy).

Guice moving to Maven

It's not new news, but it was new to me: Maven is switching to Guice to wire itself and plugins. From Sonatype:

1. From Plexus to Guice (#1): Why Guice?
2. From Plexus to Guice (#2): The Guice/Plexus Bridge and Custom Bean Injection
3. From Plexus to Guice (#3): Creating a Guice Bean Extension Layer

The plugin framework is named spice inject and glancing through it's straight-forward glue between Guice and OSGi within a Maven object domain.

It looks very clean. Nice.

Large-scale refactoring

Vikas Hazrati has a nice post on large-scale refactoring on InfoQ. I'm a strangler. You?

Have tools, will code

Thanks to the benevolent JetBrains, I now have an OSS license for IntelliJ IDEA to develop an Android app. I first started using IDEA at ThoughtWorks where it quickly became my favorite development tool.

And thanks to Turbine, I have my data.lotro.com developer key! So I am ready to start an LOTRO Android App. And I've already been getting encouraging words from the LOTRO forums. Go, community!

Nexus out, Android in

My dreams of a Nexus One came to naught. But...

I now have an excellent Samsung Galaxy S with Android 2.1, and I asked JetBrains for an open-source license for their Android plugin to IntelliJ IDEA. I have plans brewing!

Hands-free Coding: Code Generation with Annotation Processors

I'm speaking at Houston TechFest on October 9! My talk is Hands-free Coding: Code Generation with Annotation Processors. Wish me luck. Here's the summary:

Have you ever written hundreds or thousands of lines of boilerplate code, or struggled to write Java idioms correctly every time? There is great way to save time and effort and use to standard code patterns automatically: Code Generation.

In this talk, I will walk you through using the JDK6 Annotation Processor API together with Velocity templates to read your Java source code and write new Java source files for you from code templates. It is magic, but magic you can understand and use to be more productive.

Let the compiler write your code for you!

Please come watch!

Lua coming to LOTRO

With Lua coming to LOTRO I may have to dust off my Lua, last used in hacking at T.o.M.E.. It will be a nice break from Java and work, and give me something nice to do for my wife who doesn't know (yet) about MMORPG add-ons.

JB Rainsberge's 10 things

Courtesy of Naresh Jain comes J B Rainsberge's list of 10 things to stop doing or he will bury you alive in a box (about 1hr30m of video), a really delightful presentation on agile.

For the impatient, the list (which is only about half the presentation):

1. Stop writing comments that describe what the code does
2. Stop working in code bases with compile errors
3. Stop insisting that writing the tests after is just as good as writing the tests before
4. Stop insisting that evolutionary design leads to poor architecture
5. Stop writing tests for "coverage"
6. Stop arguing about whether a story is done
7. Stop trying to increase your velocity
8. Stop breaking up working teams
9. Stop acting as though agile is "what the project teams do"
10. Stop telling your people to "go agile" on their own

Go watch the whole thing, and please, Stop It.

Why did Sun fail?

James Gosling tells a great tale on why Sun failed. Spoiler: the alpha particles did it.

Competition for mindshare

Mindshare is a term I rarely hear outside of business or technology magazines, but is very appropriate here. Java has long been king in the unit testing space, and though hoary, JUnit is still one of the best unit testing libraries around.

But Ruby is giving Java serious competition. I am late to the party, but just ran across metric_fu in this post. I wish I had metric_fu for Java!

In the 90s the evolving Java ecosystem was the top reason I moved from C/C++ to Java. JDK7 looks to keep Java the language on life support so it will not immediate wither away (although who came up with the name public defender methods [PDF]?). But the library momentum is not in Java's favor at the moment.

Faster, JRuby! Faster!

Paul Holser's property-binder library

Paul Holser, a fellow ThoughtWorks alum, wrote a nice library for handling properties, property-binder. More to the point, it is an example of PICA style (proxied interfaces configured with annotations), something Paul wrote about recently.

property-binder works using dynamic proxies to turn one kind of call (a typed API) into another (lookups of properties). This is elegantly done.

An alternative for situations where proxies are not appropriate is to use the code-generation facility of annotations, to write Java code for the conversion. The user of annotations cannot tell the difference: the same annotations, interfaces and static factory methods appear in both cases.

This is an interesting corner of Java, bringing together several advanced features but leaving them accessible to the caller with elementary techniques.

Tracking changing data in Java

I need to track changing data in Java. For example, say I am collecting data sets of some real world behavior, and I'd like to know what has changed between individual readings: additions, removals, changes.

I start with a single datum:

public static class Example {
    public final int index; // primary key
    public final double value; // satelite data
    public final long access; // other data

    public Example(final int index, final double value,
            final long access) {
        this.index = index;
        this.value = value;
        this.access = access;
    }

    @Override
    public boolean equals(final Object o) {
        if (this == o)
            return true;
        if (o == null || getClass() != o.getClass())
            return false;

        final Example example = (Example) o;
        if (index != example.index)
            return false;

        return true;
    }

    @Override
    public int hashCode() {
        return index;
    }
}

Reducing to essentials:

• Primary key values: different Example instances with the same index represent the same "thing"
• Satellite data: these are the measurements I want to track
• "Other" data: these are part of my data set but are not part of my change tracking

Now equals and hashCode take care of matching equivalent data so I can compare values between data sets. But I also need to know if the satellite data has changed:

public static class ExampleComparator
        implements Comparator<Example> {
    @Override
    public int compare(final Example a, final Example b) {
        final int c = Integer.valueOf(a.index).compareTo(b.index);
        if (0 != c)
            return c;
        return Double.valueOf(a.value).compareTo(b.value);
    }
}

An important point to consider: why not have Example implement Comparable<Example> instead of the separate comparator class? Read the Comparator javadocs closely: you really want equals and compareTo to agree (the discussion is good — I have a new source of bugs to comb out of my code base).

But here equals looks at only primary keys and the comparator looks at both primary keys and satellite data: they do not agree.

Having all the tools I need, time for comparing data sets (please be kind about the weak class name):

public class Differator<T, C extends Comparator<T>> {
    private final Map<T, T> items = new HashMap<T, T>();

    private final C comparator;

    public static <T, C extends Comparator<T>> Differator<T, C> newDifferator(
            final C comparator) {
        return new Differator<T, C>(comparator);
    }

    public Differator(final C comparator) {
        this.comparator = comparator;
    }

    public Changes<T> replaceAll(final Set<T> newItems) {
        final Changes<T> changes = new Changes<T>();

        for (final T item : items.keySet())
            if (!newItems.contains(item)) {
                changes.deleted.add(item);
                items.remove(item);
            }
        for (final T newItem : newItems) {
            if (!items.containsKey(newItem))
                changes.inserted.add(newItem);
            else {
                final T item = items.get(newItem);
                if (0 != comparator.compare(item, newItem))
                    changes.updated.add(new Delta<T>(item, newItem));
            }
            items.put(newItem, newItem);
        }

        return changes;
    }

    public static class Delta<T> {
        public final T item;
        public final T newItem;

        public Delta(final T item, final T newItem) {
            this.item = item;
            this.newItem = newItem;
        }

        @Override
        public boolean equals(final Object o) {
            if (this == o)
                return true;
            if (o == null || getClass() != o.getClass())
                return false;

            final Delta delta = (Delta) o;

            if (!item.equals(delta.item))
                return false;
            if (!newItem.equals(delta.newItem))
                return false;

            return true;
        }

        @Override
        public int hashCode() {
            int result = item.hashCode();
            result = 31 * result + newItem.hashCode();
            return result;
        }
    }

    public static class Changes<T> {
        public final Set<T> inserted = new HashSet<T>();
        public final Set<Delta<T>> updated = new HashSet<Delta<T>>();
        public final Set<T> deleted = new HashSet<T>();

        @Override
        public boolean equals(final Object o) {
            if (this == o)
                return true;
            if (o == null || getClass() != o.getClass())
                return false;

            final Changes changes = (Changes) o;

            if (!deleted.equals(changes.deleted))
                return false;
            if (!inserted.equals(changes.inserted))
                return false;
            if (!updated.equals(changes.updated))
                return false;

            return true;
        }

        @Override
        public int hashCode() {
            int result = inserted.hashCode();
            result = 31 * result + updated.hashCode();
            result = 31 * result + deleted.hashCode();
            return result;
        }
    }
}

It was not much effort to write Differator. replaceAll swaps out a new data set for an old one, returning how they two differed. Credit for immediately spotting that Differator is not merely thread unsafe, it is very unsafe. But this is not hard to address in a simple way.

Lastly, some tests:

public class DifferatorTest {
    private Differator<Example, ExampleComparator> differator;
    private Example item;
    private Changes<Example> changes;

    @Before
    public void setUp()
            throws Exception {
        differator = newDifferator(new ExampleComparator());
        item = new Example(1, 3.14159, 123);
        changes = differator.replaceAll(singleton(item));
    }

    @Test
    public void testFirstInsert() {
        assertThat(changes.inserted, is(equalTo(singleton(item))));
        assertThat(changes.updated.isEmpty(), is(true));
        assertThat(changes.deleted.isEmpty(), is(true));
    }

    @Test
    public void testUpdateUnimportantData() {
        final Changes<Example> changes = differator
                .replaceAll(singleton(new Example(1, 3.14159, 456)));
        assertThat(changes.inserted.isEmpty(), is(true));
        assertThat(changes.updated.isEmpty(), is(true));
        assertThat(changes.deleted.isEmpty(), is(true));
    }

    @Test
    public void testUpdateSatelliteData() {
        final Example newItem = new Example(1, 2.71828, 123);
        final Changes<Example> changes = differator
                .replaceAll(singleton(newItem));
        assertThat(changes.inserted.isEmpty(), is(true));
        assertThat(changes.updated, is(equalTo(
                singleton(new Differator.Delta<Example>(item, newItem)))));
        assertThat(changes.deleted.isEmpty(), is(true));
    }

    @Test
    public void testUpdatePrimaryKey() {
        final Example newItem = new Example(2, 3.14159, 123);
        final Changes<Example> changes = differator
                .replaceAll(singleton(newItem));
        assertThat(changes.inserted, is(singleton(newItem)));
        assertThat(changes.updated.isEmpty(), is(true));
        assertThat(changes.deleted, is(singleton(item)));
    }
}

Summary of lambda changes for JDK7

Alex Blewitt writing on InfoQ posts an excellent write up on the state of lambda in JDK7.

It is exciting to see Oracle move on this front publicly after long months invisibility. Two interesting potential features I am rooting for: SAMs and this for recursion.

Hope springs eternal in the human breast.

12 Maven tips from Johathan Wolter

Repurposing test code for production

For various reasons I often wrap the well-known logback logging library; more precisely, I wrap the SLF4J API it implements.
However, this has a strong drawback: filename and line numbers in the log which should tell me the origin of the log calls instead show me my wrapper class. From logback's perspective the wrapper is the caller; from my perspective the wrapper should be invisible.
The culprit is CallerData.isDirectlyInvokingClass(String, String). It tells logback if a given stack frame is internal to logback or from the calling application. It is static and logback has no hooks for changing its behavior. What to do?
The hack:

public class Wrapper {
    static {
        new MockUp() {
            CallerData it;
 
            @Mock(reentrant = true)
            public boolean isDirectlyInvokingClass(
                    final String currentClass,
                    final String fqnOfInvokingClass) {
                return it.isDirectlyInvokingClass(
                        currentClass, fqnOfInvokingClass)
                    || currentClass.equals(Wrapper.class.getName());
            }
        };
    }

    // Rest of class
}

Enter the clever jmockit library sitting at the high end of the testing mock library pile up (or deep end, if you prefer). Jmockit is not the easiest library to use, but is has more code-fu per line than most other swiss-army knives.
I replace the static method in logback with my own when my wrapper class is loaded. I even get a handle (it) to the original method so I may forward appropriately.
This is not my first try at a solution to wrapping logback. Nor my second. Nor my third. But it is the only one so far which:

• Works
• Looks like Java
• Is compact and limited to the wrapper class
• Can be explained to others (mostly)
• Can be maintained (by some)

I have penance to pay for pulling a testing library into production code, but I get some colleague credit for cleverness. It might even be a wash.
Do watch out for:

• Running the application right in the presence of jmockit
• The obligatory AOP warning for the "it" trick

Guicing your jars

Through the magic of ServiceLoader you can automate the wiring of your classpath components:

public static void main(final String... args) {
    final Injector injector = Guice.createInjector(
            ServiceLoader.load(Module.class).iterator());
    // Use injector here, etc.
}

For each jar or classpath component, include a META-INF/services/com.google.inject.Module file containing the class name of a Guice module which can configure the jar, e.g., hm.binkley.rice.MobModule.

This makes your jars auto-configuring. Merely including them in the classpath is sufficient to be injected into your application. You will find multibindings useful for each jar to provide a well-known list of services without them colliding.

Raw types make my head hurt

This compiles:

public static void main(final String... args) {
    final Fred fred = new Fred();
    final Integer i = fred.get(Integer.class);
}

public interface Bob {
    <T> T get(final Class<T> type);
}

public static class Fred implements Bob {
    @Override
    public <T> T get(final Class<T> type) {
        return type.cast(null);
    }
}

This does not:

public static void main(final String... args) {
    final Fred fred = new Fred();
    final Integer i = fred.get(Integer.class);
}

public interface Bob {
    <T> T get(final Class<T> type);
}

public static class Fred<Q> implements Bob {
    @Override
    public <T> T get(final Class<T> type) {
        return type.cast(null);
    }
}

But this does:

public static void main(final String... args) {
    final Fred<?> fred = new Fred();
    final Integer i = fred.get(Integer.class);
}

public interface Bob {
    <T> T get(final Class<T> type);
}

public static class Fred<Q> implements Bob {
    @Override
    public <T> T get(final Class<T> type) {
        return type.cast(null);
    }
}

And this does:

public static void main(final String... args) {
    final Fred fred = new Fred();
    final Integer i = ((Bob) fred).get(Integer.class);
}

public interface Bob {
    <T> T get(final Class<T> type);
}

public static class Fred<Q> implements Bob {
    @Override
    public <T> T get(final Class<T> type) {
        return type.cast(null);
    }
}

What is going on here?

UPDATE: Thanks to Bob Lee in the comments, I see what is going on. Using a class-level raw type results in the methods of that class also being treated as raw types, even though the type parameters of the class and method are separate. (NB — this does not apply to static methods, only instance methods.)

readlink for better scripting

Fritz Thomas solves a common shell scripting problem for me: finding where your script lives. This script saved as ~/bin/x:

#!/bin/bash
echo "$0"
readlink -f "$0"

With ~/bin in PATH produces:

$ cd ~/bin; ./x
./x
/home/where/the/heart/is/bin/x
$ cd; x
/home/where/the/heart/is/bin/x
/home/where/the/heart/is/bin/x

Just what I need! (Yes, quite an odd home directory.)

Do note the caveats for readlink(1).

JPMorgan hiring in Houston

JPMorgan is hiring in Houston for experienced core Java hands. Please drop me a line if you are interested.

No time for Windows

I'm timing code on Windows and I notice calls take either 0ms or 15ms. Odd. A little research reveals the culprit: Windows. (Although apparently fixed in Windows 7.) How to address this?

If I am simply timing code I'll use System.nanoTime(), but in my case I am relying on a software clock buried deeply in a library which calls System.currentTimeMillis(). But I can fix that code!

I need to continue using wall-clock millis but I'd like actual millisecond granularity. My solution:

class ActualMillis {
    private final long millis = System.currentTimeMillis();
    private final long nanos = System.nanoTime();

    public long currentTimeMillis() {
        return millis + (System.nanoTime() - nanos) / 1000000;
    }
}

The idea here is to record a snapshot of "now" at construction, and then use offsets from the snapshot on query. This addresses two issues at once:

1. Millis have wall-clock accuracy but 15ms precision.
2. Nanos have no accuracy but better than 1ms precision.

But still some things bother me:

• Will this drift over long periods of time? I ran a series of tests over a few minutes or less, and there was no drift, but I have not tried full day or multi-day runs.
• The contract for System.nanoTime() is difficult. I believe dividing by 1,000,000 obliterates problems like backwards time, but I cannot prove it (although Java 7 seems to address this).
• What happens on different hardware than mine, on different JVMs (1.6.0_18), on different versions of Windows or perhaps if I just hold my breath wrong? I really cannot say.
• The voodoo factor. This code is non-obvious and needs explication. I do not trust explications.

Other solutions very welcome.

JetBrains and the Meta-Programming System

I'm embarrassed to have missed this when version 1.0 came out: JetBrains MPS 1.1: Performance Improvements and Easier Debugging. Or, from the firehose.

Upgrading read lock to write lock in Java

Setup

I ran into a great question on stackoverflow while researching a concurrency question: how to you upgrade a read lock to a write lock?

The answer is easy: you can't. Trying to do so gives you deadlock.

The discussion covers a DAG data structure, but I wanted to try a simpler case.

Simplest case

Say you have an idempotent "resource" (intentionally vague):

public interface Resource {
    void something() throws IOException;
}

The resource can spoil, and you want try another instance automatically. The obvious thing:

public class RetryResourceFacade
        implements Resource {
    private Resource resource;

    public RetryResourceFacade() {
        refreshResource();
    }

    public void something() {
        while (true) {
            try {
                resource.something();
                return;
            } catch (final IOException e) {
                refreshResource();
            }
        }
    }

    private void refreshResource() {
        while (true) {
            try {
                resource = newResource();
                return;
            } catch (final IOException e) {
                // try another one
            }
        }
    }

    public static Resource newResource()
            throws IOException {
        return null; // unlikely in real life
    }
}

Essentially when a resource spoils, you try another one until one works. But this is horrible for concurrent code: how to fix it?

Thread-safe

public class RetryResourceFacade
        implements Resource {
    private final Object lock = new Object();

    private volatile Resource resource;

    public RetryResourceFacade() {
        refreshResource();
    }

    public void something() {
        synchronized(lock) {
            while (true)
                try {
                    resource.something();
                    return;
                } catch (final IOException e) {
                    refreshResource();
                }
            }
        }
    }

    private void refreshResource() {
        while (true) {
            try {
                resource = newResource();
                return;
            } catch (final IOException e) {
                // try another one
            }
        }
    }

    public static Resource newResource()
            throws IOException {
        return null; // unlikely in real life
    }
}

Well, better, but still—do I really want to single-thread calls to something()? I can do yet better still:

More throughput

public class RetryResourceFacade
        implements Resource {
    private final Lock readLock;
    private final Lock writeLock;

    private volatile Resource resource;

    public RetryResourceFacade() {
        final ReentrantReadWriteLock lock = new ReentrantReadWriteLock();
        readLock = lock.readLock();
        writeLock = lock.writeLock();

        refreshResource();
    }

    public void something() {
        while (true) {
            readLock.lock();
            try {
                resource.something();
                readLock.unlock();
                return;
            } catch (final IOException e) {
                readLock.unlock();
                refreshResource();
            }
        }
    }

    private void refreshResource() {
        writeLock.lock();
        while (true) {
            try {
                resource = newResource();
                writeLock.unlock();
                return;
            } catch (final IOException e) {
                // try another one
            }
        }
    }

    public static Resource newResource()
            throws IOException {
        return null; // unlikely in real life
    }
}

By introducing read/write locks, I can let many callers use something() but single-thread refreshing the resource (I do not want several callers refreshing resource at once).

Best of all

But there is one final problem. What happens when two callers are both in something() and the same resource spoils? They both race to refreshResource(), one of them wins and gets a valid resource, but the other then throws that good one away and gets another new resource. Wasteful and embarrassing. The final solution:

public class RetryResourceFacade
        implements Resource {
    private final Lock readLock;
    private final Lock writeLock;

    private volatile Resource resource;
    private volatile boolean valid;

    public RetryResourceFacade() {
        final ReentrantReadWriteLock lock = new ReentrantReadWriteLock();
        readLock = lock.readLock();
        writeLock = lock.writeLock();

        refreshResource();
    }

    public void something() {
        while (true) {
            readLock.lock();
            try {
                resource.something();
                readLock.unlock();
                return;
            } catch (final IOException e) {
                valid = false;
                readLock.unlock();
                refreshResource();
            }
        }
    }

    private void refreshResource() {
        writeLock.lock();
        if (valid) {
            // Another caller already fixed resource
            writeLock.unlock();
            return;
        }

        while (true) {
            try {
                resource = newResource();
                valid = true;
                writeLock.unlock();
                return;
            } catch (final IOException e) {
                // try another one
            }
        }
    }

    public static Resource newResource()
            throws IOException {
        return null; // unlikely in real life
    }
}

Only refresh the resource if it is invalid. This is a little bit like handling spurious wakeup in that several threads compete for the resource, but only the first one gets to actually do something with it.

Key points

• Back to the setup: avoid deadlock. You cannot obtain the write lock while holding the read lock, so release the read lock first, then obtain the write lock.
• Use a status flag to ensure the write lock section is not executed needlessly.
• You must mark resource and valid as volatile to ensure "publish" semantics (Jeremy Mason has a great post on this), otherwise anything could happen (really).

UPDATE: Thanks to Darren Accardo for key ideas in this post; his threading is stronger than mine!

Neil Bartlett on OSGi history

I like to brag about my ex-coworkers at ThoughtWorks, they are some pretty amazing folks. But I should not overlook my coworkers at JPMorgan, who are also pretty amazing folks.

Take this great post on OSGi history from Neil Bartlett who was at JPMorgan in London. I think I've been pretty lucky at picking employers and coworkers (with one huge exception).

(JPMorgan is the kind of company where its easy to connect to others across the globe. It is unlucky I never worked with Neil.)

How to pick pairs

A beautiful post from Michael Norton explaining how to optimize agile pairing, though he does not so in those words.

The answer? Follow the (simple) math: pair your weaker teammates with the solid middle to strengthen them, and let your high-fliers continue to soar.

Explaining agile to management: a good top 10 list

Alberto Gutierrez has a great list of development principles any agilist should take to heart. I like that the list is organized in such a way I can easily present it to management.

I also enjoy his post formatting and layout — it's like reading a good newspaper with modern typography. Into my reader feed goes Alberto.

Using java.lang.Process

Kyle Cartmell has a year-old post entitled Five Common java.lang.Process Pitfalls that recently turned up as "new" on DZone. The post is really top flight and will set you straight if you need to fork processes from inside Java.

Massive agile

James Shore writes about Large-scale Agile. Half a decade ago I was at ThoughtWorks and heard tale of massive agile undertakings in the UK by TW under the aegis, "distributed agile". I never experienced it first hand (but I can read about one such project) and I'd love to hear from any other TWers or alums who worked firsthand on these.

(Thanks to Rod Coffin for pointing out Shore's post to me.)

Clever deployment: blue v. green

Martin Fowler writes about blue-green deployment, a technique I will try out for my next production project. With posts like these, Fowler shows why ThoughtWorks continues to turn out brilliant agilists.

C# envy

Tim MacFarlane posts on continuations in C# (be sure to continue). Java 7 cannot come quickly enough.

I require a ... subtle Scala

C++ is complex in part because it is multi-paradigm. That is, C++ supports many styles of programming. It is a rabbit hole, but a necessary one—no one likes to hear "sorry, you can't do that".

Scala is so exciting a follow-on to Java in part because it too is multi-paradigm. Witness Jesse Eichar on self-annotation:

1. class Base {
2.   def magic = "bibbity bobbity boo!!"
3. }
5. trait Extender extends Base {
6.   def myMethod = "I can "+magic
7. }
9. trait SelfTyper {
10.   self : Base => 
11.   
12.   def myMethod = "I can "+magic
13. }

But the two are completely different. Extender can be mixed in with any class and adds both the "magic" and "myMethod" to the class it is mixed with. SelfType can only be mixed in with a class that extends Base and SelfTyper only adds the method "myMethod" NOT "magic".

Why is the "self annotations" useful? Because it allows several provides a way of declaring dependencies. One can think of the self annotation declaration as the phrase "I am useable with" or "I require a".

Java is my daily bread and butter (and puts food on the table). There is no easy way in Java to express this subtle point of inheritance—delegation is the closest you come.

In C++ the template system lets you parameterize your base class which you cannot of course do in Java:

template<class T>
class template_base_class : public T
{
};

Billions and billions

Lambda the Ultimate brings me A Few Billion Lines of Code Later: Using Static Analysis to Find Bugs in the Real World. These fellows have a wicked dry sense of humor. Typically:

We thought this approach was bulletproof. Unfortunately, as the astute reader has noted, it requires a command prompt. Soon after implementing it we went to a large company, so large it had a hyperspecialized build engineer, who engaged in the following dialogue:

"How do I run your tool?"

"Oh, it's easy. Just type 'cov-build' before your build command."

"Build command? I just push this [GUI] button..."

Hilarity ensues. Unfortunately the article generated only one comment:

I hugely enjoyed this article. I can't think when I last read an article that made so many important points. Thanks for not sugar-coating the description of real-world problems and real solutions. And thanks for not drowning those points in jargon.

   — Bjarne Stroustrup, January 29, 2010

Happy President's Day.

Scala is Java, or how to grow to love the bomb

Twitter's Nick Kallen posts Why I love everything you hate about Java, a satisfying read about Scala, scaling and design patterns in the real world. More authors (including myself) should post like this.

That he dispenses with explanation why a Java article is really about Scala evidences this fact: Scala is Java, or what Java could have become in a better universe.

UPDATE: Why I love the blogosphere, also entitled Why I hate everything you love about Java courtesy of Roman Roelofsen.

Snakes on the backplane

This presentation is so delightful, I weep that I was not present. And when I thought it could not get better, I learned the term duck-punching. Where have I been?

Scala on Cygwin: Help from Andrew Beacock

Andrew Beacock posts exactly the advice I needed: how to get the scala prompt working on Cygwin. Thanks!

Google has the best Java engineers

Are you reading Jeremy Manson? Why not? (Hint: yet another reason finalizers are wrong even if Gosling won't say it. Alternatives here.)

Against magic

A wonderful post against magic by Stephan Schmidt.

Remember: If you don't understand it, don't use it.

The Rosetta stone, what a paper

I am really enjoying this paper by John Baez (yes, that John Baez) and Mike Stay. My background is in physics (and music) and the crossing of physics, topology, logic and computation is too much to keep to myself. I hope you enjoy it, too.

Update on Google Phone saga

An update from Google on my continuing Google phone quest:

XXX YYY has posted an answer to the question "Ineligible to purchase phone: T-mobile contract changed too recently": Howdy binkley, We've listened very carefully to everyone's feedback and concerns regarding the upgrade issue and are working with our partners to improve the situation for existing customers. In all instances, Google's web store only passes your information along. The carrier then determines eligibility for a full or partial upgrade, as your service contract is with the carrier. You must be fully upgrade eligible in order to get this device at a discount. Stay tuned for more updates about upgrading! In the meantime, you can use the sticky post at the top of this forum to get the latest information and share any more thoughts you may have: http://www.google.com/support/forum/p/Google+Mobile/thread?tid=3c0fd3870ec370a8&#all Cheers, -XXX

UPDATE: I've received a variant of this text in a second message from another Google employee. It's good they are trying to deal with this mess. I should not be surprised at the form letter template employees work from.

Rusty Java: in-place updating a collection while iterating

Some areas of my Java grow rusty with time, as with all things. This code looks nice, but completely fails:

public static void main(final String... args) {
    final Set<Foo> foos = new HashSet<Foo>(asList(FOO));

    for (final Foo foo : foos)
        foos.addAll(foo.more);

    System.out.println("foos = " + new HashSet<Foo>(foos));
}

enum Foo {
    QUUX,
    BAZ(QUUX),
    BAR(BAZ),
    FOO(BAR, BAZ);

    private final Set<Foo> more;

    Foo(final Foo... more) {
        this.more = new HashSet<Foo>(asList(more));
    }
}

The goal is to collect all transitive dependencies. What is wrong?

The iterator is not a "current picture" of the collection; it is a snapshot. This code prints only foos = [FOO], updates vanish. Changing to ArrayList only worsens things with the dreaded ConcurrentModificationException.

Correct is to use a list iterator and be careful with cursor position:

public static void main(final String... args) {
    final List<Foo> foos = new ArrayList<Foo>(asList(FOO));
    final ListIterator<Foo> lit = foos.listIterator();

    while (lit.hasNext())
        for (final Foo foo : lit.next().more) {
            lit.add(foo);
            lit.previous();
        }

    System.out.println("foos = " + new HashSet<Foo>(foos));
}

This prints foos = [QUUX, BAZ, BAR, FOO] with the happy side effect of depth-first ordering.

Class token nuisance

Fabrizio Giudici posts a nice description of type-safe map keys, a clever way to simplify life for an API user.

It exposes one of the nuisances of class tokens:

@Nonnull
public <T> T get (final @Nonnull Key<T> key)
  throws NotFoundException
  {
    return NotFoundException.throwWhenNull(
        (T)itemMap.get(key), "Not found: " + key);
  }

(Kudos to him for using JSR-305 annotations.)

See the cast (T)? That generates a compile-time warning (link just one of many complaints around this). The correct way to get ride of this warning is to use a class token:

@Nonnull
public <T> T get (final @Nonnull Key<T> key)
  throws NotFoundException
  {
    return NotFoundException.throwWhenNull(
        key.cast(itemMap.get(key)), "Not found: " + key);
  }

Where the Key<T> type is redefined:

@Immutable
public final static class Key<T>
    implements Comparable<Key<T>>
  {
    @Nonnull
    private final String name;
    @Nonnull
    private final Class<T> type;

    protected Key (final @Nonnull String name, final @Nonnull Class<T> type)
      {
        this.name = name;
        this.type = type;
      }

    public T cast(final Object that)
      {
        return type.cast(that);
      }

    @Override
    public int compareTo (final @Nonnull Key<T> other) { ... }

    @Override
    public boolean equals (final @CheckForNull Object object) { ... }

    @Override
    public int hashCode() { ... }
  }

What a nuisance.

ANI, not your usual language

This is interesting: anic, a compiler for the language ANI. It is really the language which is the interesting part, of course.

ANI is automatically parallelizing, and very non-inperative. In its own buzzwords: experimental, high-performance, statically-safe, fully implicitly parallel, object-oriented, general-purpose dataflow programming language.

Take a breath now.

Bad fanboi - no gphone

Continuing my Nexus One story...

So far I cannot get a Google phone. At first the site did not like my T-mobile family plan. So I went from $60/mo. to $130/mo. to break up our plan into a his/hers data/voice plan.

Now Google still does not like my T-mobile plan:

You are not eligible for an upgrade at this time.
You are not eligible because your current contract was initiated too recently.

No love from Google.

IntelliJ and Scala

Thomas Jung writes about using IntelliJ and Scala and is very enthusiastic, as he should be: IntelliJ IDEA remains the best IDE.

I've long been pessimistic of IDEA retaining this position in the face of thousands of Eclipse code monkeys eventually typing the works of Knuth, but JetBrains has admirably kept up.

The struggle between the cathedral and the bazaar continues.

UPDATE: When it rains, it pours.

Those lucky web developers

Those lucky web developers have it so good. Automated functional testing for enterprise integration middleware is a complete pain. You try simulating a dozen distinct systems. Helps keep me warm at night just thinking about it.

Actor thinking from Kresten Krab Thorup

Kresten Krab Thorup posts a long think on objects and processes, full of great references, which I enjoy. Maybe you will, too.

isolate: a new tool for my toolbox

isolate is the newest tool in my toolbox. Paranoids are sometimes correct. (Found thanks to LWN.net.)

Nexus One URL almost there

What is the typical error page for a bad link into Google, say, http://www.google.com/bobsyerunkel? But for http://www.google.com/phone I get a prosaic default 404 page, perhaps in preparation for something more interesting later today.

UPDATE: Not happy! http://www.google.com/phone works now, but T-mobile is disabled as a choice. Only the unlocked phone is offered:

We're unable to sell T-Mobile service plans at this time, but you can still buy the unlocked phone.

This was my Christmas present from my wife!

UPDATE #2: I was premature. The T-mobile option now works, very nicely, too. However, it checked my existing T-mobile phone number and said family plans were not eligible. Time to confer with the wife.

UPDATE #3: Spent 20 minutes on line with T-mobile. She was courteous and did her best, but management left them unprepared for Nexus One release beyond a cursory memo. Several checks with different supervisors, still no answer on how to split family plan, add Nexus One plan for myself. There's always tomorrow.

Revisiting an old post: mixing OSGi, Scala, et al with pax

I overlooked the coolness of Brian Murphy's post the first time I ran across it: OSGi With Scala, Java, Groovy, Maven and PAX. As I am playing with OSGi, guice and peaberry, I ran across his instructions and was struck by how straight-forward pax makes everything.

My contribution to the conversation, bootstrapping a peaberry project:

$ pax-add-repository -i google-maven-repository \
    -u http://google-maven-repository.googlecode.com/svn/repository
$ pax-add-repository -i ops4j.releases \
    -u http://repository.ops4j.org/maven2
$ pax-import-bundle -g com.google.inject -a guice -v 2.0
$ pax-import-bundle -g org.ops4j -a peaberry -v 1.1.1
$ pax-create-module -a wrappers
$ cd wrappers
$ pax-wrap-jar -g aopalliance -a aopalliance -v 1.0
$ cd ..
$ pax-create-bundle -p $your_groupId -n $your_artifactId

This presumes a simple project with only one module ("$your_artifactId").

Top notch post on JDK 1.6 update 14 optimizations

A really good post on JDK 1.6 update 14 optimizations: lock coarsening, biased locking and escape analysis. Found via the Java Specialist newletter.

Google collections: Happy birthday!

Felicem diem natalem!