Wednesday, May 02, 2007

Increasing log4j verbosity

To programmatically increase log4j verbosity, a simple code snippet does the trick:

final Logger rootLogger = Logger.getRootLogger();

rootLogger.setLevel(Level.toLevel(
        rootLogger.getLevel().toInt() + 1,
        Level.ALL));

I use this to handle the -verbose command-line switch. Repeating the switch keeps increasing my program's verbosity.

Naturally there are wrinkles with a complex log4j configuration, but the idiom remains.

Tuesday, May 01, 2007

Subversion best practices, branching

A useful link on Subversion, Subversion Best Practices. Really nice are the links into the Red Bean book filling out each point.

What I am looking for at the moment is a good discussion on using Subversion to fork a project. I have internal libraries at work which are not ameniable to multiple teams working on them. My best option is to fork the code and merge back in fixes from the original.

This looks like a good start on this kind of branching. I like the example shell scripts for automating portions of the process.

Thursday, April 12, 2007

Good temporary files in Ant

I was reading this nice post on unsigning jars with Ant when this line caught my eye:

<tempfile destdir="${java.io.tmpdir}"
          prefix="unsign-"
          property="temp.file"/>

I was struck by an Aha! moment—using ${java.io.tmpdir} is clearly the Right Thing for creating temporary files in Java. Why not also in Ant?

Wednesday, April 11, 2007

The fallacy of soft coding

Today's Worse Than Failure has a brilliant post on soft coding.

It begins with a strong exposition answering "what is soft coding?" — the example business method is just about ideal for what a business method should be. I wish I could hire him!

The moral is simple: don't be a soft coder; in some respects it is worse than hard coding.

Most used tool

What is the most used tool in my daily work life? The Java compiler? IntelliJ IDEA? Perl?

Actually it is Mark Edgar's PuTTYcyg, a PuTTY patch for Cygwin terminal. This runs BASH from Cygwin inside a PuTTY xterm-ish window, making my daily work on Windows rather more productive.

You see, I'm a command-line sort of guy. And BASH is indispensable to my habits; hence Cygwin. And the default DOS box terminal is lame.

Using PuTTYcyg is a cinch. Download, unpack and add a shortcut to your desktop for C:\puttycyg-20070320\putty.exe - (or wherever you unpack to; the important part is the trailing solo hyphen). I usually change the icon to the Cygwin one (%SystemDrive%\cygwin\cygwin.ico, YMMV).

Tuesday, April 03, 2007

Google's excellent code coloring

This looks like fun:

<html>
<head>
<title>Example</title>
<link href="http://google-code-prettify.googlecode.com/svn/trunk/src/prettify.css" type="text/css" rel="stylesheet" />
<script type="text/javascript" src="http://google-code-prettify.googlecode.com/svn/trunk/src/prettify.js"></script>
</head>
<body onload="prettyPrint()">
<h1>C</h1>
<pre class="prettyprint">void
main(char* argv[])
{
    printf("Hello, Bob.\n");
}</pre>
<h1>Java</h1>
<pre class="prettyprint">public class Main {
    public static void main(final String[] args) {
        System.out.println("Hello, Bob.");
    }
}</pre>
<h1>Shell</h1>
<pre class="prettyprint">echo "Hello, Bob."</pre>
</body>
</html>

Friday, March 23, 2007

Writing JUnit 4 parameterized tests with varargs data

One of the many excellent new features of JUnit 4 is parameterized tests. These are data-driven tests where a test case runs repeatedly against a collection of test data, as if each run were its own test case.

A standard example:

@RunWith(Parameterized.class)
public class ExampleDataDrivenTest {
    public static final class Data {
        private final String name;

        public Data(final String name) {
            this.name = name;
        }

        @Override
        public String toString() {
            return name;
        }
    }

    private final List<Data> data;

    public ExampleDataDrivenTest(final Data datum1, final Data datum2) {
        System.out.println("ExampleDataDrivenTest");

        this.data = asList(datum1, datum2);
    }

    @Test
    public void dumpData() {
        for (final Data datum : data)
            System.out.println("datum = " + datum);
    }

    @Parameters
    public static Collection<Data[]> data() {
        final Data[][] data = new Data[][]{
                {new Data("apple"), new Data("core")}};

        return asList(data);
    }
}

This produces:

ExampleDataDrivenTest
datum = apple
datum = core

This works great for a fixed number of test data, but I want to test a variable number of data. The change is straight-forward once you recall that the varargs Java language feature is implemented as an array:

@RunWith(Parameterized.class)
public class ExampleDataDrivenTest {
    public static final class Data {
        private final String name;

        public Data(final String name) {
            this.name = name;
        }

        @Override
        public String toString() {
            return name;
        }
    }

    private final List<Data> data;

    public ExampleDataDrivenTest(final Data... data) {
        System.out.println("ExampleDataDrivenTest");

        this.data = asList(data);
    }

    @Test
    public void dumpData() {
        for (final Data datum : data)
            System.out.println("datum = " + datum);
    }

    @Parameters
    public static Collection<Data[][]> data() {
        final Data[][][] data = new Data[][][]{
                {{new Data("apple"), new Data("core")}}};

        return asList(data);
    }
}

Reflect on the constructor and you see:

public ExampleDataDrivenTest(ExampleDataDrivenTest$Data[])

This makes the change more obvious.

I can now write my annotated parameters as:

    @Parameters
    public static Collection<Data[][]> data() {
        final Data[][][] data = new Data[][][]{{{ /* no data */ }},
                {{new Data("apple"), new Data("core")}},
                {{new Data("baltimore")}}};

        return asList(data);
    }

To produce:

ExampleDataDrivenTest
ExampleDataDrivenTest
datum = apple
datum = core
ExampleDataDrivenTest
datum = baltimore

Saturday, March 10, 2007

Storing jars in a Maven project

One of the sweetest things about Maven is that depedencies are stored external to your sources: if you depend on Apache Commons Lang 2.3, for example, then you can stick this in the dependencies section of your pom.xml and stop worrying:

<depedency>
    <groupId>commons-lang</groupId>
    <artifactId>commons-lang</artifactId>
    <version>2.3</version>
</dependency>

But what do you do when your dependency is not available in a public Maven repository? For example, the library is too new to be widely available, or is proprietary like the Oracle JDBC driver?

One tack is to establish an internal repository and refer to that in your pom.xml. But sometimes that is not practical, perhaps because of corporate network use restrictions, or for very small projects.

Before Maven, common practice was to stash dependencies into a directory named lib/ or similar underneath the root of your project sources. When no other solution presents itself, you can follow that practice but keep using Maven.

In your pom.xml, add this top-level section:

<repositories>
    <repository>
        <id>project</id>
        <name>Project Maven Repository</name>
        <layout>default</layout>
        <url>file:///./lib</url>
    </repository>
</repositories>

This establishes the lib/ directory underneath your project as a Maven repository relative to your project root. Now you can stash dependencies there, store them with the rest of your project resources, but keep using Maven.

A little bit more

Often I have to re-lookup the documentation on deploy:deploy-file so that I can put a dependency in an internal repository or in the project repository as described above. Here is a magic incantation example:

$ cd /PATH-TO-guice-1.0-DOWNLOAD
# Checkout the SVN 1.0 tag sources to src/
# Note -- need to use the -Dversion=1.0 switch
# because pom says 1.0-RC2 in spite of tag.  Then:
$ mvn deploy:deploy-file \
    -Durl=file://C:/FULL-PROJECT-PATH/lib \
    -DrepositoryId=project -Dfile=guice-1.0.jar \
    -DpomFile=src/pom.xml -Dversion=1.0
# Pack up the javadocs and sources into jars, then:
$ mvn deploy:deploy-file \
    -Durl=file://C:/FULL-PROJECT-PATH/lib \
    -DrepositoryId=project \
    -Dfile=guice-1.0-javadocs.jar \
    -DpomFile=src/pom.xml -Dversion=1.0 \
    -Dclassifier=javadocs
$ mvn deploy:deploy-file \
    -Durl=file://C:/FULL-PROJECT-PATH/lib \
    -DrepositoryId=project \
    -Dfile=guice-1.0-sources.jar \
    -DpomFile=src/pom.xml -Dversion=1.0 \
    -Dclassifier=sources

I now have the splediferous Guice 1.0 dependency in my project repository, and can use it in my pom.xml with:

<depedency>
    <groupId>com.google.inject</groupId>
    <artifactId>guice</artifactId>
    <version>1.0</version>
</dependency>

Saturday, February 17, 2007

A TreeList implementation

Following up on my previous post for a tree list collection in Java, here is an implementation in two parts.

First is an abstract base for tree lists along the lines of AbstractSequentialList, but with an important difference.

For the standard Collection classes, the elements of the collection are the only objects under consideration, and all elements are explicitly manipulated by calls to the collection: addition, ordering and removal. In contrast, TreeList considers both the elements themselves—the values—and the nodes which refer to those elements.

Because of this dual life, Treelist must manipulate nodes implicitly to shadow elements manipulated explicitly. And to preserve proper typing, TreeList must have a way to create nodes beyond what is possible presently with straight-forward generics.

Why? To permit subclassing of TreeList classes. Unfortunately, that effort is the cause of the several TODO markers in the sample code. Soon, I think, I will revisit this point with Gafter's Gadget.

(Yes, the repeated reference around the same material is intentional. Gafter was rather inspired for his idea, and it points to a near certain improvement in the language for Java 7.)

Problems aside, this code illustrates the good design of the Java Collections and relies heavily on the extension points provided for collection implementors.

Without further ado:

/**
* {@code LinkedTreeList} is a linked node implementation of {@code TreeList}
* using an array list of children.  It does not support mixing nodes between it
* and other implementations of {@code TreeList}.
* <p/>
* It a <em>sequential list</em>, not a random-access one.  Although it supports
* all list operations, {@code LinkedTreeList} performs similarly to {@link
* LinkedList} rather than {@link ArrayList}.  For example, {@link #size()} is
* an <em>O(N)</em> operation rather than <em>O(1)</em>.  In constrast,
* insertion and deletion are very efficient.
*
* @author <a href="mailto:binkley@alumni.rice.edu">B. K. Oxley (binkley)</a>
* @todo How to make extension type-safe w/o hosing generics syntax?
*/
public abstract class AbstractTreeList<E, C extends AbstractTreeList<E, C>>
       extends AbstractSequentialList<E>
       implements com.jpmorgan.eggplant.util.TreeList<E> {
   private final List<C> children = new ArrayList<C>();

   private C parent;
   private E value;

   protected AbstractTreeList(final E rootValue) {
       this(null, rootValue);
   }

   protected AbstractTreeList(final C parent, final E value) {
       setParent(parent);
       setValue(value);
   }

   protected AbstractTreeList(final C parent, final int index, final E value) {
       setParent(parent, index);
       setValue(value);
   }

   /**
    * @todo Smell.
    */
   protected abstract C newInstance(final C parent, final E value);

   /**
    * @todo Smell.
    */
   protected abstract C newInstance(final C parent, final int index,
           final E value);

   // TreeList

   public boolean isRoot() {
       return null == parent;
   }

   public C getParent() {
       if (isRoot())
           throw new NoSuchElementException();

       return parent;
   }

   public boolean isLeaf() {
       return children.isEmpty();
   }

   public List<TreeList<E>> children() {
       // TODO: How to avoid wonky cast?
       return (List) new Children();
   }

   public TreeList<E> addChild(final E value) {
       return newInstance(self(), value);
   }

   public TreeList<E> addChild(final int index, final E value) {
       return newInstance(self(), index, value);
   }

   /**
    * @todo Merge with {@link Children#remove}.
    */
   public void removeChild(final TreeList<E> child) {
       if (!(child instanceof AbstractTreeList))
           throw new NoSuchElementException();

       ((C) child).setParent(null);
   }

   public TreeList<E> removeChild(final int index) {
       return children().remove(index);
   }

   public boolean removeAllChildren(final Collection<TreeList<E>> c) {
       return children().removeAll(c);
   }

   public boolean retainAllChildren(final Collection<TreeList<E>> c) {
       return children().retainAll(c);
   }

   public E getValue() {
       return value;
   }

   public void setValue(final E value) {
       this.value = value;
   }

   // AbstractSequentialList

   @Override
   public ListIterator<E> listIterator(final int index) {
       if (0 > index)
           throw new IndexOutOfBoundsException();

       final RootValueListIterator lit = new RootValueListIterator();

       for (int i = 0; i < index; ++i)
           if (lit.hasNext())
               lit.next();
           else
               throw new IndexOutOfBoundsException();

       return lit;
   }

   // AbstractCollection

   @Override
   public int size() {
       final ListIterator<E> lit = listIterator();

       while (lit.hasNext())
           lit.next();

       return lit.nextIndex();
   }

   private C self() {
       return (C) this;
   }

   private void adopt(final C parent, final Adopter<C> adopter) {
       if (this.parent == parent)
           return;

       if (this == parent)
           throw new IllegalArgumentException();

       if (null != this.parent)
           siblings().remove(this);

       if (null != (this.parent = parent))
           adopter.adopt(self());
   }

   private List<C> siblings() {
       return parent.children;
   }

   private void setParent(final C parent) {
       adopt(parent, new Adopter<C>() {
           public void adopt(final C child) {
               siblings().add(self());
           }
       });
   }

   private void setParent(final C parent, final int index) {
       adopt(parent, new Adopter<C>() {
           public void adopt(final C child) {
               siblings().add(index, self());
           }
       });
   }

   private interface Adopter<C> {
       void adopt(final C child);
   }

   private class RootValueListIterator
           implements ListIterator<E> {
       private final List<ListIterator<C>> previousNodeIts
               = new ArrayList<ListIterator<C>>();
       private ListIterator<C> currentNodeIt = (ListIterator<C>) singletonList(
               AbstractTreeList.this).listIterator();
       private ListIterator<C> nextNodeIt = (ListIterator<C>) children()
               .listIterator();
       private C currentNode = (C) AbstractTreeList.this;

       private int index;

       public boolean hasNext() {
           return currentNodeIt.hasNext() || nextNodeIt.hasNext();
       }

       public E next() {
           ++index;

           if (currentNodeIt.hasNext())
               return (currentNode = currentNodeIt.next()).getValue();

           if (!nextNodeIt.hasNext())
               throw new NoSuchElementException();

           previousNodeIts.add(currentNodeIt);
           currentNodeIt = nextNodeIt;

           if (!currentNodeIt.hasNext())
               throw new NoSuchElementException();

           currentNode = currentNodeIt.next();
           nextNodeIt = (ListIterator<C>) currentNode.children().listIterator();

           return currentNode.getValue();
       }

       public boolean hasPrevious() {
           return currentNodeIt.hasPrevious()
                   || !previousNodeIts.isEmpty() && previousNodeIts
                   .get(previousNodeIts.size() - 1).hasPrevious();
       }

       public E previous() {
           --index;

           if (currentNodeIt.hasPrevious())
               return (currentNode = currentNodeIt.previous()).getValue();

           if (previousNodeIts.isEmpty())
               throw new NoSuchElementException();

           nextNodeIt = currentNodeIt;
           currentNodeIt = previousNodeIts.remove(previousNodeIts.size() - 1);

           if (!currentNodeIt.hasPrevious())
               throw new NoSuchElementException();

           return (currentNode = currentNodeIt.previous()).getValue();
       }

       public int nextIndex() {
           return index;
       }

       public int previousIndex() {
           return index - 1;
       }

       public void remove() {
           final C remove = currentNode;

           // TODO: Testing, thinking, testing!
           if (hasPrevious()) {
               previous();
               remove.setParent(null);
               if (hasNext())
                   next();
           } else if (hasNext()) {
               next();
               remove.setParent(null);
               if (hasPrevious())
                   previous();
           } else {
               throw new UnsupportedOperationException(
                       "Cannot remove only node.");
           }

           // For fail-fast iterators - see {@link #modCount}.
           ++modCount;
       }

       public void set(final E o) {
           currentNode.setValue(o);
       }

       public void add(final E o) {
           currentNode.addChild(0, o);

           // For fail-fast iterators - see {@link #modCount}.
           ++modCount;
       }
   }

   private class Children
           extends AbstractList<C> {
       @Override
       public C get(final int index) {
           return children.get(index);
       }

       @Override
       public int size() {
           return children.size();
       }

       @Override
       public void add(final int index, final C element) {
           if (null == element)
               throw new NullPointerException();
           if (children.contains(element))
               throw new IllegalArgumentException();

           element.setParent(self(), index);
       }

       @Override
       public C set(final int index, final C element) {
           if (null == element)
               throw new NullPointerException();
           if (children.contains(element))
               throw new IllegalArgumentException();

           final C oldChild = get(index);

           oldChild.setParent(null);
           element.setParent(self(), index);

           return oldChild;
       }

       @Override
       public C remove(final int index) {
           final C child = get(index);

           child.setParent(null);

           return child;
       }
   }
}

Finally is a concrete class putting it all together:

/**
* {@code LinkedTreeList} is a linked node implementation of {@code TreeList}
* using an array list of children.  It does not support mixing nodes between it
* and other implementations of {@code TreeList}.
* <p/>
* It a <em>sequential list</em>, not a random-access one.  Although it supports
* all list operations, {@code LinkedTreeList} performs similarly to {@link
* LinkedList} rather than {@link ArrayList}.  For example, {@link #size()} is
* an <em>O(N)</em> operation rather than <em>O(1)</em>.  In constrast,
* insertion and deletion are very efficient.
*
* @author <a href="mailto:binkley@alumni.rice.edu">B. K. Oxley (binkley)</a>
* @todo How to make extension type-safe w/o hosing generics syntax?
*/
public class LinkedTreeList<E>
       extends AbstractTreeList<E, LinkedTreeList<E>> {
   public LinkedTreeList(final E rootValue) {
       super(rootValue);
   }

   protected LinkedTreeList(final LinkedTreeList<E> parent, final E value) {
       super(parent, value);
   }

   protected LinkedTreeList(final LinkedTreeList<E> parent, final int index,
           final E value) {
       super(parent, index, value);
   }

   @Override
   protected LinkedTreeList<E> newInstance(final LinkedTreeList<E> self,
           final E value) {
       return new LinkedTreeList<E>(self, value);
   }

   @Override
   protected LinkedTreeList<E> newInstance(final LinkedTreeList<E> self,
           final int index, final E value) {
       return new LinkedTreeList<E>(self, index, value);
   }
}

Saturday, February 10, 2007

TreeList, a Java collection

One hole in the Java collection classes is no tree. I have seen in production code in a prior job in non-UI software developers choose to use Swing JTree as a replacement collection for a tree data structure.

Rather than take that tortured path, here is a straight-forward tree interface:

/**
* {@code TreeList} is a list with left-to-right, depth-first traversal over a
* tree.  There is no limitation to the number of child nodes for a given node.
* <p/>
* Ordinary {@code List} operations behave as if they were operation on a list
* of node values.  For example, given a any tree list node, say
* <var>node</var>, then {@code node.getValue()} and {@code node.get(0)} are the
* same object.
*
* @author <a href="mailto:binkley@alumni.rice.edu">B. K. Oxley (binkley)</a>
*/
public interface TreeList<E>
       extends List<E> {
   /**
    * Checks if this node is a root node, this is, has no parent.
    *
    * @return {@code true} if a root node
    */
   boolean isRoot();

   /**
    * Gets the parent of this node or throw {@code NoSuchElementException} if
    * this is a root node, hence, never {@code null}.
    *
    * @return the node parent, if any
    *
    * @throws NoSuchElementException if this is a root node
    * @see #isRoot()
    */
   TreeList<E> getParent();

   /**
    * Checks if this node is a leaf node, that is, has no children.
    *
    * @return {@code true} if a leaf node
    */
   boolean isLeaf();

   /**
    * Gets the list of child nodes or an empty list if none, hence, never
    * {@code null}.
    * <p/>
    * The list is read-write.  Changes to the list are reflected in the tree.
    *
    * @return the child nodes, if any, or the empty list
    */
   List<TreeList<E>> children();

   /**
    * Adds a child node to this node with the given <var>value</var> to the end
    * of the list of children.  The new node has this node as its parent.
    *
    * @param value the new child node value
    *
    * @return the child node
    */
   TreeList<E> addChild(final E value);

   /**
    * Adds a child node to this node with the given <var>value</var> to the
    * <var>index</var> position in the list of children.  The new node has this
    * node as its parent.
    *
    * @param index the new child position, 0-based
    * @param value the new child node value
    *
    * @return the child node
    *
    * @throws IndexOutOfBoundsException if <var>index</var> is negative or
    * larger than the count of children of this node
    */
   TreeList<E> addChild(final int index, final E value);

   /**
    * Removes the given <var>child</var> node from this node.
    *
    * @param child the child node to remove
    *
    * @throws NoSuchElementException if <var>child</var> is not a child of this
    * node
    */
   void removeChild(final TreeList<E> child);

   /**
    * Removes the child node at the given <var>index</var> position from this
    * node, and returns it.  Afterwards, the returned node is a <em>root
    * node</em>.
    *
    * @param index the child position, 0-based
    *
    * @return the removed node
    *
    * @throws IndexOutOfBoundsException if <var>index</var> is negative or
    * larger than the count of children of this node
    */
   TreeList<E> removeChild(final int index);

   /**
    * Removes the children nodes in the given collection, <var>c</var>.
    *
    * @param c the child nodes to remove
    *
    * @return {@code true} if the tree changed as a result of this call
    */
   boolean removeAllChildren(final Collection<TreeList<E>> c);

   /**
    * Removes all but the children nodes in the given collection,
    * <var>c</var>.
    *
    * @param c the child nodes to retain
    *
    * @return {@code true} if the tree changed as a result of this call
    */
   boolean retainAllChildren(final Collection<TreeList<E>> c);

   /**
    * Gets the value of this node.  Ordinary {@code List} operations behave as
    * if they were operation on a list of node values.
    *
    * @return the node value
    */
   E getValue();

   /**
    * Sets the value of this node to the given <var>value</var>.   Ordinary
    * {@code List} operations behave as if they were operation on a list of
    * node values.
    *
    * @param value the new value
    */
   void setValue(final E value);
}

An implementation to follow in a later post.

Thursday, February 01, 2007

Outstanding post on closures

Neal Gafter, a co-author of the closure proposal, has just posted an outstanding entry on closures.

In it he describes not just what is happening for Java, but the history of closures, touching on LISP, Scheme and Smalltalk. A tour de force of a technical blog post.

Thursday, January 18, 2007

Controlling a blocking queue in Java

Recently I needed a blocking queue in Java in which I could pause the reader side. Normal behavior is for queue.take() to block so that a reader only runs when there are elements in the queue to process. What I wanted was for there to also be a throttle, so I could pause reading even when there were elements to process. It is not difficult, in fact, to make such a queue in which either end might be paused or stopped, and resumed or restarted:

public class LinkedControllableBlockingQueue<E>
        extends LinkedBlockingQueue<E>
        implements ControllableBlockingQueue<E> {
    // Read side
    private final Lock readLock = new ReentrantLock();
    private final Condition resumed = readLock.newCondition();
    private final AtomicReference<Thread> pausingThread
            = new AtomicReference<Thread>();

    // Write side
    private final Lock writeLock = new ReentrantLock();
    private final Condition restarted = writeLock.newCondition();
    // Begin started
    private final AtomicReference<Thread> startingThread
            = new AtomicReference<Thread>(currentThread());

    // Controllable

    public boolean isStarted() {
        return null != startingThread.get();
    }

    public boolean isPaused() {
        return null != pausingThread.get();
    }

    public void start() {
        if (!startingThread.compareAndSet(null, currentThread()))
            throw new IllegalStateException("Already started");

        writeLock.lock();
        try {
            restarted.signalAll();
        } finally {
            writeLock.unlock();
        }
    }

    public void pause() {
        if (!pausingThread.compareAndSet(null, currentThread()))
            throw new IllegalStateException("Already paused");
    }

    public void resume() {
        if (!pausingThread.compareAndSet(currentThread(), null))
            throw new IllegalStateException("Not paused by current thread");

        readLock.lock();
        try {
            resumed.signalAll();
        } finally {
            readLock.unlock();
        }
    }

    public void stop() {
        if (!startingThread.compareAndSet(currentThread(), null))
            throw new IllegalStateException("Not started by current thread");
    }

    // LinkedBlockingQueue

    // Read side

    @Override
    public E take()
            throws InterruptedException {
        readLock.lock();
        try {
            while (isPaused())
                resumed.await();
        } finally {
            readLock.unlock();
        }

        return super.take();
    }

    /**
     * {@inheritDoc}
     * <p/>
     * Time spent <em>paused</em> is subtracted from <var>timeout</var>.  That
     * is, the total time spent in {@code poll} is the same for the pausing and
     * non-pausing versions of {@code LinkedBlockingQueue} for a given
     * <var>timeout</var> and <var>unit</var>.
     */
    @Override
    public E poll(final long timeout, final TimeUnit unit)
            throws InterruptedException {
        long nanos = unit.toNanos(timeout);

        readLock.lock();
        try {
            while (isPaused())
                if (0 >= (nanos = resumed.awaitNanos(nanos)))
                    return null;
        } finally {
            readLock.unlock();
        }

        return super.poll(nanos, NANOSECONDS);
    }

    /**
     * {@inheritDoc}
     * <p/>
     * Returns {@code null} if paused.
     */
    @Override
    public E poll() {
        readLock.lock();
        try {
            if (isPaused())
                return null;
        } finally {
            readLock.unlock();
        }

        return super.poll();
    }

    /**
     * {@inheritDoc}
     * <p/>
     * Returns {@code null} if interrupted while paused.
     */
    @Override
    public E peek() {
        readLock.lock();
        try {
            while (isPaused())
                resumed.await();
        } catch (final InterruptedException e) {
            return null;
        } finally {
            readLock.unlock();
        }

        return super.peek();
    }

    /**
     * {@inheritDoc}
     * <p/>
     * Returns {@code false} if interrupted while paused.
     */
    @Override
    public boolean remove(final Object o) {
        readLock.lock();
        try {
            while (isPaused())
                resumed.await();
        } catch (final InterruptedException e) {
            return false;
        } finally {
            readLock.unlock();
        }

        return super.remove(o);
    }

    /**
     * {@inheritDoc}
     * <p/>
     * Also throws {@code NoSuchElementException} if interrupted while paused.
     */
    @Override
    public E remove() {
        readLock.lock();
        try {
            while (isPaused())
                resumed.await();
        } catch (final InterruptedException e) {
            throw new NoSuchElementException(e.getMessage());
        } finally {
            readLock.unlock();
        }

        return super.remove();
    }

    // Write side

    @Override
    public void put(final E o)
            throws InterruptedException {
        writeLock.lock();
        try {
            while (!isStarted())
                restarted.await();
        } finally {
            writeLock.unlock();
        }

        super.put(o);
    }

    /**
     * {@inheritDoc}
     * <p/>
     * Time spent <em>stopped</em> is subtracted from <var>timeout</var>.  That
     * is, the total time spent in {@code offer} is the same for the stopping
     * and non-stopping versions of {@code LinkedBlockingQueue} for a given
     * <var>timeout</var> and <var>unit</var>.
     */
    @Override
    public boolean offer(final E o, final long timeout, final TimeUnit unit)
            throws InterruptedException {
        long nanos = unit.toNanos(timeout);

        writeLock.lock();
        try {
            while (!isStarted())
                if (0 >= (nanos = restarted.awaitNanos(nanos)))
                    return false;
        } finally {
            writeLock.unlock();
        }

        return super.offer(o, nanos, NANOSECONDS);
    }

    /**
     * {@inheritDoc}
     * <p/>
     * Also returns {@code false} if interrupted while stopped.
     */
    @Override
    public boolean offer(final E o) {
        writeLock.lock();
        try {
            while (!isStarted())
                restarted.await();
        } catch (final InterruptedException e) {
            return false;
        } finally {
            writeLock.unlock();
        }

        return super.offer(o);
    }

    /**
     * {@inheritDoc}
     * <p/>
     * Also returns {@code false} if stopped.
     */
    @Override
    public boolean add(final E o) {
        return isStarted() && super.add(o);
    }
}

Wednesday, December 27, 2006

Making a one-jar with Maven

(This is a lengthier post than usual. You can skip forward to the finished example, if you like.)

UPDATE:The One-JAR author comments:

Nice article, thanks for using (and persisting with) One-JAR. I just released a new version (0.97), checkout http://one-jar.sourceforge.net/

The one-jar-maven plugin has been updated with this new release, and there is also a new project in the CVS repository showing how to use maven2 to build (http://one-jar.cvs.sourceforge.net/viewvc/one-jar/one-jar-maven/)

A major part of this release was making it easier to set up One-JAR projects, to which end there is an application generator (one-jar-appgen) which will create a basic One-JAR directory tree, with support for building under Eclipse/Ant, and which contains a JUnit test harness. I’d be interested in hearing your thoughts on this if you’re still working with the product.

One-JAR

I recently came across Simon Tuffs' One-JAR Java project. It uses a custom "boot" classloader to place whole JARs within JARs and fix up the classpath accordingly.

The standard classloader will not do this and requires that supporting JARs be external to each other. So if I have Main.jar with dependency log4j.jar where Main.jar holds the main entry point—hm.binkley.Main, say—I cannot bundle log4j.jar into Main.jar but must distribute it alongside separately.

This has no effect on running Java, but changes distribution as I can no longer provide a single file (Main.jar in this example) for others to run.

Contrast these two ZIP files:

  • Foo-1.0-with-one-jar.zip:
    • foo-1.0/Main.jar
    • foo-1.0/README
  • Foo-1.0-without-one-jar.zip:
    • foo-1.0/Main.jar
    • foo-1.0/log4j.jar
    • foo-1.0/README

In this example, it does not seem to make a lot of difference; but when a project pulls in several Jakarta Commons JARs, several other open-source JARs, and several in-house, proprietary JARs, it becomes very obvious that distribution is an issue. One project I work on has a java command line between 5000 and 6000 characters long because of external jar dependencies in the classpath.

A traditional solution is to repack all the classes, internal and external, in to a single über-JAR, losing the independent qualities of each JAR including interesting MANIFEST.MF entries.

One-JAR solves this elegantly by packing everything into one JAR without unpacking the dependencies. My command line becomes just:

$ java [-options] -jar Main.jar

Ant

An Ant script for One-JAR is simple. One-JAR requires that you pack your original Main.jar into the final, single JAR along with dependencies:

  • main/Main.jar
  • lib/log4j.jar

They pack in with the One-JAR classes.

One-JAR then provides a custom classloader and alternative main entry point to glue it all together and a custom URL for classloading, onejar:.

As One-JAR provides you with a prototype outer JAR, all you need do is update the prototype, adding in main/Main.jar and lib/log4j.jar with Main.jar!main/Main.jar!META-INF/MANIFEST.MF unchanged:.

Manifest-Version: 1.0
Main-Class: hm.binkley.Main
Class-Path: log4j.jar

Leave unchanged the Main.jar!META-INF/MANIFEST.MF provided in the prototype:

Manifest-Version: 1.0
Main-Class: com.simontuffs.onejar.Boot

(Note that the first Main.jar is the prototype copied from one-jar-boot-0.95.jar, provided with the One-JAR download, and the second Main.jar is your original executable JAR.)

One-JAR follows the convention that your "real" JAR is in main/ and all dependency JARs are elsewhere within your single, distributable JAR. Nothing is unpacked.

The jar task handles this ably:

<target name="one-jar" depends="jar" description="Build one ONE-JAR">
    <property name="onejardir" location="${pom.build.directory}/one-jar"/>
    <mkdir dir="${onejardir}/main"/>
    <mkdir dir="${onejardir}/lib"/>

    <copy tofile="${onejardir}/${jarfile}"
            file="lib/one-jar-boot-0.95.jar"/>
    <copy todir="${onejardir}/main"
            file="${pom.build.directory}/${jarfile}"/>
    <copy todir="${onejardir}/lib" flatten="true">
        <fileset dir="lib" includes="*.jar"
                excludes="one-jar-boot-0.95.jar"/>
        <fileset refid="runtime.dependency.fileset"/>
    </copy>

    <jar jarfile="${onejardir}/${jarfile}" update="true"
            basedir="${onejardir}" excludes="${jarfile}"/>
</target>

But what of Maven?

Maven

Where Ant asks, How do I make this cake? - Maven asks Where can I find cake ingredients? But Maven does provide a mechanism for adding new recipes, the assembly plugin.

An assembly is a description of packaging for Maven, and is usually hooked into the package phase in your build lifecycle. (The assembly plugin is much simpler than adding new packaging with Plexus, the method described in the link.)

To build a One-JAR with Maven and the assembly plugin, add a new assembly descriptor which follows the outline of building with Ant, taking care to unpack the One-JAR prototype JAR and add to it Main.jar and its dependencies.

This is better explained by example.

Example

The sources

Unfortunately, One-JAR is not in the Maven central repository, so it is included here as part of the project. That is the reason for the added repository in the POM and the lib/ files. Ignoring directories:

  • lib/com/simontuffs/one-jar/0.95/one-jar-0.95.jar
  • lib/com/simontuffs/one-jar/0.95/one-jar-0.95.pom
  • pom.xml
  • src/assembly/one-jar.xml
  • src/main/java/hm/binkley/Main.java
  • src/main/resources/log4j.properties

The POM

<project xmlns="http://maven.apache.org/POM/4.0.0"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
                             http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>hm.binkley</groupId>
    <artifactId>Main</artifactId>
    <packaging>jar</packaging>
    <version>1.0</version>
    <name>One-JAR Example</name>
    <url>http://binkley.blogspot.com/</url>

    <repositories>
        <repository>
            <id>project</id>
            <name>Project Repository</name>
            <url>file:///${basedir}/lib</url>
            <layout>default</layout>
        </repository>
    </repositories>

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-jar-plugin</artifactId>
                <configuration>
                    <archive>
                        <manifest>
                            <mainClass>hm.binkley.Main</mainClass>
                            <addClasspath>true</addClasspath>
                        </manifest>
                    </archive>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <configuration>
                    <archive>
                        <manifest>
                            <mainClass>com.simontuffs.onejar.Boot</mainClass>
                        </manifest>
                    </archive>
                    <descriptors>
                        <descriptor>src/assembly/one-jar.xml</descriptor>
                    </descriptors>
                </configuration>
                <executions>
                    <execution>
                        <id>make-assembly</id>
                        <phase>package</phase>
                        <goals>
                            <goal>attached</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>com.simontuffs</groupId>
            <artifactId>one-jar</artifactId>
            <version>0.95</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.14</version>
        </dependency>
    </dependencies>
</project>

The assembly

In src/main/assembly/one-jar.xml:

<assembly>
    <id>one-jar</id>
    <formats>
        <format>jar</format>
    </formats>
    <includeBaseDirectory>false</includeBaseDirectory>
    <dependencySets>
        <dependencySet>
            <outputDirectory/>
            <unpack>true</unpack>
            <includes>
                <include>com.simontuffs:one-jar</include>
            </includes>
        </dependencySet>
        <dependencySet>
            <outputDirectory>main</outputDirectory>
            <includes>
                <include>${groupId}:${artifactId}</include>
            </includes>
        </dependencySet>
        <dependencySet>
            <outputDirectory>lib</outputDirectory>
            <scope>runtime</scope>
            <excludes>
                <exclude>com.simontuffs:one-jar</exclude>
                <exclude>${groupId}:${artifactId}</exclude>
            </excludes>
        </dependencySet>
    </dependencySets>
</assembly>

The finished One-JAR

  • META-INF/MANIFEST.MF
  • com/simontuffs/onejar/Boot.class
  • com/simontuffs/onejar/Boot.java
  • com/simontuffs/onejar/Handler$1.class
  • com/simontuffs/onejar/Handler.class
  • com/simontuffs/onejar/Handler.java
  • com/simontuffs/onejar/JarClassLoader$ByteCode.class
  • com/simontuffs/onejar/JarClassLoader.class
  • com/simontuffs/onejar/JarClassLoader.java
  • doc/one-jar-license.txt
  • main/Main-1.0.jar
  • lib/log4j-1.2.14.jar

Thursday, December 14, 2006

Brilliant: C to MIPS to Java bytecode

NestedVM is one of the cleverest solutions to the problem of legacy C and C++ code I have ever seen: use GCC to compile to MIPS machine instructions, then translate those directly into Java bytecode.

There is also a good slideshow.

A straight-forward example is a pure-Java translation of the SQLite JDBC driver.

Wednesday, November 22, 2006

What is the type of this in Java?

I was late to read: Peter Ahé has a wonderful post on the type of this.

Among the gems:

  • Design issues with generics
  • A lengthy, well-developed example
  • Laird Nelson is a great name

Thursday, November 02, 2006

Java enum misuse

The Java enum is a nice feature, but like anything it is open to abuse. Witness:

enum OverTheTop {
    A, B;

    private String value;

    public String getValue() {
        return value;
    }

    public void setValue(final String value) {
        this.value = value;
    }
}

Even though the instances of OverTheTop are fixed and immutable, enums are after all simply class instances and can be use anyway a class can be used. This is the same problem with unmodifiable collections having modifiable members.

You could view enums as a generalization of singletons, having a fixed number of members not limited to just one. (Class loading is thread safe — does this mean one can use a single-member enum to implement a thread-safe singleton?)

As abusive as enums with setters seems, worse still remains:

enum TrulyUnpredictable {
    A, B;

    private Runnable runnable;

    public void setRunnable(final Runnable runnable) {
        this.runnable = runnable;
    }

    public void run() {
        new Thread(runnable).start();
    }
}

As unmaintainable as this seems (what does TrulyUnpredictable.A.run() do?), there are legitimate uses for complex enums.

Say you have a limited set of algorithm implementations and would like to switch on them:

enum Algorithm {
    FAST(ImplementationFactory.getInstance("FAST")),
    ACCURATE(ImplementationFactory.getInstance("ACCURATE")),
    EFFICIENT(ImplementationFactory.getInstance("EFFICIENT"));

    private final Implementation implementation;

    Algorithm(final Implementation implementation) {
        this.implementation = implementation;
    }

    public double calculate() {
        return implementation.calculate();
    }
}

And for the fan of difficult syntax:

enum Algorithm {
    FAST() {
        public double calculate() {
            new FastImplementation().calculate();
        }
    },
    ACCURATE() {
        public double calculate() {
            new AccurateImplementation().calculate();
        }
    },
    EFFICIENT() {
        public double calculate() {
            new EfficientImplementation().calculate();
        }
    };

    public abstract double calculate();
}

There are very many variations on these themes, but it is not too difficult to reach the point where no one can read your code except you.

Monday, October 30, 2006

The null collection anti-pattern

NEVER do this:

public Collection<T> doTheMacarena() {
    if (isPresidentialElection())
        return null;

    return visitSouthAmerica();
}

ALWAYS throw exceptions or return empty collections; never use null as a kind of "soft exception". Either:

if (isPresidentialElection())
    throw new UnwiseCampaignTacticException();

Or:

if (isPresidentialElection())
    return Collections.emptySet();

Are preferable, depending on whether the return-condition is fast-failure or the "no data" sort.

How would you like to spend your day debugging:

for (final Dancer dander : doTheMacarena())
    dancer.dazzleWithSmoothMoves();

From inside miles of nested code across several transaction and remote boundaries?

UPDATE: A nice post from Nishanth Shastry on the for-each loop. Look at item #6, Return zero length arrays or empty lists rather than nulls.

Monday, October 16, 2006

Live or Memorex?

The Daily WTF has run a series of posts on Virtudyne, a $200 million spectacular failure of a software company.

Many commenters have suggested that the anonymized company referred to throughout is my former employer, SimDesk.

I signed a document upon leaving SimDesk which prevents me from expressing my opinion about such things, but I hope you might enjoy reading the articles nonetheless. And please sweep for bugs, just to be sure.

Sunday, October 01, 2006

Correction for read-only Java objects

The problem

In my own code I sometimes bypass a getter for a final field in favor of direct field access for struct-like classes:

public class BigFoot {
    public final int shoeSize;

    public BigFoot(final int shoeSize) {
        this.shoeSize = shoeSize;
    }
}

I figure: why not? As the field is final there is no need for modifying behavior in a getter method. This seems ideal for data transfer objects (DTOs).

However Miles Barr points out the big flaw with this idea. It is subtle and has to with how the Java compiler deals with direct field access. Miles explains the problem as well as anyone and has a clear example. Fundamentally, you lose polymorphism for shadowed variables and bad things result.

The solution

The mistake for my DTO was not carrying the notion of simple struct-ness to its full conclusion. How much sense does it make to extend DTOs?

public final class BigFoot {
    public final int shoeSize;

    public BigFoot(final int shoeSize) {
        this.shoeSize = shoeSize;
    }
}

If the class is marked final then there will never be a problem with shadowing fields in subclasses.

A caveat

Sometimes I want to extend my DTOs when I have a set of similar data to represent and extract out the common fields in to a base class.

But this is an implementation detail. Logically there is no class relationship among the DTOs and the problem with shadowed fields is one which exposes an implementation detail to the user.

So don't do that.

Wednesday, September 20, 2006

Tom Ball's Hacking javac

Tom Ball posts a chok-full-of-nuts run through of the nifty javac happenings comming out of JDK6, leading to things like Sorcerer.

Normally I avoid posts which simply link to someone else (I rely on Erik's Linkblog for that beneficial service, but he's on vacation right now), but Ball is really on the ball with this one. I can't wait to try these out in a real project.

Wednesday, September 13, 2006

JPMorgan Investment Bank

I've been hired on this week to a permanent position with JPMorgan Investment Bank—no more contracting for me.

It's a double pleasure: I've been picked by a team with former ThoughtWorkers, an all-agile team. After getting some preliminaries out of the way, today I'll dial in for my first daily standup in several years.

I was just reading Tim Ottinger in More Work? discuss some of the benefits of agile development. It's great to see a smart place like JPMorgan with agile teams and a management that believes in its staff.

Wednesday, September 06, 2006

The continuing influence of Smalltalk

My friend Paul Holser likes to point out to me how great Smalltalk is. Just this past weekend he and I were chatting about it at the Rice's opening game (we beat the spread).

Lo and behold, why the lucky stiff mentions on 06 Sep 2006 at 13:17 (emphasis mine):

So, Matz cites four reasons for this decision:

  1. The spread of ActiveSupport has increased the need for strings and symbols to be united as hash keys.
  2. To address RCR 342 , which would allow sorting of symbols. (Try: Symbol.all_symbols.sort.)
  3. Smalltalk’s symbols are a subclass of string. (He adds that this is his most motivating reason to do it.)
  4. Using symbols as immediate values can cause them to venture into pointer territory, particularly on OSX .

Smalltalk never dies, it just slowly gets absorbed by languages with good taste.

Tuesday, September 05, 2006

Java applications, messaging and exceptions

On and off I have looked at messaging for Java applications as an alternative, more loosely coupled design pattern. Particularly for smaller applications messaging has some interesting solutions in lieu of other approaches.

Take the event listener model for AWT/Swing. Each component is responsible for maintaining its own listener event list, and each listener is responsible for finding the component it wants to get events from. This is, of course, the Observer pattern.

I want more decoupling. Message busses decouple this pattern by managing listeners and delivering events—I grew up calling this publish/subscribe—mixing in the Mediator pattern models the bus.

Channels and Subscribers

I'm going to call the bus a channel. There are many common vocabulary choices for the kind of simple message bus I have in mind, and channel is the one I used most with Gregory Hohpe who introduced me to this design. Start with the channel interface:

public interface Channel {
    void subscribe(Object subscriber);
    void unsubscribe(Object subscriber);
    void publish(Object message) throws Exception;
}

Choices, choices. Most messaging interfaces I see use a topic text to distinguish messages when publishing. But like the AWT/Swing event system, I prefer objects so that I may use inheritance to narrow or widen the scope of events I receive. Subscribers receive any intersection or union of message types they want using simple method overloading. For example:

interface BobTheBuilder {
    void onMessage(CanHeFixItRequest message);
    void onMessage(YesHeCanDispute message);
}

class MessageLogger {
    public void onMessage(Object message) {
        SomeLogger.logMessage(message);
    }
}

MessageLogger subscribes to all possible message types planning to ambitiously log everything. BobTheBuilder is only interested in requests to fix something or in criticisms of his abilities.

Supporting the union of disjoint message types explains why subscribe(Object):void and unsubscribe(Object):void take object arguments rather than using a Subscriber interface. Sketch out what Subscriber might look like with typed messages:

interface Subscriber<T> {
    void onMessage(T message);
}

This works great is a subscriber wants a single type of message, but Java does not support classes implementing the same interface multiple times—generics does not produce different interfaces:

interface IllegalJavaSubscriber {
        implements Subscriber<String>, Subscriber<Integer> {
    // Will not compile.
}

And the lack of typed interfaces for subscribers implies that the signature for onMessage(Object):void cannot narrow either. What to do? The only option is reflection. So I reflect for "onMessage" methods with the correct signature and handle overriding and overloading in the bus.

All in all I get a lot of flexibility for very little work by subscribers using the bus.

Exceptions

But what of exceptions? I realized, given the inheritance of message types, that I could simply publish exceptions as messages, same as any other type. Exception handlers simply subscribe to messages:

class ExceptionLogger {
    public void onMessage(final Exception e) {
        SomeLogger.logException(e);
    }
}

This is akin to how AOP works but without the extra syntax or new language to learn. The try/catch normally in each subscriber for logging exceptions is gathered together into the message bus with pseudo-code akin to:

for (final Subscriber subscriber
        : getSubscribersForMessage(message))
    try {
        invokeOnMessage(subscriber, message);
    } catch (final Exception e) {
        this.publish(e); // I am a channel
    }

This simplicity is very appealing, but it leaves out an interesting possibility: that subscribers could try redeliving a message to just the failed cases. Adding one more reflected method adds this option:

class SystemMonitor {
    public void onMessageException(Exception e,
        Object subscriber, Object message) {
        fixSystem(); // magic
        retryMessage(subscriber, message); // more magic
    }
}

In principal a subscriber could republish using the signature after fixing some broken part of the system. It would also make logging more informative.

Active Messages

Using pure reflection opens up another design choice, active messages. Simply having messages themselves provide an onMessage method and subscribe to the channel. Messages can publish themselves. As clever as this seems, it does leave me in fear of maintaining someone else's system years hence in which the entire thing is a ball of spaghetti with messages and subscribers calling hither, thither and yon.

Debugging is a problem with systems like this. YMMV.

Code

Some simple code illustrating this post.