Wednesday, December 29, 2004

Using apt

While exploring the new JDK 5 tool apt (annotation processing tool), I figured out how to write new source code that is compiled into my build tree along side my regular Java sources. Here is a trivial example.

First, I need an annotation processing factory (I ignore imports and such throughout):

public class MyAnnotationProcessorFactory
        implements AnnotationProcessorFactory {
    public Collection supportedOptions() {
        return Collections.emptySet();
    }

    public Collection supportedAnnotationTypes() {
        return Collections.singleton(
                getClass().getPackage().getName() + ".*");
    }

    public AnnotationProcessor getProcessorFor(
            final Set atds,
            final AnnotationProcessorEnvironment env) {
        return new MyAnnotationProcessor(atds, env);
    }
}

Second is to have an annotation processor:

public class MyAnnotationProcessor
        implements AnnotationProcessor {
    private final Set atds;
    private final AnnotationProcessorEnvironment env;

    MyAnnotationProcessor(final Set atds,
            final AnnotationProcessorEnvironment env) {
        this.atds = atds;
        this.env = env;
    }

    public void process() {
        for (final AnnotationTypeDeclaration atd : atds) {
            for (final Declaration decl
                    : env.getDeclarationsAnnotatedWith(atd)) {
                final String typeName = decl.getSimpleName() + "Example";
                final String fullTypeName = getPackageName() + "." + typeName;

                try {
                    final PrintWriter writer
                            = env.getFiler().createSourceFile(fullTypeName);

                    writer.println("package " + getPackageName() + ";");
                    writer.println("public class " + typeName + " {");
                    writer.println("}");

                } catch (final IOException e) {
                    throw new RuntimeException(fullTypeName, e);
                }
            }
        }
    }

    private String getPackageName() {
        return getClass().getPackage().getName();
    }
}

Last is to tell apt how to fit it all together (I'm using a Maven-style layout):

apt -cp target/classes
    -s target/gen-java -d target/gen-classes
    -target 1.5 -factorypath target/classes
    -factory MyAnnotationProcessorFactory
    src/java/AnnotatedExample

When I run the apt command, given suitable annotations in AnnotatedExample, it pulls them out, instantiates my annotation processor via my factory, and hands them to process() therein. The key is to use com.sun.mirror.apt.Filer, a class in $JAVA_HOME/lib/tools.jar. There are no online javadocs that I have found yet. Here is what the JDK 1.5.0_01 sources say about the Filer interface:

This interface supports the creation of new files by an annotation processor. Files created in this way will be known to the annotation processing tool implementing this interface, better enabling the tool to manage them. Four kinds of files are distinguished: source files, class files, other text files, and other binary files. The latter two are collectively referred to as auxiliary files.

There are two distinguished locations (subtrees within the file system) where newly created files are placed: one for new source files, and one for new class files. (These might be specified on a tool's command line, for example, using flags such as -s and -d.) Auxiliary files may be created in either location.

During each run of an annotation processing tool, a file with a given pathname may be created only once. If that file already exists before the first attempt to create it, the old contents will be deleted. Any subsequent attempt to create the same file during a run will fail.

My next step is to glue velocity into my processor so I can use templates for writing the new Java sources.

Tuesday, December 28, 2004

The future in futures

Brian McCallister has started a great set of posts on futures. This fits in perfectly with my work with Gregor Hohpe on Mercury, a light-weight messaging library for Java. Brian explains futures:

Futures are really nice conceptually, and provide for much more natural and easy to use concurrent design than Java style threading and monitors. It relates to a lot of functional programming concepts, but the key idea is that a future represents an evaluation which has not yet occured. Maybe it is being lazily evaluated, maybe it is being evaluated concurrently in another thread.

Java has no futures, but I do fake closures using reflection and a sort of distended proxy. The core of Mercury is that when code publishes a message to a channel, it is not immediately delivered. Instead, the channel records an activation record (binding) and begins reordering the records before activating the bindings. This lets Mercury have breadth-first message delivery instead of depth-first, and circumvents the effects of using a call stack in a messaging system.

Gregor's excellent Enemy of the State post describes the thinking in more detail along with some ramifications. I'm just the humble mechanic who coded the library. :-)

UPDATE: I just got an anonymous comment pointing me to java.util.concurrent.Future. Zowie! That's a cool thing to find.

Sunday, December 26, 2004

Using JDK 5 varags for testing

One JDK 5 feature I have started using to clean up my testing is varargs, the ... (or for the entity-aware). It is particularly elegant to turn this:

public void testWidgetProcessor() throws Exception {
    final Widget expected1 = createWidget(), expected2 = createWidget();

    processor.swallow(new Widget[]{
        expected1,
        expected2
    });

    assertWidgetsGained(new Widget[] {
        expected1,
        expected2
    });
}

Into this:

public void testWidgetProcessor() throws Exception {
    final Widget expected1 = createWidget(), expected2 = createWidget();

    processor.swallow(expected1, expected2);

    assertWidgetsGained(expected1, expected2);
}

And even better:

public void testWidgetProcessorHandlesTrivialCase() throws Exception {
    assertWidgetsGained(); // nothing swallowed, nothing gained
}

XOM Design Principles

A very, very interesting article, XOM Design Principles, on Java library design. What took me particularly off-guard was the preference for classes over interfaces. The argument over this point presents much to digest.

Sunday, December 19, 2004

What are Java annotations?

This must be described elsewhere, but a quick Googling didn't give it to me. A little experimentation reveals to me that JDK 5.0 annotations are dynamic proxies under the hood.

To find this out, I made an annotation named @annotate and exammined annotation.class; it revealed itself to be an interface. I then decorated a method with @annotate, got the Method with reflection, pulled the appropriate annotation class off with getAnnotation(annotate.class).getClass() and examined that: dynamic proxy, $Proxy3(java.lang.reflect.InvocationHandler).

I wonder how I can use this knowlege for some real Java-fu.

UPDATE: An even stronger answer: Proxy.isProxyClass(Class) returns true on the method annotation class, and the proxy invocation handler is a sun.reflect.annotation.AnnotationInvocationHandler. Good thing Sun provides source for the non-public bits.

Saturday, December 18, 2004

A taste of things to come

Now to make it work!

/**
 * Marks a method with a <em>pre-condition</em>.  The required annotation
 * {@link #value()} is a <code>boolean</code> condition.  The optional
 * {@link #message()} is used in {@link DBCException} if the condition
 * fails.
 */
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface pre {
    boolean value();

    String message() default "";
}

/**
 * Marks a method with a <em>post-condition</em>.  The required annotation
 * {@link #value()} is a <code>boolean</code> condition.  The optional
 * {@link #message()} is used in {@link DBCException} if the condition
 * fails.
 */
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface post {
    boolean value();

    String message() default "";
}

/**
 * Marks a method with an <em>invariant</em>.  The required annotation
 * {@link #value()} is a <code>boolean</code> invariant.  The optional
 * {@link #message()} is used in {@link DBCException} if the invariant
 * fails.
 */
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface inv {
    boolean value();

    String message() default "";
}

UPDATE: I'll have to think about this longer. Turns out that annotations only accept constants or expressions which evaluate to constants. A constant pre-condition isn't very interesting.

Friday, December 17, 2004

Looking up the location of a class file

IBM's JAR Class Finder reminds me of a snippet of utility code I wrote for PCGen which used a clever trick to find the location on disk of the definition for a given class, either a jar or a directory in the classpath:

new File(clazz.getProtectionDomain().getCodeSource().getLocation().toURI().normalize())

The only drawback: for the JDK itself the protection domain has a null code source. Since Class.getProtectionDomain() relies internally on a native method to create the protection domain, I'm unsure if this is a bug or intended behavior. It certainly isn't documented in the javadocs. I consider this a bug, but others disagree and it has changed between JDK versions.

Thursday, December 16, 2004

Clever IDEA

I just noticed very clever behavior by IntelliJ IDEA 4.5.3 on Windows. I ran a junit test that crashed the test JVM from badly behaved JNI code. Unfortunately, this caused the JVM for IDEA itself to eventually exhaust memory and crash. IDEA handled this well, informed me of the problem, and gracefully shutdown.

The cool part: when I launched it again, it ran the console window for the LAX launch wrapper. Normally this does not appear so that you just have the main application window, but the console emits all sorts of interesting information useful to someone developing IDEA. Presumably, had the crash I mentioned been IDEA's fault, I could use that information to help IntelliJ fix the bug.

Further, I closed IDEA normally and relaunched it. This time it come up normally with no extra console window. The console window feature is only triggered by a crash in IDEA. Nifty!

Wednesday, December 15, 2004

Messaging with annotations

I'm working on a talk for SD West 2005 with a former coworker, Gregor Hohpe of ThoughtWorks. We coded a simple Java messaging system for single VM apps. Most messaging systems are based on messages segregated by subject, usually a String field. Our system though is type-based. Messages implement (or extend) Message, and receivers provide methods receive(Message) to receive published messages.

The dispatch loop makes extensive use of reflection to find a "best matching" method. If you have:

public class FooReceiver extends Receiver {
    public void receiver(final Message message) { }

    public void receiver(final FooMessage message) { }
}

Then if a FooMessage (or a subtype) comes along, the better matching method receives the message, otherwise the more general method does.

But I have found a better way: annotations :-) Consider:

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface MessageHandler {
    Class[] value();
}

This bit of annotation magic lets me say this in my receiver class:

public class FooReceiver {
    @MessageHandler({Message.class})
    void receiveAllTypes(final Message message) { }

    @MessageHandler({FooMessage.class})
    void receiveFooTypes(final FooMessage message) { }
}

Now if a FooMessage comes along, both methods see the message: logically, there is no method overloading to consider since they are differently-named methods. But this has several other advantages:

  • No need to extend or implement anything; just annotate the methods
  • Methods can have better names than just receive; it's the annotation, not the name, which counts
  • Interestingly, you can write methods to handle disjoint types:
public class DisjointReceiver {
    @MessageHandler({FooMessage.class, BarMessage.class})
    void handleEitherFooOrBarType(final Message message) { }
}

Nifty — there is no need for FooMessage or BarMessage to be related types. In fact, to carry the disjunction one step further, I could dispense entirely with the Message class! Hmmm...

UPDATE: I fixed the bad links. That's what I get for blogging while woozy with flu.

Maven and test cases

I have a problem with maven I am hoping someone knows the best solution to. As is typical with Java projects using maven, I have separate src/java and src/test trees and they compiles to build/classes and build/test-classes, respectively. When I build a distribution jar, maven zips up build/classes into the jar (along with LICENSE.txt and any resources). This much I know and appreciate.

However, my project uses JUnit for testing, so I have *Test classes, one for each class under test. And my project provides *TestCase base classes for other test classes to extend. The test case classes add functionality to junit.framework.TestCase. These live under the src/test tree since they are only used for testing, and they are dependent on junit-3.8.1.jar. Of course, they compile to build/test-classes.

But since the dist goal only packages up build/classes, the test case classes do not become part of my distribution jar. And I want to package them for distribition. Oops.

For now, I've moved the test case classes from src/test to src/java so that maven will bundle them in the distribution jar, but I feel awkward doing that.

Is there some way to teach maven to pull src/test/**/*TestCase.java classes from build/test-classes into the distribution jar, but no other test classes?

Delegation, a problem and solution

I'm adding unit testing to existing code and encounter this problem:

public class Foo {
    private static Foo SINGLETON;
    private static Trouble TROUBLE;

    private Foo() { }

    public Foo newInstance() {
        if (null == SINGLETON) SINGLETON = new Foo();
        return SINGLETON;
    }

    public Trouble getTrouble() {
        if (null == TROUBLE) TROUBLE = new Trouble();
        return TROUBLE;
    }

    public void doSomething() {
        getTrouble().doSomethingElse();
    }
}

Callers are expected to write Foo.newInstance().doSomething(). Fair enough. For unit testing, I want to replace SINGLETON with a stub or mock object that extends Foo in setUp() and put it back to null in tearDown(). Foo is not under test, but classes I am testing call to it and I need to control the interaction.

What about TROUBLE? It is a complex object with its own behavior, so I need to stub or mock it as well. But here's the rub: I don't have control over its source. It could be a SWIG-generated wrapper for JNI, or from a third-party jar I lack the sources to. And the class looks something like this:

public final class Trouble {
    public void doSomethingElse() { }
}

Oops! No extending for me. what to do? It is time to rely on delegation.

First, extract an interface from Trouble, say Troubling, which contains all the public methods in Trouble. We cannot change Trouble to implement Troubling, but I'll overcome that in a moment.

Second, update Foo and all callers of Foo.getTrouble to refer to Troubling and not Trouble. This decouples them from dependency on Trouble.

Third, create a new class which implements Troubling and delegates to Trouble:

public class NoTrouble implements Troubling {
    private final Trouble trouble;

    public NoTrouble(final Trouble trouble) {
        this.trouble = trouble;
    }

    public void doSomethingElse() {
        trouble.doSomethingElse();
    }
}

Lastly, update Foo to use NoTrouble instead of Trouble:

public class Foo {
    private static Foo SINGLETON;
    private static Troubling TROUBLE;

    private Foo() { }

    public Foo newInstance() {
        if (null == SINGLETON) SINGLETON = new Foo();
        return SINGLETON;
    }

    public Troubling getTrouble() {
        if (null == TROUBLE) TROUBLE = new NoTrouble(new Trouble());
        return TROUBLE;
    }

    public void doSomething() {
        getTrouble().doSomethingElse();
    }
}

We're following the ancient dictum, any problem can be solved by introducing an extra level of indirection. Foo was already delegating doSomething() to Trouble; we just replaced that relationship with an extra level of delegation, Foo to NoTrouble to Trouble. Now I can mock or stub NoTrouble without needing access to Trouble.

Go have a peanut butter sandwich.

Saturday, December 11, 2004

The easy way

While updating Java code to permit replacement of one implementation of a system-wide object with another (in this case, we have a physical device which is needed for production code, but which is a problem to require for unit tests — I want to use a mock or stub implementation for testing to avoid accessing a real device)

I started down the inversion of control path and faced a daunting problem: large portions of the system needed updating to support appropriate constructors, new class members to hold the IoC objects, rewritten callers to cope with the new constructors, ad nauseum. Yikes!

What to do?

After some tinkering, I decided to give up on the "pure" solution and hack something just for the immediate problem at hand. Rather than adjust code all through out the system (and try to adjust the culture around me to accept those changes "just for testing"), I limited the scope of change to just those places which actually need a device object.

First I implemented the singleton pattern for the device class, then refactored all the users to call Device.newInstance() instead of new Device().

Next, I made sure the singleton implementation used lazy initialization:

private static Device SINGLETON;

public static Device newInstance() {
    if (null == SINGLETON)
        SINGLETON = new Device();

    return SINGLETON;
}

There's the crux: now testing code can put a stub or mock implementation into the SINGLETON field with reflection in setUp() and replace it with null in tearDown(). This ensures that tests see only a stub or mock implementation anywhere in the production code which uses a device object. And no code need know any details about who uses a device object or how.

Finally, tearDown sees to it that other tests get a real device implementation if they need one since it sets the singleton back to null (see the code above).

The only other update to the device object is to make the constructor protected so that a stub or mock implementation can extend the class.

Thursday, December 09, 2004

What is remove in a map?

While coding a utility map class I was struck by this problem: what is the meaning of remove? That is, what does does it mean to remove a key when the map has more than one value for that key? How could that happen, you ask. My map isn't the ordinary sort: it's a union map, or a view of a list of maps.

Picture a stack of maps like a column of egg cartons over which you stand. As you look down through a particular cell from above, some cartons have an egg there, some do not. My union map is as if you collapsed the stack of egg cartons. Into each cell goes some kind of combination of the eggs from the corresponding cells in the original stack. The usual rule selects the top-most egg; remove just the top-most egg, and the rule selects the next egg down in the stack of cells; if there are no eggs in any cell, the collapsed cell is also lacking.

So I am faced with the question, what does it mean to remove a key from the union map? Do I remove all eggs in the matching cell from all cartons, or do I just remove the top-most one and the cell has a new value? Either way seems to me not quite right. As a practical matter, I chose for remove to remove all values for the key in all stacked maps so that collectively they would still follow the requirements of remove in the JDK, and I provided a removeOne to remove just the top-most value. Linguistically, I'd prefer for remove to perform this function, and for a removeAll to remove the key-value pairs from all stacked maps, but that would break expectations.

Suggestions?

Friday, December 03, 2004

Reinventing Smalltalk

What a cool idea!

But ah-hah! We do, we can use a different implementation of FunctionTable which stores the method body as a String in Derby and uses BeanShell (or Janino!) to invoke it. So a method invocation is completely configurable at runtime! Not deploy time, runtime. If you find a bug, you can fix it without a redeploy.
Yes, Virginia, there is runtime update of objects in memory. What these two have come up with is the redefinition of object methods in a running system, no restart required. There's infrastructure to build to make it happen, but the principle is demonstrated. Java is catching up with Smalltalk bit by bit.

Thursday, December 02, 2004

Have IDEA mark all qualifying variables as final

I finally figured out a trick with IDEA that I've been searching for for more than a year now. I like to mark all qualifying variables as final when I work in Java. I used to strip final as noise, but was convinced by Chapter 2[PDF] of Hardcore Java that final is a fine thing and good coding practice.

But back to IntelliJ IDEA. Marking all qualifying variables as final is a real pain. Why not have the editor do it for me? I certainly can flag unmarked variables with warnings (Settings|Errors|Local Code Analysis|Local variable or parameter can be final).

To have the editor mark all qualifying variables globally as final, run the analysis tool on a file or your whole project with Analyze|Inspect code...|Local Code Analysis|Local variable or parameter can be final, then Run.

When the results return, the left pane displays analysis problems. Expand the Local Code Analysis node, select the Local variable or parameter can be final node underneath, and push the "lightbulb" (Apply a quickfix) button. This fixes the problem for all qualifying variables all in one go. Wonderful!