Wednesday, December 27, 2006

Making a one-jar with Maven

(This is a lengthier post than usual. You can skip forward to the finished example, if you like.)

UPDATE:The One-JAR author comments:

Nice article, thanks for using (and persisting with) One-JAR. I just released a new version (0.97), checkout http://one-jar.sourceforge.net/

The one-jar-maven plugin has been updated with this new release, and there is also a new project in the CVS repository showing how to use maven2 to build (http://one-jar.cvs.sourceforge.net/viewvc/one-jar/one-jar-maven/)

A major part of this release was making it easier to set up One-JAR projects, to which end there is an application generator (one-jar-appgen) which will create a basic One-JAR directory tree, with support for building under Eclipse/Ant, and which contains a JUnit test harness. I’d be interested in hearing your thoughts on this if you’re still working with the product.

One-JAR

I recently came across Simon Tuffs' One-JAR Java project. It uses a custom "boot" classloader to place whole JARs within JARs and fix up the classpath accordingly.

The standard classloader will not do this and requires that supporting JARs be external to each other. So if I have Main.jar with dependency log4j.jar where Main.jar holds the main entry point—hm.binkley.Main, say—I cannot bundle log4j.jar into Main.jar but must distribute it alongside separately.

This has no effect on running Java, but changes distribution as I can no longer provide a single file (Main.jar in this example) for others to run.

Contrast these two ZIP files:

  • Foo-1.0-with-one-jar.zip:
    • foo-1.0/Main.jar
    • foo-1.0/README
  • Foo-1.0-without-one-jar.zip:
    • foo-1.0/Main.jar
    • foo-1.0/log4j.jar
    • foo-1.0/README

In this example, it does not seem to make a lot of difference; but when a project pulls in several Jakarta Commons JARs, several other open-source JARs, and several in-house, proprietary JARs, it becomes very obvious that distribution is an issue. One project I work on has a java command line between 5000 and 6000 characters long because of external jar dependencies in the classpath.

A traditional solution is to repack all the classes, internal and external, in to a single über-JAR, losing the independent qualities of each JAR including interesting MANIFEST.MF entries.

One-JAR solves this elegantly by packing everything into one JAR without unpacking the dependencies. My command line becomes just:

$ java [-options] -jar Main.jar

Ant

An Ant script for One-JAR is simple. One-JAR requires that you pack your original Main.jar into the final, single JAR along with dependencies:

  • main/Main.jar
  • lib/log4j.jar

They pack in with the One-JAR classes.

One-JAR then provides a custom classloader and alternative main entry point to glue it all together and a custom URL for classloading, onejar:.

As One-JAR provides you with a prototype outer JAR, all you need do is update the prototype, adding in main/Main.jar and lib/log4j.jar with Main.jar!main/Main.jar!META-INF/MANIFEST.MF unchanged:.

Manifest-Version: 1.0
Main-Class: hm.binkley.Main
Class-Path: log4j.jar

Leave unchanged the Main.jar!META-INF/MANIFEST.MF provided in the prototype:

Manifest-Version: 1.0
Main-Class: com.simontuffs.onejar.Boot

(Note that the first Main.jar is the prototype copied from one-jar-boot-0.95.jar, provided with the One-JAR download, and the second Main.jar is your original executable JAR.)

One-JAR follows the convention that your "real" JAR is in main/ and all dependency JARs are elsewhere within your single, distributable JAR. Nothing is unpacked.

The jar task handles this ably:

<target name="one-jar" depends="jar" description="Build one ONE-JAR">
    <property name="onejardir" location="${pom.build.directory}/one-jar"/>
    <mkdir dir="${onejardir}/main"/>
    <mkdir dir="${onejardir}/lib"/>

    <copy tofile="${onejardir}/${jarfile}"
            file="lib/one-jar-boot-0.95.jar"/>
    <copy todir="${onejardir}/main"
            file="${pom.build.directory}/${jarfile}"/>
    <copy todir="${onejardir}/lib" flatten="true">
        <fileset dir="lib" includes="*.jar"
                excludes="one-jar-boot-0.95.jar"/>
        <fileset refid="runtime.dependency.fileset"/>
    </copy>

    <jar jarfile="${onejardir}/${jarfile}" update="true"
            basedir="${onejardir}" excludes="${jarfile}"/>
</target>

But what of Maven?

Maven

Where Ant asks, How do I make this cake? - Maven asks Where can I find cake ingredients? But Maven does provide a mechanism for adding new recipes, the assembly plugin.

An assembly is a description of packaging for Maven, and is usually hooked into the package phase in your build lifecycle. (The assembly plugin is much simpler than adding new packaging with Plexus, the method described in the link.)

To build a One-JAR with Maven and the assembly plugin, add a new assembly descriptor which follows the outline of building with Ant, taking care to unpack the One-JAR prototype JAR and add to it Main.jar and its dependencies.

This is better explained by example.

Example

The sources

Unfortunately, One-JAR is not in the Maven central repository, so it is included here as part of the project. That is the reason for the added repository in the POM and the lib/ files. Ignoring directories:

  • lib/com/simontuffs/one-jar/0.95/one-jar-0.95.jar
  • lib/com/simontuffs/one-jar/0.95/one-jar-0.95.pom
  • pom.xml
  • src/assembly/one-jar.xml
  • src/main/java/hm/binkley/Main.java
  • src/main/resources/log4j.properties

The POM

<project xmlns="http://maven.apache.org/POM/4.0.0"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
                             http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>hm.binkley</groupId>
    <artifactId>Main</artifactId>
    <packaging>jar</packaging>
    <version>1.0</version>
    <name>One-JAR Example</name>
    <url>http://binkley.blogspot.com/</url>

    <repositories>
        <repository>
            <id>project</id>
            <name>Project Repository</name>
            <url>file:///${basedir}/lib</url>
            <layout>default</layout>
        </repository>
    </repositories>

    <build>
        <plugins>
            <plugin>
                <artifactId>maven-jar-plugin</artifactId>
                <configuration>
                    <archive>
                        <manifest>
                            <mainClass>hm.binkley.Main</mainClass>
                            <addClasspath>true</addClasspath>
                        </manifest>
                    </archive>
                </configuration>
            </plugin>
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <configuration>
                    <archive>
                        <manifest>
                            <mainClass>com.simontuffs.onejar.Boot</mainClass>
                        </manifest>
                    </archive>
                    <descriptors>
                        <descriptor>src/assembly/one-jar.xml</descriptor>
                    </descriptors>
                </configuration>
                <executions>
                    <execution>
                        <id>make-assembly</id>
                        <phase>package</phase>
                        <goals>
                            <goal>attached</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>com.simontuffs</groupId>
            <artifactId>one-jar</artifactId>
            <version>0.95</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.14</version>
        </dependency>
    </dependencies>
</project>

The assembly

In src/main/assembly/one-jar.xml:

<assembly>
    <id>one-jar</id>
    <formats>
        <format>jar</format>
    </formats>
    <includeBaseDirectory>false</includeBaseDirectory>
    <dependencySets>
        <dependencySet>
            <outputDirectory/>
            <unpack>true</unpack>
            <includes>
                <include>com.simontuffs:one-jar</include>
            </includes>
        </dependencySet>
        <dependencySet>
            <outputDirectory>main</outputDirectory>
            <includes>
                <include>${groupId}:${artifactId}</include>
            </includes>
        </dependencySet>
        <dependencySet>
            <outputDirectory>lib</outputDirectory>
            <scope>runtime</scope>
            <excludes>
                <exclude>com.simontuffs:one-jar</exclude>
                <exclude>${groupId}:${artifactId}</exclude>
            </excludes>
        </dependencySet>
    </dependencySets>
</assembly>

The finished One-JAR

  • META-INF/MANIFEST.MF
  • com/simontuffs/onejar/Boot.class
  • com/simontuffs/onejar/Boot.java
  • com/simontuffs/onejar/Handler$1.class
  • com/simontuffs/onejar/Handler.class
  • com/simontuffs/onejar/Handler.java
  • com/simontuffs/onejar/JarClassLoader$ByteCode.class
  • com/simontuffs/onejar/JarClassLoader.class
  • com/simontuffs/onejar/JarClassLoader.java
  • doc/one-jar-license.txt
  • main/Main-1.0.jar
  • lib/log4j-1.2.14.jar

Thursday, December 14, 2006

Brilliant: C to MIPS to Java bytecode

NestedVM is one of the cleverest solutions to the problem of legacy C and C++ code I have ever seen: use GCC to compile to MIPS machine instructions, then translate those directly into Java bytecode.

There is also a good slideshow.

A straight-forward example is a pure-Java translation of the SQLite JDBC driver.