Thursday, November 13, 2014

Aiming for the right level

Vinod Kumaar Ramakrishnan writes It is just a road not a roadmap making a strong point visually: software needs a map, not a road. Roadmap

This is important to understand at any level of an organization. The problem comes as you perform larger roles.

You can track only so much detail—minds have a capacity. Using a Road rather than a Map overcomes this for viewing lower down the organization. Pull back your view and substitutes rough pictures for details. Then build a bigger map, less granular, covering a larger area. It's still a map, but a map of a country rather than a region or a place.

Managing calls for finding the right level of detail, be it the application or team or project or programme or department. But remember to keep rebuilding your map and explore some.

Sunday, August 31, 2014

Overcoming Java 8 streams

Java 8 streams provide functional and LINQ-like features in a fluent API. But streams are not without drawbacks:

  • Referenced methods and lambdas cannot throw checked exceptions
  • Controlling the threads used, especially for parallel streams, is awkward
  • Streams are not designed for extension

Overcoming these drawbacks requires a "look-a-like" API. For example, implementing java.util.stream.Stream does not help: none of the existing methods throw checked exceptions, and none of the existing stream factory helpers would return your implementation with new methods.

So I wrote my own, copying the existing stream API, updating the methods to throw checked exceptions:

hm.binkley.util.stream.CheckedStream ('develop' branch for now)

From the javadoc:

CheckedStream is a throwing Stream look-a-like with control over thread pool. It cannot be a Stream as it takes throwing versions of suppliers, functions and consumers. Otherwise it is a faithful reproduction.

Write this:

   long beanCount() throws SomeException, OtherException {
       checked(Stream.of(1, 2, 3)).
           map(this::someThrowingFunction).
           peek(That::oldBean).
           count();
   }

not this:

   long beanCount() throws SomeException, OtherException {
       try {
           Stream.of(1, 2, 3).
               map(i -> {
                   try {
                       someThrowingFunction(i);
                   } catch (final SomeException e) {
                       throw new RuntimeException(e);
                   }
               }).
               peek(i -> {
                   try {
                       That.oldBean(i);
                   } catch (final OtherException e) {
                       throw new RuntimeException(e);
                   }
               }).
               count();
       } catch (final RuntimeException e) {
           final Throwable x = e.getCause();
           if (x instanceof SomeException)
               throw (SomeException) x;
           if (x instanceof OtherException)
               throw (OtherException) x;
           throw e;
       }
   }

"Intentional" exceptions (checked exceptions plus CancellationException) have "scrubbed" stacktraces: frames from framework/glue packages are removed before the intentional exception is rethrown to calling code. Scrubbed stacktraces are much easier to understand, the framework and glue code having been removed.

To see the unscrubbed stacktrace, set the system property "hm.binkley.util.stream.CheckedStream.debug" to "true".

Controlling the thread pool used by Stream is a challenge. Deep in the implementation, it checks if being run in a ForkJoinTask, and uses that thread if so, otherwise using the common pool. So with CheckedStream write this:

       checked(stream, new ForkJoinPool()).
           map(currentThread()).
           forEach(System.out::println);

not this:

       try {
           new ForkJoinPool().submit(() -> stream.
                   map(currentThread()).
                   forEach(System.out::println)).
               get();
       } catch (final ExecutionException e) {
           final Throwable x = e.getCause();
           if (x instanceof Error)
               throw (Error) x;
           if (x instanceof RuntimeException)
               // Much tricker when stream functions throw runtime
               throw (RuntimeException) x;
           throw new Error(e); // We have no checked exceptions in this example
       }

Care is taken to respect lazy and terminal operations in using thread pools. Changing thread pool or thread mode mid-stream is supported, and are "immediate" operations: they terminate the existing stream, and start a new one with the changes:

stream.sequential().
    filter(this::someFilter).
    parallel(threads). // Existing lazy operations terminated
    map(this:someMapper).
    forEach(System.out::println);

Immediate operations ensure stream methods are run in the correct threading context.

I hope you'll agree: CheckedStream is nicer to use, especially with existing code using checked exceptions.

Suggestions, bug fixes, improvements welcome!

Tuesday, July 15, 2014

Java 8 magic exception copying

Since I can in Java 8 now parameterize constructors as functions, I can write a generic exception copier:

<E extends Throwable>
E copy(final Throwable from, final Function<String, E> ctor) {
    final E to = ctor.apply(from.getMessage());
    to.setStackTrace(from.getStackTrace());
    for (final Throwable s : from.getSuppressed())
        to.addSuppressed(s);
    return to;
}

Example:

try {
    // Something throws
} catch (final AnException | BeException | KatException e) {
    throw copy(e, IOException::new);
}

This is not a common strategy but one I sometimes use to reduce the incredible size of layered exceptions, especially for logging. It is also handy for shoehorning 3rd-party exceptions into a common framework exception, a nice feature for APIs to simplify calling code. Copy helps reduce boilerplate code.

Thursday, July 10, 2014

Lambda annotations

I am still startled by Java. While playing with Byte Buddy I noodled with adding annotations to a class at runtime. I wrote this:

final Class<? extends X> dynamicType
            = (Class<< extends X>) new ByteBuddy().
        subclass(Object.class).
        implement(X.class).
        annotateType(() -> Inject.class).
        method(named("toString")).intercept(value("Hello World!")).
        method(named("foo")).intercept(value(3)).
        make().
        load(ByteBuddyMain.class.getClassLoader(), WRAPPER).
        getLoaded();

Get a load of "annotatedType"! It wants an annotation instance. Originally I tried:

new Inject() {
    @Override
    public Class<Inject> annotationType() {
        return Inject.class;
    }
}

What the hey, then I thought to try the lambda expression and live dangerously. It works!

Sunday, July 06, 2014

Modern XML to Java

How many frameworks are there for converting XML to Java? Hard to count. As an experiment I tried my hand at one. I have two top-level classes plus an annotation:

public final class XMLFuzzy
        implements InvocationHandler {
    private static final XPath xpath = XPathFactory.newInstance().
            newXPath();
    private static final Map<Method, XPathExpression> expressions
     = new ConcurrentHashMap<>();

    private final Node node;
    private final Converter converter;

    public static final class Factory {
        private final Converter converter;

        public Factory(final Converter converter) {
            this.converter = converter;
        }

        public <T> T of(@Nonnull final Class<T> itf,
                @Nonnull final Node node) {
            return XMLFuzzy.of(itf, node, converter);
        }
    }

    public static <T> T of(@Nonnull final Class<T> itf,
     @Nonnull final Node node,
            @Nonnull final Converter converter) {
        return itf.cast(newProxyInstance(itf.getClassLoader(),
                new Class[]{itf},
                new XMLFuzzy(node, converter)));
    }

    private XMLFuzzy(final Node node, final Converter converter) {
        this.node = node;
        this.converter = converter;
    }

    @Override
    public Object invoke(final Object proxy, final Method method,
         final Object[] args)
            throws Throwable {
        return converter.convert(method.getReturnType(), expressions.
                computeIfAbsent(method, XMLFuzzy::compile).
                evaluate(node));
    }

    private static XPathExpression compile(@Nonnull final Method method) {
        final String expression = asList(method.getAnnotations()).stream().
                filter(From.class::isInstance).
                map(From.class::cast).
                findFirst().
                orElseThrow(() -> new MissingAnnotation(method)).
                value();
        try {
            return xpath.compile(expression);
        } catch (final XPathExpressionException e) {
            throw new BadXPath(method, expression, e);
        }
    }

    public static final class MissingAnnotation
            extends RuntimeException {
        private MissingAnnotation(final Method method) {
            super(format("Missing @X(xpath) annotation: %s", method));
        }
    }

    public static final class BadXPath
            extends RuntimeException {
        private BadXPath(final Method method, final String expression,
                final XPathExpressionException e) {
            super(format("Bad @X(xpath) annotation on '%s': %s: %s",
                    method, expression, e.getMessage()));
            setStackTrace(e.getStackTrace());
        }
    }
}

I have left out Converter; it turns strings into objects of a given type, another example of overimplemented framework code in Java. And the annotation:

@Documented
@Inherited
@Retention(RUNTIME)
@Target(METHOD)
public @interface From {
    String value();
}

The idea is straight-forward: drive the object mapping from XML with XPaths. Credit to XMLBeam for introducing to me the elegant use of JDK proxies for this purpose.

Of course tests:

public final class XMLFuzzyTest {
    private Top top;

    @Before
    public void setUp()
            throws ParserConfigurationException, IOException, SAXException {
        final Document document = DocumentBuilderFactory.newInstance().
                newDocumentBuilder().
                parse(new InputSource(new StringReader(XML)));
        top = new XMLFuzzy.Factory(new Converter()).of(Top.class, document);
    }

    @Test
    public void shouldHandleString() {
        assertThat(top.a(), is(equalTo("apple")));
    }

    @Test
    public void shouldHandlePrimitiveInt() {
        assertThat(top.b(), is(equalTo(3)));
    }

    @Test
    public void shouldHandleRURI() {
        assertThat(top.c(), is(equalTo(URI.create("http://some/where"))));
    }

    @Test(expected = MissingAnnotation.class)
    public void shouldThrowOnMissingAnnotation() {
        top.d();
    }

    @Test(expected = BadXPath.class)
    public void shouldThrowOnBadXPath() {
        top.e();
    }

    public interface Top {
        // For the purposes of this blog post, pretend Java supports
 // multiline string literals
        @Language("XML")
        String XML = "<top>
                    <a>apple</a>
      <b>3</b>
      <c>http://some/where</c>
         </top>";

        @From("//top/a")
        String a();

        @From("//top/b")
        int b();

        @From("//top/c")
        URI c();

        void d();

        @From("dis' ain't xpath")
        void e();
    }
}

Sunday, June 29, 2014

Java 8 predicate for tee

A "tee" method did not make it into Java 8 streams, but you can make your own easily enough:

public final class Tee<T>
        implements Predicate<T> {
    private final Predicate<T> test;
    private final Consumer<T> reject;

    @Nonnull
    public static <T> Predicate<T> tee(
            @Nonnull final Predicate<T> test,
            @Nonnull final Consumer<T> reject) {
        return new Tee<>(test, reject);
    }

    private Tee(final Predicate<T> test,
            final Consumer<T> reject) {
        this.test = test;
        this.reject = reject;
    }

    @Override
    public boolean test(final T t) {
        if (test.test(t))
            return true;
        reject.accept(t);
        return false;
    }

    public static void main(final String... args) {
        asList("a", "bb", "ccc", "dddd").stream().
                filter(tee(Tee::even, err::println)).
                forEach(out::println);
    }

    private static boolean even(final String s) {
        return 0 == s.length() % 2;
    }
}

Monday, May 26, 2014

Help for Java parameterized tests

JUnit has many great features. One I especially like is parameterized tests.

The problem

One itch for me however is setting up the parameters and getting failed tests to name themselves.

Modern JUnit has help on the second itch with the name parameter to @Parameters. You still need to get the test name passed into the constructor.

But what about setting up the test data?

I write clunky code like this:

@Parameters(name = "{index}: {0}")
public static Collection<Object[]> data() {
    return asList(
            new Object[]{"Some test", 1, "a string"},
            new Object[]{"Another test", 2, "another string"}
            /* and so on for each test case */);
}

This is not the worst code, but with more complex inputs or data it grows into a long list of eye-glazing use cases. For me this is hard to maintain and document.

The solution

As an alternative I scouted around and settled on the venerable Windows INI file format. It has several virtues in this context:

  • The format is simple and well known
  • Section titles can be use to name tests
  • Comments are supported

Looking at libraries for INI support in Java, one stands out: ini4j. In particular:

  • Is stable and acceptably documented
  • Supports C-style escape sequences, e.g. \t for TAB—import for my particular tests
  • Accepts reasonable ways to input the INI (URL, input stream, et al)

Integrating this into @Parameters is straight-forward. One approach is this:

@Nonnull
public static List<Object[]> parametersFrom(
        @Nonnull final Ini ini, final Key... keys) {
    final List<Object[]> parameters = new ArrayList<>();
    for (final Section section : ini.values()) {
        final Object[] array = new Object[1 + keys.length];
        array[0] = section.getName();
        for (int i = 0; i < keys.length; i++) {
            final Key key = keys[i];
            final String value = section.fetch(key.name);
            array[1 + i] = null == value ? null : key.get.apply(value);
        }
        parameters.add(array);
    }
    return parameters;
}

public static final class Key {
    public final String name;
    public final Function<String, ?> get;

    @Nonnull
    public static Key of(@Nonnull final String name, @Nonnull final Function<String, ?> get) {
        return new Key(name, get);
    }

    @Nonnull
    public static Key of(@Nonnull final String name) {
        return of(name, Function.identity());
    }

    private Key(final String name, final Function<String, ?> get) {
        this.name = name;
        this.get = get;
     }
}

A key might look like Key.of("amount", BigDecimal::new)

Example

I simplified tests of a money value object to a test name, a value to parse and a currency and amount to expect:

@RunWith(Parameterized.class)
public class MoneyTest {
    // Failed test messages look like:
    // 0: Missing currency: 1
    @Parameters(name = "{index}: {0}: {1}")
    public static Collection<Object[]> parameters()
            throws IOException {
        return parametersFrom(
                new Ini(MoneyTest.class.getResource("MoneyTest.ini")),
                Key.of("value"),
                Key.of("currency", Currency::getInstance),
                Key.of("amount", BigDecimal::new));
    }

    @Rule
    public final ExpectedException thrown = ExpectedException.none();

    private final String description;
    private final String value;
    private final Currency currency;
    private final BigDecimal amount;

    public MoneyTest(@Nonnull final String description,
            @Nonnull final String value,
            @Nullable final Currency currency,
            @Nullable final BigDecimal amount) {
        this.description = description;
        this.value = value;
        this.currency = currency;
        this.amount = amount;

        if (!((null == currency && null == amount)
                || (null != currency && null != amount)))
            throw new IllegalArgumentException(
                    format("%s: currency and amount must both be null or non-null",
                            description));
    }

    @Test
    public void shouldParse() {
        if (null == currency) {
            thrown.expect(MoneyFormatException.class);
            thrown.expectMessage(value);
        }

        final Money money = Money.parse(value);

        assertThat(money.getCurrency(), is(equalTo(currency)));
        assertThat(money.getAmount(), is(equalTo(amount)));
    }
}
; Format - INI file
; Section title is description of test, used in reporting failure
; value is the input to `Money.parse(String)`
; IFF parsing should fail:
;   - Do not provide currency/amount
; IFF parsing should pass:
;   - currency is input to `Currency.getInstance(String)`
;   - amount is input is `new BigDecimal(String)`

[Missing currency]
value = 1

[Single dollar, no whitespace]
value = USD1
currency = USD
amount = 1.00

I like being able to add new tests by text editing of the INI file.

Thursday, March 06, 2014

World's smallest DAO

The world's smallest Java DAO (with heavy lifting provided by Spring Framework):

public class SimpleDAO {
    private final DataSourceTransactionManager transactionManager;

    public SimpleDAO(DataSourceTransactionManager transactionManager) {
        this.transactionManager = transactionManager;
    }

    public <T> T dao(Dao<T> dao) {
        return dao.using(transactionManager);
    }

    public interface Dao<T> {
        default T using(DataSourceTransactionManager transactionManager) {
            return new TransactionTemplate(transactionManager).execute(
                    status -> on(new JdbcTemplate(transactionManager.getDataSource()), status));
        }

        T on(JdbcTemplate jdbcTemplate, TransactionStatus status);
    }
}

Usage:

class InviteesDAO {
    private final SimpleDAO transact;

    InviteesDAO(DatabaseTransactionManager transactionManager) {
        transact = new SimpleDAO(tranactionManager);
    }

    List<String> getInvitees() {
        return transact.dao((jdbcTemplate, status) -> jdbcTemplate.queryForList(
            "SELECT username FROM invitees", String.class));
    }

    void invite(String username) {
        transact.dao((jdbcTemplate, status) -> jdbcTemplate.update(
            "INSERT INTO invitees (username) VALUES (?)", username));
    }
}

UPDATE: Demonstrate with composition rather than inheritance.

Saturday, February 08, 2014

ServiceBinder

I've released ServiceBinder 0.2 to Maven Central (details below; soon to be 0.3 with more documentation). I wrote this to fix a common problem for my projects.

I like using the JDK ServiceLoader for discovery: it is simple to use and understand and always available. And Kawaguchi's META-INF/services generator makes use as simple as an annotation.

However, ServiceLoader is missing one key feature for me. It requires a default constructor for implementations, and I am a fan of constructor injection.

ServiceBinder fills this gap. It discovers service classes as ServiceLoader does, and injects them with Guice or Spring. See the GitHub site for examples and source. A taste:

Guice

public final class SampleModule
        extends AbstractModule {
    @Override
    protected void configure() {
        ServiceBinder.with(binder()).bind(Bob.class);
    }
}

Spring Framework

AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();
context.register(AppConfig.class);
ServiceBinder.with(context).bind(Bob.class);
context.refresh();

Saturday, February 01, 2014

The Java fake return trick

Sometimes we have a method that does nothing but throw an exception (after munging arguments a bit). For example:

static void fail(Bob bob, Fred fred, Exception cause) {
    throw new SadError(format(
            "Sorry, but {} and {} did not meet today",
            bob, fred), cause);
}

Great when used as:

void friends(Bob bob, Fred fred) {
    try {
        meetAgain(bob, fred);
    } catch (MissedTrainException e) {
        fail(bob, fred, e);
    }
}

But what about this?

PhoneNumber exchange(Bob bob, Fred fred {
    try {
        return beamBusinessCards(bob, fred);
    } catch (LostPhoneException e) {
        fail(bob, fred, e);
        return null; // Never reached
    }
}

There's a neat generics trick to help:

static <R> R fail(Bob bob, Fred fred, Exception cause) {
    // Code unchanged - return ignored
}

Now we can write:

PhoneNumber exchange(Bob bob, Fred fred {
    try {
        return beamBusinessCards(bob, fred);
    } catch (LostPhone e) {
        return fail(bob, fred, e);
    }
}

The main downside is readability for the casual reviewer. Unless expecting this technique, he may think the new catch body returns a value. It may help to limit use to Exception, forcing handling in the caller method, but not with RuntimeException.

JDK8: Improved Iterator

One of my favorite changes in Java 8, default methods on interfaces, adds this gem to venerable Iterator:

    default void remove() {
        throw new UnsupportedOperationException("remove");
    }

By default now when you implement an iterable you need not supply "remove()". Beautiful.

There is also a new, defaulted "forEachRemaining(Consumer)".