Friday, June 26, 2015

Pattern matching for Java

Java does not (yet) have pattern matching although writing a legible DSL for it is not too hard. I'm certainly not the first to take a hand at it.

An example DSL in action. Lambdas and method references sure help a lot:

public static void main(final String... args) {
    asList(0, 1, 2, 3, 13, 14, null, -1).stream().
            peek(n -> out.print(format("%d -> ", n))).
            map(matching(Integer.class, Object.class).
                    when(Objects::isNull).then(n -> "!").
                    when(is(0)).then(nil()).
                    when(is(1)).then("one").
                    when(is(13)).then(() -> "unlucky").
                    when(is(14)).then(printIt()).
                    when(even()).then(scaleBy(3)).
                    when(gt(2)).then(dec()).
                    none().then("no match")).
            map(MatchingTest::toString).
            forEach(out::println);
    out.flush(); // Avoid mixing sout and serr

    matching(Integer.class, Void.class).
            none().thenThrow(RuntimeException::new).
            apply(0);
}

Implementation:

/**
 * {@code Matching} represents <a href="https://en.wikipedia.org/wiki/Pattern_matching">Pattern
 * Matching</a> in Java as a function production an optional.  Example: <pre>
 * asList(0, 1, 2, 3, 13, 14, null, -1).stream().
 *         peek(n -> out.print(format("%d -> ", n))).
 *         map(matching(Integer.class, Object.class).
 *             when(Objects::isNull).then(n -&gt; "!").
 *             when(is(0)).then(nil()).
 *             when(is(1)).then("one").
 *             when(is(13)).then(() -&gt; "unlucky").
 *             when(is(14)).then(printIt()).
 *             when(even()).then(scaleBy(3)).
 *             when(gt(2)).then(dec()).
 *             none().then("no match")).
 *         map(MatchingTest::toString).
 *         forEach(out::println);</pre>
 * <p>
 * <i>NB</i> &mdash; There is no way to distinguish from an empty optional if
 * there was no match, or if a match mapped the input to {@code null}, without
 * use of a {@link When#then(Object) sentinel value} or {@link
 * When#thenThrow(Supplier) thrown exception}.
 * <p>
 * <strong>NB</strong> &mdash; There is no formal destructuring, but this can
 * be simulated in the {@code Predicate} to {@link #when(Predicate) when}.
 *
 * @param <T> the input type to match against
 * @param <U> the output type of a matched pattern
 *
 * @author <a href="mailto:binkley@alumni.rice.edu">B. K. Oxley (binkley)</a>
 */
@NoArgsConstructor(access = PRIVATE)
public final class Matching<T, U>
        implements Function<T, Optional<U>> {
    private final Collection<Case> cases = new ArrayList<>();

    /**
     * Begins pattern matching with a new pattern matcher.
     *
     * @param inType the input type token, never {@code null}
     * @param outType the output type token, never {@code null}
     * @param <T> the input type to match against
     * @param <U> the output type of a matched pattern
     *
     * @return the pattern matcher, never {@code null}
     *
     * @todo Avoid the type tokens
     */
    public static <T, U> Matching<T, U> matching(final Class<T> inType,
            final Class<U> outType) {
        return new Matching<>();
    }

    /**
     * Begins a when/then pair.
     *
     * @param when the pattern matching test, never {@code null}
     *
     * @return the pattern continuance, never {@code null}
     */
    public When when(final Predicate<? super T> when) {
        return new When(when);
    }

    /**
     * Begins a default when/then pair, always placed <strong>last</strong> in
     * the list of cases (evaluates no cases after this one).
     *
     * @return then pattern continuance, never {@code null}
     */
    public When none() {
        return when(o -> true);
    }

    /**
     * Evaluates the pattern matching.
     *
     * @param in the input to match against, possibly {@code null}
     *
     * @return the match result (empty if no match), never {@code null}
     */
    @Override
    public Optional<U> apply(final T in) {
        return cases.stream().
                filter(c -> c.p.test(in)).
                findFirst().
                map(c -> c.q.apply(in));
    }

    @RequiredArgsConstructor(access = PRIVATE)
    public final class When {
        /**
         * Number of frames to discard when creating an exception for a match.
         * Very sensitive to implementation.  This aids in understanding stack
         * traces from matching, discarding internal machinery and leaving the
         * actual throwing call at the top of the stack.
         */
        private static final int N = 7;
        private final Predicate<? super T> when;

        /**
         * Ends a when/then pair, evaluating <var>then</var> against the input
         * if matched.
         *
         * @param then the pattern matching function, never {@code null}
         *
         * @return the pattern matcher, never {@code null}
         */
        public Matching<T, U> then(
                final Function<? super T, ? extends U> then) {
            cases.add(new Case(when, then));
            return Matching.this;
        }

        /**
         * Ends a when/then pair, returning <var>then</var> if matched.
         *
         * @param then the pattern matching value, possibly {@code null}
         *
         * @return the pattern matcher, never {@code null}
         */
        public Matching<T, U> then(final U then) {
            cases.add(new Case(when, x -> then));
            return Matching.this;
        }

        /**
         * Ends a when/then pair, evaluating <var>then</var> independent of
         * supplier if matched.
         *
         * @param then the pattern matching supplier, never {@code null}
         *
         * @return the pattern matcher, never {@code null}
         */
        public Matching<T, U> then(final Supplier<? extends U> then) {
            cases.add(new Case(when, x -> then.get()));
            return Matching.this;
        }

        /**
         * Ends a when/then pair, evaluating <var>then</var> to {@code null}
         * if matched.
         *
         * @param then the input consumer, never {@code null}
         *
         * @return the pattern matcher, never {@code null}
         */
        public Matching<T, U> then(final Consumer<? super T> then) {
            cases.add(new Case(when, o -> {
                then.accept(o);
                return null;
            }));
            return Matching.this;
        }

        /**
         * Ends a when/then pair, evaluating <var>then</var> independent of
         * supplier and throwing the new exception if matched.
         *
         * @param then the pattern matching exception supplier, never {@code
         * null}
         *
         * @return the pattern matcher, never {@code null}
         */
        public Matching<T, U> thenThrow(
                final Supplier<RuntimeException> then) {
            cases.add(new Case(when, x -> {
                final RuntimeException e = then.get();
                final List<StackTraceElement> stack = asList(
                        e.getStackTrace());
                e.setStackTrace(stack.subList(N, stack.size()).
                        toArray(new StackTraceElement[stack.size() - N]));
                throw e;
            }));
            return Matching.this;
        }
    }

    @RequiredArgsConstructor(access = PRIVATE)
    private final class Case {
        private final Predicate<? super T> p;
        private final Function<? super T, ? extends U> q;
    }
}

UPDATE: I recently found the very nice Derive4J annotation processor library which provides similar functionality in the form of Algebraic Data Types (ADTs), aka, Sum Types in functional languages.

Thursday, May 28, 2015

RESTful errors, Simple Boot

Handling Errors

Reading Travis McChesney's RESTful API Design Part III: Error Handling, there are more ways to represent errors in REST APIs than the returned body. A toy API I am writing as a playground for these idea is named Simple Boot, a jest on "Spring Boot". There I have an "X-Correlation-ID" header for tracking calls through services.

  • Should "X-Correlation-ID" be required?
  • Should it be supplied automatically?
  • When required and missing, what should the caller receive?

Answering the third the caller sees:

HTTP/1.1 400 Bad Request
Server: Apache-Coyote/1.1
Warning: 250 localhost:8080 "Missing X-Correlation-ID header"
Content-Type: application/json;charset=UTF-8
Transfer-Encoding: chunked
Date: Thu, 28 May 2015 12:46:40 GMT
Connection: close

{
  "timestamp": 1432817125588,
  "status": 400,
  "error": "Bad Request",
  "message": "Missing X-Correlation-ID header",
  "path": "/hello/Bob"
}

Here rather than choosing among options, I take them all:

  1. Return a 400 for a bad request
  2. Describe the error in the response body
  3. Add a header describing the error

As McChesney points out, there is not general agreement on reporting errors. For example, Facebook returns 200 for errors, requiring parsing of the response body.

Using the "Warning" header for this purpose is uncommon but supported in the specification albeit obliquely. However not everyone agrees. I use warning code 250, one I made up. The 1xx codes are transient errors; the 2xx codes permanent. In another example I use a 1xx code for an upstream service currently unavailable, and return cached data, which is closer to the described uses of "Warning".

Other header options:

  • Using a custom code outside 1xx or 2xx with "Warning". This makes sense for in-house services, may cause issues with caching devices but unclear
  • Use a custom HTTP header, a common solution, again may have issues with intermediate devices

Some Spring Bonus

Simple Boot has been fun and instructive. One Spring Boot feature I stumbled on is automated binding of configuration properties. For services requiring "X-Correlation-ID" I configure with @ConfigurationProperties(prefix = "headers.correlation-id.server"). To automate clients I use @ConfigurationProperties(prefix = "headers.correlation-id.client").

As an example of solving one issue with headers, I write Feign pass-through support.

Of course there are tests.

Sunday, March 15, 2015

Dynamic properties in Java

Motivation

I've been looking at the problem of dynamic properties in Java. Typically properties are injected into a program at start and never change. Common examples include timeouts, host names, etc. Changing these requires and edit and restart (edit can also mean an update to a remote data source).

What I want is runtime updates to properties, and the program uses the new values. Hence "dynamic properties".

There are several schemes for this built around frameworks or external data sources. I want something using only the JDK.

Interfaces

My interfaces became one for tracking key-value pairs, one for updating them (javadoc munged to display in post):

/**
 * Tracking tracks string key-value pairs, tracking external updates
 * to the boxed values.  Optionally tracks them as a type converted from
 * string.  Example use: 
 * Tracking dynafig = ...;
 * Optional<AtomicReference<String>> prop = dynafig.track("prop");
 * boolean propDefined = prop.isPresent();
 * AtomicReference<String> propRef = prop.get();
 * String propValue = propRef.get();
 * // External source updates key-value pair for "prop"
 * String newPropValue = propRef.get();
* * In an injection context:
 * class Wheel {
 *     private final AtomicInteger rapidity;
 *
 *     @Inject
 *     public Wheel(final Tracking dynafig) {
 *         rapidity = dynafig.track("rapidity").
 *                 orElseThrow(() -> new IllegalStateException(
 *                         "Missing 'rapidity' property));
 *     }
 *
 *     public void spin() {
 *         spinAtRate(rapidity.get());
 *     }
 * }
* * @author Brian Oxley * @see Updating Updating key-value pairs * @see Default Reference implementation */ public interface Tracking { /** * Tracks the given key value as a string. Returns empty if * key is undefined. If key is defined, may stil * return a null boxed value. * * @param key the key, never missing * * @return the optional atomic value string, never missing * * @todo Some way to work out type from convert */ @Nonnull Optional<AtomicReference<String>> track(@Nonnull final String key); /** * Tracks the given key value as a boolean. Returns empty if * key is undefined. * * @param key the key, never missing * * @return the optional atomic value boolean, never missing */ @Nonnull Optional<AtomicBoolean> trackBool(@Nonnull final String key); /** * Tracks the given key value as an integer. Returns empty if * key is undefined. * * @param key the key, never missing * * @return the optional atomic value integer, never missing */ @Nonnull Optional<AtomicInteger> trackInt(@Nonnull final String key); /** * Tracks the given key value as type. Returns * empty if key is undefined. If key is defined, * may stil return a null boxed value. * * @param key the key, never missing * @param type the value type token, never missing * @param convert the value converter, never missing * @param <T> the value type * * @return the optional atomic value reference, never missing * * @todo Some way to work out type from convert */ @Nonnull <T> Optional<AtomicReference<T>> trackAs(@Nonnull final String key, @Nonnull final Class<T> type, // TODO: Can this be worked out? @Nonnull final Function<String, T> convert); }
/**
 * {@code Updating} updates key-value pairs.
 *
 * @author Brian Oxley
 * @see Tracking Tracking key-value pairs
 * @see Default Reference implementation
 */
public interface Updating {
    /**
     * Updates a key-value pair with a new value.
     *
     * @param key the key, never missing
     * @param value the value, possibly null
     *
     * @return true if updated else false if created
     */
    boolean update(@Nonnull final String key, @Nullable final String value);

    /**
     * Updates a key-value pair as a map entry for convenience.
     *
     * @param entry the entry, never missing
     *
     * @return true if updated else false if created
     * @see #update(String, String)
     */
    default boolean update(@Nonnull final Map.Entry<String, String> entry) {
        return update(entry.getKey(), entry.getValue());
    }

    /**
     * Updates a set of key-value pairs for convenience.  Each key is
     * invidually updated in entry-set order.
     *
     * @param values the key-value set, never missing
     *
     * @see #update(String, String)
     */
    default void updateAll(@Nonnull final Map<String, String> values) {
        values.entrySet().stream().
                forEach(this::update);
    }
}

Team lead

I built a reference implementation to demonstrate these were plausible. To my team I explained like this:

  • Coded very much in Java 8 functional style
  • Trivial API - 4 tracking calls (variants on return type), 1 updating call (plus 2 convenience)
  • Minimal dependencies - JDK + javax.annotations (@Nonnull)
  • Thread-safe, not concurrency-safe (lost updates, etc). Fine if properties are not updated on top of each other (i.e., several in the same microsecond)
  • I expect it to be straight-forward to integrate into JSR107, Cassandra, etc, but I've not tried this from home

And offered some advice to my juniors:

  • Be functional where sensible, the benefits are almost beyond enumeration
  • Corollary: Avoid state - in Java that means fields. Temp variables (locals) are debatable and come down to individual taste. I lean to avoiding them, but there's nothing wrong with creating locals that clarify the code for others (including yourself 6 mos. later)
  • Corollary: Push fields down as low as possible, avoid globals and "local globals" (static fields)
  • Complicate your data structure, not your calls. People reason about complex data structures much better than they do about complex logic
  • Avoid null. Really. Treat all uses of null as code smell - think of it as "machine level" programming. You should do high-level programming

Implementation

All tests pass. The reference implementation:

public final class Default
        implements Tracking, Updating {
    private final Map<String, Value> keys = new ConcurrentHashMap<>();
    private final Map<String, Value> values = new ConcurrentHashMap<>();

    public Default() {
    }

    public Default(@Nonnull final Map<String, String> keys) {
        updateAll(keys);
    }

    @Nonnull
    @Override
    public Optional<AtomicReference<String>> track(
            @Nonnull final String key) {
        return Optional.ofNullable(keys.get(key)).
                map(Value::get);
    }

    @Nonnull
    @Override
    public Optional<AtomicBoolean> trackBool(@Nonnull final String key) {
        return Optional.ofNullable(keys.get(key)).
                map(Value::getBool);
    }

    @Nonnull
    @Override
    public Optional<AtomicInteger> trackInt(@Nonnull final String key) {
        return Optional.ofNullable(keys.get(key)).
                map(Value::getInt);
    }

    @Nonnull
    @Override
    public <T> Optional<AtomicReference<T>> trackAs(@Nonnull final String key,
            @Nonnull final Class<T> type,
            @Nonnull final Function<String, T> convert) {
        return Optional.ofNullable(keys.get(key)).
                map(v -> v.getAs(type, convert));
    }

    @Override
    public boolean update(@Nonnull final String key,
            @Nullable final String value) {
        return null != keys.put(key, values.compute(key,
                (k, v) -> null == v ? new Value(value) : v.update(value)));
    }

    private static final class Atomic<T> {
        private final T atomic;
        private final Consumer<String> update;

        private static Atomic<AtomicReference<String>> of(
                final String value) {
            final AtomicReference<String> atomic = new AtomicReference<>(
                    value);
            return new Atomic<>(atomic, atomic::set);
        }

        private static Atomic<AtomicBoolean> boolOf(final String value) {
            final AtomicBoolean atomic = new AtomicBoolean(
                    null == value ? false : Boolean.valueOf(value));
            return new Atomic<>(atomic,
                    v -> atomic.set(null == v ? false : Boolean.valueOf(v)));
        }

        private static Atomic<AtomicInteger> intOf(final String value) {
            final AtomicInteger atomic = new AtomicInteger(
                    null == value ? 0 : Integer.valueOf(value));
            return new Atomic<>(atomic,
                    v -> atomic.set(null == v ? 0 : Integer.valueOf(v)));
        }

        private static <T> Atomic<AtomicReference<T>> asOf(final String value,
                final Function<String, T> convert) {
            final AtomicReference<T> atomic = new AtomicReference<>(
                    convert.apply(value));
            return new Atomic<>(atomic, v -> atomic.set(convert.apply(v)));
        }

        private Atomic(final T atomic, final Consumer<String> update) {
            this.atomic = atomic;
            this.update = update;
        }

        private void update(final String value) {
            update.accept(value);
        }
    }

    private final class Value {
        @Nullable
        private final String value;
        private final Map<Class<?>, Atomic<?>> values;

        private Value(final String value) {
            this(value, new ConcurrentHashMap<>(3));
        }

        private Value(@Nullable final String value,
                final Map<Class<?>, Atomic<?>> values) {
            this.value = value;
            this.values = values;
        }

        private Value update(final String value) {
            final Value newValue = new Value(value, values);
            newValue.values.values().stream().
                    forEach(a -> a.update(value));
            return newValue;
        }

        @SuppressWarnings("unchecked")
        private AtomicReference<String> get() {
            return (AtomicReference<String>) values.
                    computeIfAbsent(String.class,
                            k -> Atomic.of(value)).atomic;
        }

        private AtomicBoolean getBool() {
            return (AtomicBoolean) values.
                    computeIfAbsent(Boolean.class,
                            k -> Atomic.boolOf(value)).atomic;
        }

        private AtomicInteger getInt() {
            return (AtomicInteger) values.
                    computeIfAbsent(Integer.class,
                            k -> Atomic.intOf(value)).atomic;
        }

        @SuppressWarnings("unchecked")
        private <T> AtomicReference<T> getAs(final Class<T> type,
                final Function<String, T> convert) {
            return (AtomicReference<T>) values.
                    computeIfAbsent(type,
                            k -> Atomic.asOf(value, convert)).atomic;
        }

        @Override
        public boolean equals(final Object o) {
            if (this == o)
                return true;
            if (o == null || getClass() != o.getClass())
                return false;
            final Value that = (Value) o;
            return Objects.equals(value, that.value);
        }

        @Override
        public int hashCode() {
            return Objects.hash(value);
        }
    }
}

Wednesday, March 04, 2015

Does refactoring worsen code?

Thought provoking: Study finds that refactoring doesn’t improve code quality. The opening:

Refactoring software, that is, restructuring existing source code to make it more readable, efficient, and maintainable, is something all developers do every now and again. Of course, the implicit assumption behind refactoring is that the benefits (time and headaches saved in the future) outweigh the costs (time and effort spent now). However, new experimental research suggests that this may not be the case and that software code quality may not be improved much, if at all, by refactoring.

The study was done by researchers in Sri Lanka and recently published in the International Journal of Software Engineering & Applications titled An Empirical Evaluation of Impact of Refactoring On Internal and External Measures of Code Quality. The goal was to test whether common refactoring techniques resulted in measurable improvements in software quality, both externally (e.g., Is the code more maintainable?) and internally (e.g., Number of lines of code).

The researchers selected a small-scale application (about 4,500 lines of C# code) used by the academic staff at the University of Kelaniya for scheduling events and managing online documents for evaluation. 10 common refactoring techniques were applied to the code (e.g., Replace Type Code with Subclasses, Replace Conditional with Polymorphism).

Reading the press account I expect objections such as:

C#?
Well, language should not matter here
Too small an example!
This is a fair complaint, the impact of refactoring scales with code base size
Those weren't professionals
The reviewers were a mix, but this complaint overlooks how much code is written and maintained by "non-programmers", an oxymoron as programming is an activity, not a type of person

My own objection is that the refactoring did not go far enough. Deepening a type hierarchy does often make code "worse", delegation is often a better choice than inheritance. In a larger code base than the sample used, this become more obvious.

Friday, February 27, 2015

Architects, engineers?

Delightful passage from Sam Newman's new book, Building Microservices:

Part of us wants recognition, so we borrow names from other professions that already have the recognition we as an industry crave. But this can be doubly harmful. First, it implies we know what we are doing, when we plainly don’t. I wouldn’t say that buildings and bridges never fall down, but they fall down much less than the number of times our programs will crash, making comparisons with engineers quite unfair. Second, the analogies break down very quickly when given even a cursory glance. To turn things around, if bridge building were like programming, halfway through we’d find out that the far bank was now 50 meters farther out, that it was actually mud rather than granite, and that rather than building a footbridge we were instead building a road bridge. Our software isn’t constrained by the same physical rules that real architects or engineers have to deal with, and what we create is designed to flex and adapt and evolve with user requirements.

A lot of times I really don't understand my own job.

Sunday, February 01, 2015

Dice, Test First and saving time

tl;dr — Test first saves time. Also a dice expression calculator.

The setup

Larry Wall famously remarked, "the three great virtues of a programmer: laziness, impatience, and hubris." A skill earned by good programmers is knowing when to favor which virtue. I failed while writing a parser for "dice expressions" ala Dungeons & Dragons. (Example: "3d6 + 1" means roll 3 6-sided dice, sum the results and add 1.)

The parser is straight-forward using the popular ANTLR4 tools:

grammar DiceExpression;

expr: left=expr op=('*' | '/') right=expr #opExpr
    | left=expr op=('+' | '-') right=expr #opExpr
    | '(' group=expr ')' #groupExpr
    | van=('adv' | 'dis')? number=NUMBER? sides=SIDES #diceExpr
    | number=NUMBER #numberExpr
    ;

dice: number=NUMBER? sides=SIDES;

NUMBER: '1'..'9' '0'..'9'*;
SIDES: 'd' ('4' | '6' | '8' | '10' | '12' | '20' | '100')
    { setText(getText().substring(1)); };

WS: [ \t]+ -> skip;

The grammar supports a 5th Edition feature, Advantage/Disadvantage—where you get to roll twice and pick the better/worse result—, with the keywords adv/dis. A less-5th Edition-specific grammar would leave out that term from "#diceExpr".

Enter the rube

All tests pass: good! Then I thought, "What about whitespace in dice rolls or with adv/dis?" I was concerned with expressions such as "1 +disd6" or "3 d4". Surely those are invalid, but I don't check for them. So I began adding explicit whitespace parsing in the #diceExpr rule. Mistake!

Several of my tests failed for perfectly good dice expressions.

Okay, so I reverted my grammar changes. To understand what was going on, I added tests I should have started with before editing (lines marked "FAIL"):

@RunWith(Parameterized.class)
public final class DiceExpressionTest {
    @Parameters(name = "{index}: '{0}'")
    public static Collection<Object> parameters() {
        return asList(args("1", true),
                args("d6", true),
                args("3d6", true),
                args("1+3d6", true),
                args("adv d6", true),
                args("dis 3d6", true),
                args("disd6", false), // FAIL
                args("1 + 3 d6", false)); // FAIL
    }

    private final ExpectedException thrown = ExpectedException.none();

    @Parameter(0)
    public String in;
    @Parameter(1)
    public boolean ok;

    @Test
    public void shouldParse() {
        if (!ok)
            thrown.expect(ParseCancellationException.class);

        // Just check parsing works, ignore results
        DiceExpression.valueOf(in).evalWith(Roll.die());
    }

    private static Object[] args(final Object... args) {
        return args;
    }
}

And ... the "FAIL" tests pass with the original grammar. This got me to thinking more about how ANTLR works. It is a lexer/parser. The lexer grabs strings from the input and hands them to the parser to figure out. The lexer has no brains, it's an expert at chopping up input. (Great post on this.)

The cases I worried about? The lexer does not know how to chop up the first one ("disd6"): it matches no token pattern. The parser does not understand the tokens in the second one ("1 + 3 d6"): it expects an arithmetic operator between "3" and "d6". My grammar already did the right thing.

Had I started with tests for these cases, instead of jumping right to coding, I would have not spent time messing with whitespace in the grammar. Coding first cost time rather than saved it.

Enlightenment by coding Kōan

Write tests first! No, really. Even when you "know" the solution, write tests first. There are many ways it saves time in the long run, it helps you think about your API, and it can save time in the short run. My dice expression example is not unique: test-first avoids unneeded work.

Epilogue

What does the dice expression implementation look like?

public final class DiceExpression {
    private final ExprContext expression;

    @Nonnull
    public static DiceExpression valueOf(@Nonnull final String expression) {
        final DiceExpressionLexer lexer = new DiceExpressionLexer(
                new ANTLRInputStream(expression));
        final DiceExpressionParser parser = new DiceExpressionParser(
                new CommonTokenStream(lexer));
        parser.setErrorHandler(new BailErrorStrategy());

        return new DiceExpression(parser.expr());
    }

    private DiceExpression(final ExprContext expression) {
        this.expression = expression;
    }

    public int evalWith(final Roll roll) {
        return expression.accept(new EvalWithVisitor(roll));
    }

    private static final class EvalWithVisitor
            extends DiceExpressionBaseVisitor<Integer> {
        private final Roll roll;

        public EvalWithVisitor(final Roll roll) {
            this.roll = roll;
        }

        @Override
        public Integer visitOpExpr(final OpExprContext ctx) {
            final int left = visit(ctx.left);
            final int right = visit(ctx.right);
            switch (ctx.op.getText().charAt(0)) {
            case '+': return left + right;
            case '-': return left - right;
            case '*': return left * right;
            case '/': return left / right;
            }
            throw new ArithmeticException(
                    "Unknown operator: " + ctx.getText());
        }

        @Override
        public Integer visitGroupExpr(final GroupExprContext ctx) {
            return visit(ctx.group);
        }

        @Override
        public Integer visitDiceExpr(final DiceExprContext ctx) {
            return Dice.valueOf(ctx).evalWith(roll);
        }

        @Override
        public Integer visitNumberExpr(final NumberExprContext ctx) {
            return Integer.valueOf(ctx.number.getText());
        }
    }
}

And Dice, overly complicated until I can think through Advantage/Disadvantage further:

public class Dice
        implements Comparable<Dice> {
    @FunctionalInterface
    public interface Roll {
        int roll(final int sides);

        static Roll die() {
            final Random random = new Random();
            return sides -> random.nextInt(sides) + 1;
        }
    }

    public enum Advantage {
        DIS(2, Math::min),
        NONE(1, (a, b) -> a),
        ADV(2, Math::max);

        private final int rolls;
        private final IntBinaryOperator op;

        Advantage(final int rolls, final IntBinaryOperator op) {
            this.rolls = rolls;
            this.op = op;
        }

        public final int evalWith(final Roll roll, final Dice dice) {
            return range(0, rolls).
                    map(n -> dice.rollWith(roll)).
                    reduce(op).
                    getAsInt();
        }

        public final String toString(final Dice dice) {
            switch (this) {
            case NONE:
                return dice.stringOf();
            default:
                return name().toLowerCase() + ' ' + dice.stringOf();
            }
        }
    }

    private final Optional<Integer> number;
    private final int sides;
    private final Advantage advantage;

    @Nonnull
    public static Dice valueOf(@Nonnull final String expression) {
        final DiceExpressionLexer lexer = new DiceExpressionLexer(
                new ANTLRInputStream(expression));
        final DiceExpressionParser parser = new DiceExpressionParser(
                new CommonTokenStream(lexer));
        parser.setErrorHandler(new BailErrorStrategy());

        return valueOf((DiceExprContext) parser.expr());
    }

    @Nonnull
    static Dice valueOf(@Nonnull final DiceExprContext ctx) {
        final Optional<Integer> number = Optional.ofNullable(ctx.number).
                map(Token::getText).
                map(Integer::valueOf);
        final Integer sides = Integer.valueOf(ctx.sides.getText());
        final Advantage advantage = Optional.ofNullable(ctx.van).
                map(Token::getText).
                map(String::toUpperCase).
                map(Advantage::valueOf).
                orElse(NONE);
        return new Dice(number, sides, advantage);
    }

    public Dice(@Nonnull final Optional<Integer> number, final int sides,
            final Advantage advantage) {
        this.number = number;
        this.sides = sides;
        this.advantage = advantage;
    }

    public Dice(@Nullable final Integer number, final int sides,
            final Advantage advantage) {
        this(Optional.ofNullable(number), sides, advantage);
    }

    public Dice(@Nullable final Integer number, final int sides) {
        this(Optional.ofNullable(number), sides, NONE);
    }

    public int number() {
        return number.orElse(1);
    }

    public int sides() {
        return sides;
    }

    public Advantage advantage() {
        return advantage;
    }

    public int evalWith(final Roll roll) {
        return advantage.evalWith(roll, this);
    }

    protected int rollWith(final Roll roll) {
        return rangeClosed(1, number()).
                map(ignored -> roll.roll(sides())).
                sum();
    }

    @Override
    public int compareTo(@Nonnull final Dice that) {
        // Sort by roll average - adv/dis messes this up though
        final int compare = Integer.compare(number() + (sides() + 1),
                that.number() * (that.sides() + 1));
        return 0 == compare ? advantage().compareTo(that.advantage())
                : compare;
    }

    @Override
    public String toString() {
        return advantage.toString(this);
    }

    protected String stringOf() {
        return number.
                map(number -> number + "d" + sides()).
                orElse("d" + sides());
    }

    @Override
    public boolean equals(final Object o) {
        if (this == o)
            return true;
        if (o == null || getClass() != o.getClass())
            return false;

        final Dice that = (Dice) o;

        return sides() == that.sides() && number() == that.number()
                && advantage() == that.advantage();
    }

    @Override
    public int hashCode() {
        return 31 * (31 * sides() + number()) * advantage().hashCode();
    }
}
UPDATE: If you wonder about the parser visitor, you have to explicitly enable it in ANTLR4 (even though it is recommended over tree rewriting). Use the command line flag, or in Maven

<plugin>
    <groupId>org.antlr</groupId>
    <artifactId>antlr4-maven-plugin</artifactId>
    <version>${antlr4.version}</version>
    <configuration>
        <visitor>true</visitor>
    </configuration>
    <executions>
        <execution>
            <id>antlr4</id>
            <goals>
                <goal>antlr4</goal>
            </goals>
        </execution>
    </executions>
</plugin>

Thursday, December 25, 2014

CompletableFuture and ExecutorService

Introduction

CompletableFuture was one of the "small gifts" in Java 8. It is a clever class but not well-integrated into the rest of the JDK. Particularly, ExecutorService still returns Futures rather than CompletableFutures. No class in the JDK references completable futures.

The other odd thing about CompletableFuture is that methods such as get() declare throwing InterruptedException but do not do so except under narrow circumstances: tasks which are interrupted and themselves throw InterruptedException have those exceptions wrapped by ExecutionException, making is difficult to handle interrupts in a general way. This is "baked into" the API, which provides only static factory methods accepting Runnable or Supplier (e.g., supplyAsync), and clashes with standard ExecutorService implementations.

Oddly the source for CompletableFuture shows interrupts could have been addressed in a straight-forward way:

public T get() throws InterruptedException, ExecutionException {
    Object r; Throwable ex, cause;
    if ((r = result) == null && (r = waitingGet(true)) == null)
        throw new InterruptedException();
    if (!(r instanceof AltResult)) {
        @SuppressWarnings("unchecked") T tr = (T) r;
        return tr;
    }
    if ((ex = ((AltResult)r).ex) == null)
        return null;
    if (ex instanceof CancellationException)
        throw (CancellationException)ex;
    // Hypothetical approach to exposing interrupts, NOT in the JDK:
    // if (ex instanceof InterruptedException)
    //     throw (InterruptedException)ex;
    if ((ex instanceof CompletionException) &&
        (cause = ex.getCause()) != null)
        ex = cause;
    throw new ExecutionException(ex);
}

I suspect there is some deeper interaction I am missing that such an easy solution was avoided. (This also shows off nicely the new ability in Java 8 to annotate assignments.)

That I can tell CompletableFuture was modeled on other libraries and languages, especially Guava's SettableFuture and Akka's Promise (formerly named CompletableFuture). Tomasz Nurkiewicz points out the considerable value-add in the Java 8 variant. Koji Lin provides the slides.

Solution

Let's integrate CompletableFuture into ExecutorService.

The natural approach is to extend ExecutorService, overriding methods which return Future to return CompletableFuture (covariant return from Java 5). This means updating methods which construct or return ExecutorService to return, say, CompletableExecutorService. My ideal solution uses a non-existent Java language feature, assignment to this for delegation (alas not in this timeline). A practical solution is mixins. So let's write that:

public interface CompletableExecutorService extends ExecutorService {
    /**
     * @return a completable future representing pending completion of the
     * task, never missing
     */
    @Nonnull
    @Override
    <T> CompletableFuture<T> submit(@Nonnull final Callable<T> task);

    /**
     * @return a completable future representing pending completion of the
     * task, never missing
     */
    @Nonnull
    @Override
    <T> CompletableFuture<T> submit(@Nonnull final Runnable task,
            @Nullable final T result);

    /**
     * @return a completable future representing pending completion of the
     * task, never missing
     */
    @Nonnull
    @Override
    CompletableFuture<?> submit(@Nonnull final Runnable task);
}

A static factory method turns any ExecutorService into a CompletableExecutorService:

@Nonnull
public static CompletableExecutorService completable(
        @Nonnull final ExecutorService threads) {
    return newMixin(CompletableExecutorService.class,
            new Overrides(threads), threads);
}

The grunt work is in Overrides:

public static final class Overrides {
    private final ExecutorService threads;

    private Overrides(final ExecutorService threads) {
        this.threads = threads;
    }

    @Nonnull
    public <T> CompletableFuture<T> submit(
            @Nonnull final Callable<T> task) {
        final CompletableFuture<T> cf = new UnwrappedCompletableFuture<>();
        threads.submit(() -> {
            try {
                cf.complete(task.call());
            } catch (final CancellationException e) {
                cf.cancel(true);
            } catch (final Exception e) {
                cf.completeExceptionally(e);
            }
        });
        return cf;
    }

    @Nonnull
    public <T> CompletableFuture<T> submit(@Nonnull final Runnable task,
            @Nullable final T result) {
        return submit(callable(task, result));
    }

    @Nonnull
    public CompletableFuture<?> submit(@Nonnull final Runnable task) {
        return submit(callable(task));
    }
}

What is UnwrappedCompletableFuture? It handles the pesky issue mentioned above with interrupts:

private static final class UnwrappedCompletableFuture<T>
        extends CompletableFuture<T> {
    @Override
    public T get() throws InterruptedException, ExecutionException {
        return UnwrappedInterrupts.<T, RuntimeException>unwrap(super::get);
    }

    @Override
    public T get(final long timeout, final TimeUnit unit)
            throws InterruptedException, ExecutionException,
            TimeoutException {
        return UnwrappedInterrupts.<T, TimeoutException>unwrap(
                () -> super.get(timeout, unit));
    }

    @FunctionalInterface
    private interface UnwrappedInterrupts<T, E extends Exception> {
        T get() throws InterruptedException, ExecutionException, E;

        static <T, E extends Exception> T unwrap(
                final UnwrappedInterrupts<T, E> wrapped)
                throws InterruptedException, ExecutionException, E {
            try {
                return wrapped.get();
            } catch (final ExecutionException e) {
                final Throwable cause = e.getCause();
                if (cause instanceof InterruptedException)
                    throw (InterruptedException) cause;
                throw e;
            }
        }
    }
}

Wednesday, December 17, 2014

Blog code 0.5

I've published to Maven Central a set of Java jars capturing code and ideas from this blog and Internet reading. The maven coordinates are hm.binkley:*:0.5. Other vital statistics:

I still need to update the javadoc pages hosted by GitHub. I'm particularly happy to have finally worked out how to make a lombok processor.

Where I fit in

While reading on how to improve recruiting for Macquarie, I ran across an interesting job candidate description. Not a particular applicant, a description of a type of applicant: Five Tips to Hiring a Generalizing Specialist. Apparently there is a name for people like me. (More at Generalizing Specialists: Improving Your IT Career Skills.)

My career path has been atypical. I graduated a with a degree in classical music and jumped into programming out of need. I was very fortunate to have smart, capable friends who recommended the right books. And I wound up one of those, a "generalizing specialist".

It makes for a non-linear work life, which is challenging, and leads to opportunities less available otherwise. It is never dull.

Tuesday, December 09, 2014

Java validation

Martin Fowler posted Replacing Throwing Exceptions with Notification in Validations discussing alternatives to data validation than throwing exceptions. There are off-the-shelf solutions such as Commons Validator (XML driven) or Bean Validation (annotation driven) which are complete frameworks.

There is more to these frameworks than I suggest, but to explore Fowler's post better I quickly coded up my own simple-minded approach:

public final class ValidationMain {
    public static void main(final String... args) {
        final Notices notices = new Notices();
        notices.add("Something went horribly wrong %d time(s)", 1);
        try {
            foo(null);
        } catch (final Exception e) {
            notices.add(e);
        }
        notices.forEach(err::println);
        notices.fail(IllegalArgumentException::new);
    }

    public static String foo(final Object missing) {
        return missing.toString();
    }
}

Output on stderr:

lab.Notices$Notice@27f8302d
lab.Notices$Notice@4d76f3f8
Exception in thread "main" java.lang.IllegalArgumentException: 2 notices:
 Something went horribly wrong 1 time(s)
 at lab.ValidationMain.main(ValidationMain.java:21)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:483)
 at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
 Suppressed: java.lang.IllegalArgumentException: Only 1 reason(s)
  at lab.ValidationMain.main(ValidationMain.java:14)
 Suppressed: java.lang.IllegalArgumentException
  at lab.ValidationMain.main(ValidationMain.java:18)
 Caused by: java.lang.NullPointerException
  at lab.ValidationMain.foo(ValidationMain.java:25)
  at lab.ValidationMain.main(ValidationMain.java:16)

And the Notices class:

public final class Notices
        implements Iterable<Notice> {
    private final List<Notice> notices = new ArrayList<>(0);

    public void add(final String reason, final Object... args) {
        // Duplicate code so stack trace keeps same structure
        notices.add(new Notice(null, reason, args));
    }

    public void add(final Throwable cause) {
        // Duplicate code so stack trace keeps same structure
        notices.add(
                new Notice(cause, null == cause ? null : cause.getMessage()));
    }

    public void add(final Throwable cause, final String reason,
            final Object... args) {
        // Duplicate code so stack trace keeps same structure
        notices.add(new Notice(cause, reason, args));
    }

    public <E extends Exception> void fail(
            final BiFunction<String, Throwable, E> ctor)
            throws E {
        final E e = ctor.apply(toString(), null);
        notices.forEach(n -> e.addSuppressed(n.as(ctor)));
        final List<StackTraceElement> frames = asList(e.getStackTrace());
        // 2 is the magic number: lambda, current
        e.setStackTrace(frames.subList(2, frames.size())
                .toArray(new StackTraceElement[frames.size() - 2]));
        throw e;
    }

    @Override
    public Iterator<Notice> iterator() {
        return unmodifiableList(notices).iterator();
    }

    @Override
    public String toString() {
        if (notices.isEmpty())
            return "0 notices";
        final String sep = lineSeparator() + "\t";
        return notices.stream().
                map(Notice::reason).
                filter(Objects::nonNull).
                collect(joining(sep,
                        format("%d notices:" + sep, notices.size()), ""));
    }

    public static final class Notice {
        // 4 is the magic number: Thread, init, init, addError, finally the
        // user code
        private final StackTraceElement location = currentThread()
                .getStackTrace()[4];
        private final Throwable cause;
        private final String reason;
        private final Object[] args;

        private Notice(final Throwable cause, final String reason,
                final Object... args) {
            this.cause = cause;
            this.reason = reason;
            this.args = args;
        }

        public Throwable cause() {
            return cause;
        }

        public String reason() {
            return null == reason ? null : format(reason, args);
        }

        private <E extends Exception> E as(
                final BiFunction<String, Throwable, E> ctor) {
            final E e = ctor.apply(reason(), cause);
            e.setStackTrace(new StackTraceElement[]{location});
            return e;
        }
    }
}

Comments:

  • I manipulate the stack traces to focus on the caller's point of view. This is the opposite of, say, Spring Framework.
  • I haven't decided on what an intelligent toString() should look like for Notice
  • Java 8 lambdas really shine here. Being able to use exception constructors as method references is a win.

Friday, December 05, 2014

Writing your own lombok annotation

It took me quite a while to get around to writing my own lombok annotation and processor. This took more effort than I expected, hopefully this post will save someone else some.

tl;dr — Look at the source in my Github repo.

Motivation

Reading the excellent ExecutorService - 10 tips and tricks by Tomasz Nurkiewicz, I thought about tip #2, Switch names according to context, which recommends wrapping important methods and code blocks with custom thread names to aid in logging and debugging.

"This is a great use case for annotations!" I thought. The code screams boilerplate:

public void doNiftyThings() {
    final Thread thread = Thread.currentThread();
    final String oldName = thread.getName();
    thread.setName("Boy this is nifty!");
    try {
        // Do those nifty things - the actual work
    } finally {
        thread.setName(oldName);
    }
}

The whole point of the method is indented out of focus, wrapped with bookkeeping. I'd rather write this:

@ThreadNamed("Boy this is nifty!")
public void doNiftyThings() {
    // Do those nifty thing - the actual work
}

Bonus: simple text search finds those places in my code where I change the thread name based on context.

Writing the annotation

Ok, let's make this work. I started with cloning the @Cleanup annotation and processor, and editing from there. First the annotation, the easy bit. I include the javadoc to emphasize the importance of documenting your public APIs.

/**
 * {@code ThreadNamed} sets the thread name during method execution, restoring
 * it when the method completes (normally or exceptionally).
 *
 * @author <a href="mailto:binkley@alumni.rice.edu">B. K. Oxley (binkley)</a>
 */
@Documented
@Retention(SOURCE)
@Target({CONSTRUCTOR, METHOD})
public @interface ThreadNamed {
    /** The name for the thread while the annotated method executes. */
    String value();
}

Nothing special here. I've made the decision to limit the annotation to methods and constructors. Ideally I'd include blocks but that isn't an option (yet) in Java, and you can always refactor out a block to a method.

Writing the processor

This is the serious part. First some preliminaries:

  1. I have only implemented support for JDK javac. Lombok also supports the Eclipse compiler, which requires a separate processor class. I have nothing against Eclipse, but it's not in my toolkit.
  2. I'll discuss library dependencies below. For now pretend these are already working for you.
  3. I'm a big fan of static imports, the diamond operator, etc. I don't like retyping what the compiler is already thinking. You should note List below is not java.util.List; it's com.sun.tools.javac.util.List. Yeah, I don't know this class either.
  4. The implementation is hard to follow. Most of us don't spend much time with expression trees, which is how most compilers (including javac) see your source code. A language like LISP lets you write you code as the expression tree directly, which is both nifty and challenging (macros being like annotation processors).

Without further ado:

/**
 * Handles the {@code lombok.ThreadNamed} annotation for javac.
 */
@MetaInfServices(JavacAnnotationHandler.class)
@HandlerPriority(value = 1024)
// 2^10; @NonNull must have run first, so that we wrap around the
// statements generated by it.
public class HandleThreadNamed
        extends JavacAnnotationHandler<ThreadNamed> {
    /**
     * lombok configuration: {@code lab.lombok.threadNamed.flagUsage} = {@code
     * WARNING} | {@code ERROR}.
     * <p>
     * If set, <em>any</em> usage of {@code @ThreadNamed} results in a warning
     * / error.
     */
    public static final ConfigurationKey<FlagUsageType>
            THREAD_NAMED_FLAG_USAGE = new ConfigurationKey<FlagUsageType>(
            "lab.lombok.threadNamed.flagUsage",
            "Emit a warning or error if @ThreadNamed is used.") {
    };

    @Override
    public void handle(final AnnotationValues<ThreadNamed> annotation,
            final JCAnnotation ast, final JavacNode annotationNode) {
        handleFlagUsage(annotationNode, THREAD_NAMED_FLAG_USAGE,
                "@ThreadNamed");

        deleteAnnotationIfNeccessary(annotationNode, ThreadNamed.class);
        final String threadName = annotation.getInstance().value();
        if (threadName.isEmpty()) {
            annotationNode.addError("threadName cannot be the empty string.");
            return;
        }

        final JavacNode owner = annotationNode.up();
        switch (owner.getKind()) {
        case METHOD:
            handleMethod(annotationNode, (JCMethodDecl) owner.get(),
                    threadName);
            break;
        default:
            annotationNode.addError(
                    "@ThreadNamed is legal only on methods and constructors"
                            + ".");
            break;
        }
    }

    public void handleMethod(final JavacNode annotation,
            final JCMethodDecl method, final String threadName) {
        final JavacNode methodNode = annotation.up();

        if ((method.mods.flags & Flags.ABSTRACT) != 0) {
            annotation.addError(
                    "@ThreadNamed can only be used on concrete methods.");
            return;
        }

        if (method.body == null || method.body.stats.isEmpty()) {
            generateEmptyBlockWarning(annotation, false);
            return;
        }

        final JCStatement constructorCall = method.body.stats.get(0);
        final boolean isConstructorCall = isConstructorCall(constructorCall);
        List<JCStatement> contents = isConstructorCall
                ? method.body.stats.tail : method.body.stats;

        if (contents == null || contents.isEmpty()) {
            generateEmptyBlockWarning(annotation, true);
            return;
        }

        contents = List
                .of(buildTryFinallyBlock(methodNode, contents, threadName,
                        annotation.get()));

        method.body.stats = isConstructorCall ? List.of(constructorCall)
                .appendList(contents) : contents;
        methodNode.rebuild();
    }

    public void generateEmptyBlockWarning(final JavacNode annotation,
            final boolean hasConstructorCall) {
        if (hasConstructorCall)
            annotation.addWarning(
                    "Calls to sibling / super constructors are always "
                            + "excluded from @ThreadNamed;"
                            + " @ThreadNamed has been ignored because there"
                            + " is no other code in " + "this constructor.");
        else
            annotation.addWarning(
                    "This method or constructor is empty; @ThreadNamed has "
                            + "been ignored.");
    }

    public JCStatement buildTryFinallyBlock(final JavacNode node,
            final List<JCStatement> contents, final String threadName,
            final JCTree source) {
        final String currentThreadVarName = "$currentThread";
        final String oldThreadNameVarName = "$oldThreadName";

        final JavacTreeMaker maker = node.getTreeMaker();
        final Context context = node.getContext();

        final JCVariableDecl saveCurrentThread = createCurrentThreadVar(node,
                maker, currentThreadVarName);
        final JCVariableDecl saveOldThreadName = createOldThreadNameVar(node,
                maker, currentThreadVarName, oldThreadNameVarName);

        final JCStatement changeThreadName = setThreadName(node, maker,
                maker.Literal(threadName), currentThreadVarName);
        final JCStatement restoreOldThreadName = setThreadName(node, maker,
                maker.Ident(node.toName(oldThreadNameVarName)),
                currentThreadVarName);

        final JCBlock tryBlock = setGeneratedBy(maker.Block(0, contents),
                source, context);
        final JCTry wrapMethod = maker.Try(tryBlock, nil(),
                maker.Block(0, List.of(restoreOldThreadName)));

        if (inNetbeansEditor(node)) {
            //set span (start and end position) of the try statement and
            // the main block
            //this allows NetBeans to dive into the statement correctly:
            final JCCompilationUnit top = (JCCompilationUnit) node.top()
                    .get();
            final int startPos = contents.head.pos;
            final int endPos = Javac
                    .getEndPosition(contents.last().pos(), top);
            tryBlock.pos = startPos;
            wrapMethod.pos = startPos;
            Javac.storeEnd(tryBlock, endPos, top);
            Javac.storeEnd(wrapMethod, endPos, top);
        }

        return setGeneratedBy(maker.Block(0,
                        List.of(saveCurrentThread, saveOldThreadName,
                                changeThreadName, wrapMethod)), source,
                context);
    }

    private static JCVariableDecl createCurrentThreadVar(final JavacNode node,
            final JavacTreeMaker maker, final String currentThreadVarName) {
        return maker.VarDef(maker.Modifiers(FINAL),
                node.toName(currentThreadVarName),
                genJavaLangTypeRef(node, "Thread"), maker.Apply(nil(),
                        genJavaLangTypeRef(node, "Thread", "currentThread"),
                        nil()));
    }

    private static JCVariableDecl createOldThreadNameVar(final JavacNode node,
            final JavacTreeMaker maker, final String currentThreadVarName,
            final String oldThreadNameVarName) {
        return maker.VarDef(maker.Modifiers(FINAL),
                node.toName(oldThreadNameVarName),
                genJavaLangTypeRef(node, "String"),
                getThreadName(node, maker, currentThreadVarName));
    }

    private static JCMethodInvocation getThreadName(final JavacNode node,
            final JavacTreeMaker maker, final String currentThreadVarNAme) {
        return maker.Apply(nil(),
                maker.Select(maker.Ident(node.toName(currentThreadVarNAme)),
                        node.toName("getName")), nil());
    }

    private static JCStatement setThreadName(final JavacNode node,
            final JavacTreeMaker maker, final JCExpression threadName,
            final String currentThreadVarName) {
        return maker.Exec(maker.Apply(nil(),
                maker.Select(maker.Ident(node.toName(currentThreadVarName)),
                        node.toName("setName")), List.of(threadName)));
    }
}

Wasn't that easy?

Dependencies

Of course the code depends on lombok. I'm using version 1.14.8. It also needs tools.jar from the JDK for compiler innards like expression trees. (An Eclipse processor needs an equivalent.)

Unfortunately lombok itself uses "mangosdk" to generate a META-INF/services/lombok.javac.JavacAnnotationHandler file for autodiscovery of processors. I say 'unfortunately' because this library is not in maven and is unsupported. Happyily Kohsuke Kawaguchi wrote the excellent metainf-services library a while back, maintains it, and publishes to Maven central. If you're new to annotation processors it's a good project to learn from.

Conclusion

Ok, that was not actually so easy. On the other hand, finding a starting point was the biggest hurdle for me in writing a lombok annotation. Please browse my source and try your hand at one.

UPDATE — A little bonus. This code:

@ThreadNamed("Slot #%2$d")
public void doSomething(final String name, final int slot) {
    // Do something with method params
}

Produces the thread name "Slot #2" when called with "Foo", 2. Strings without formatting or methods with params treat the annotation value as a plain string.

Sunday, November 30, 2014

The ORM anti-pattern

Yegor Bugayenko writes on ORM Is an Offensive Anti-Pattern, offering a pure OO alternative.

(I've posted too many links to my Google+ account. It's a ghetto over there as my public posts are science-oriented, programming is shared privately.)

Wednesday, November 12, 2014

Aiming for the right level

Vinod Kumaar Ramakrishnan writes It is just a road not a roadmap making a strong point visually: software needs a map, not a road. Roadmap

This is important to understand at any level of an organization. The problem comes as you perform larger roles.

You can track only so much detail—minds have a capacity. Using a Road rather than a Map overcomes this for viewing lower down the organization. Pull back your view and substitutes rough pictures for details. Then build a bigger map, less granular, covering a larger area. It's still a map, but a map of a country rather than a region or a place.

Managing calls for finding the right level of detail, be it the application or team or project or programme or department. But remember to keep rebuilding your map and explore some.

Sunday, August 31, 2014

Overcoming Java 8 streams

Java 8 streams provide functional and LINQ-like features in a fluent API. But streams are not without drawbacks:

  • Referenced methods and lambdas cannot throw checked exceptions
  • Controlling the threads used, especially for parallel streams, is awkward
  • Streams are not designed for extension

Overcoming these drawbacks requires a "look-a-like" API. For example, implementing java.util.stream.Stream does not help: none of the existing methods throw checked exceptions, and none of the existing stream factory helpers would return your implementation with new methods.

So I wrote my own, copying the existing stream API, updating the methods to throw checked exceptions:

hm.binkley.util.stream.CheckedStream ('develop' branch for now)

From the javadoc:

CheckedStream is a throwing Stream look-a-like with control over thread pool. It cannot be a Stream as it takes throwing versions of suppliers, functions and consumers. Otherwise it is a faithful reproduction.

Write this:

   long beanCount() throws SomeException, OtherException {
       checked(Stream.of(1, 2, 3)).
           map(this::someThrowingFunction).
           peek(That::oldBean).
           count();
   }

not this:

   long beanCount() throws SomeException, OtherException {
       try {
           Stream.of(1, 2, 3).
               map(i -> {
                   try {
                       someThrowingFunction(i);
                   } catch (final SomeException e) {
                       throw new RuntimeException(e);
                   }
               }).
               peek(i -> {
                   try {
                       That.oldBean(i);
                   } catch (final OtherException e) {
                       throw new RuntimeException(e);
                   }
               }).
               count();
       } catch (final RuntimeException e) {
           final Throwable x = e.getCause();
           if (x instanceof SomeException)
               throw (SomeException) x;
           if (x instanceof OtherException)
               throw (OtherException) x;
           throw e;
       }
   }

"Intentional" exceptions (checked exceptions plus CancellationException) have "scrubbed" stacktraces: frames from framework/glue packages are removed before the intentional exception is rethrown to calling code. Scrubbed stacktraces are much easier to understand, the framework and glue code having been removed.

To see the unscrubbed stacktrace, set the system property "hm.binkley.util.stream.CheckedStream.debug" to "true".

Controlling the thread pool used by Stream is a challenge. Deep in the implementation, it checks if being run in a ForkJoinTask, and uses that thread if so, otherwise using the common pool. So with CheckedStream write this:

       checked(stream, new ForkJoinPool()).
           map(currentThread()).
           forEach(System.out::println);

not this:

       try {
           new ForkJoinPool().submit(() -> stream.
                   map(currentThread()).
                   forEach(System.out::println)).
               get();
       } catch (final ExecutionException e) {
           final Throwable x = e.getCause();
           if (x instanceof Error)
               throw (Error) x;
           if (x instanceof RuntimeException)
               // Much tricker when stream functions throw runtime
               throw (RuntimeException) x;
           throw new Error(e); // We have no checked exceptions in this example
       }

Care is taken to respect lazy and terminal operations in using thread pools. Changing thread pool or thread mode mid-stream is supported, and are "immediate" operations: they terminate the existing stream, and start a new one with the changes:

stream.sequential().
    filter(this::someFilter).
    parallel(threads). // Existing lazy operations terminated
    map(this:someMapper).
    forEach(System.out::println);

Immediate operations ensure stream methods are run in the correct threading context.

I hope you'll agree: CheckedStream is nicer to use, especially with existing code using checked exceptions.

Suggestions, bug fixes, improvements welcome!

Tuesday, July 15, 2014

Java 8 magic exception copying

Since I can in Java 8 now parameterize constructors as functions, I can write a generic exception copier:

<E extends Throwable>
E copy(final Throwable from, final Function<String, E> ctor) {
    final E to = ctor.apply(from.getMessage());
    to.setStackTrace(from.getStackTrace());
    for (final Throwable s : from.getSuppressed())
        to.addSuppressed(s);
    return to;
}

Example:

try {
    // Something throws
} catch (final AnException | BeException | KatException e) {
    throw copy(e, IOException::new);
}

This is not a common strategy but one I sometimes use to reduce the incredible size of layered exceptions, especially for logging. It is also handy for shoehorning 3rd-party exceptions into a common framework exception, a nice feature for APIs to simplify calling code. Copy helps reduce boilerplate code.

Thursday, July 10, 2014

Lambda annotations

I am still startled by Java. While playing with Byte Buddy I noodled with adding annotations to a class at runtime. I wrote this:

final Class<? extends X> dynamicType
            = (Class<< extends X>) new ByteBuddy().
        subclass(Object.class).
        implement(X.class).
        annotateType(() -> Inject.class).
        method(named("toString")).intercept(value("Hello World!")).
        method(named("foo")).intercept(value(3)).
        make().
        load(ByteBuddyMain.class.getClassLoader(), WRAPPER).
        getLoaded();

Get a load of "annotatedType"! It wants an annotation instance. Originally I tried:

new Inject() {
    @Override
    public Class<Inject> annotationType() {
        return Inject.class;
    }
}

What the hey, then I thought to try the lambda expression and live dangerously. It works!

Saturday, July 05, 2014

Modern XML to Java

How many frameworks are there for converting XML to Java? Hard to count. As an experiment I tried my hand at one. I have two top-level classes plus an annotation:

public final class XMLFuzzy
        implements InvocationHandler {
    private static final XPath xpath = XPathFactory.newInstance().
            newXPath();
    private static final Map<Method, XPathExpression> expressions
     = new ConcurrentHashMap<>();

    private final Node node;
    private final Converter converter;

    public static final class Factory {
        private final Converter converter;

        public Factory(final Converter converter) {
            this.converter = converter;
        }

        public <T> T of(@Nonnull final Class<T> itf,
                @Nonnull final Node node) {
            return XMLFuzzy.of(itf, node, converter);
        }
    }

    public static <T> T of(@Nonnull final Class<T> itf,
     @Nonnull final Node node,
            @Nonnull final Converter converter) {
        return itf.cast(newProxyInstance(itf.getClassLoader(),
                new Class[]{itf},
                new XMLFuzzy(node, converter)));
    }

    private XMLFuzzy(final Node node, final Converter converter) {
        this.node = node;
        this.converter = converter;
    }

    @Override
    public Object invoke(final Object proxy, final Method method,
         final Object[] args)
            throws Throwable {
        return converter.convert(method.getReturnType(), expressions.
                computeIfAbsent(method, XMLFuzzy::compile).
                evaluate(node));
    }

    private static XPathExpression compile(@Nonnull final Method method) {
        final String expression = asList(method.getAnnotations()).stream().
                filter(From.class::isInstance).
                map(From.class::cast).
                findFirst().
                orElseThrow(() -> new MissingAnnotation(method)).
                value();
        try {
            return xpath.compile(expression);
        } catch (final XPathExpressionException e) {
            throw new BadXPath(method, expression, e);
        }
    }

    public static final class MissingAnnotation
            extends RuntimeException {
        private MissingAnnotation(final Method method) {
            super(format("Missing @X(xpath) annotation: %s", method));
        }
    }

    public static final class BadXPath
            extends RuntimeException {
        private BadXPath(final Method method, final String expression,
                final XPathExpressionException e) {
            super(format("Bad @X(xpath) annotation on '%s': %s: %s",
                    method, expression, e.getMessage()));
            setStackTrace(e.getStackTrace());
        }
    }
}

I have left out Converter; it turns strings into objects of a given type, another example of overimplemented framework code in Java. And the annotation:

@Documented
@Inherited
@Retention(RUNTIME)
@Target(METHOD)
public @interface From {
    String value();
}

The idea is straight-forward: drive the object mapping from XML with XPaths. Credit to XMLBeam for introducing to me the elegant use of JDK proxies for this purpose.

Of course tests:

public final class XMLFuzzyTest {
    private Top top;

    @Before
    public void setUp()
            throws ParserConfigurationException, IOException, SAXException {
        final Document document = DocumentBuilderFactory.newInstance().
                newDocumentBuilder().
                parse(new InputSource(new StringReader(XML)));
        top = new XMLFuzzy.Factory(new Converter()).of(Top.class, document);
    }

    @Test
    public void shouldHandleString() {
        assertThat(top.a(), is(equalTo("apple")));
    }

    @Test
    public void shouldHandlePrimitiveInt() {
        assertThat(top.b(), is(equalTo(3)));
    }

    @Test
    public void shouldHandleRURI() {
        assertThat(top.c(), is(equalTo(URI.create("http://some/where"))));
    }

    @Test(expected = MissingAnnotation.class)
    public void shouldThrowOnMissingAnnotation() {
        top.d();
    }

    @Test(expected = BadXPath.class)
    public void shouldThrowOnBadXPath() {
        top.e();
    }

    public interface Top {
        // For the purposes of this blog post, pretend Java supports
 // multiline string literals
        @Language("XML")
        String XML = "<top>
                    <a>apple</a>
      <b>3</b>
      <c>http://some/where</c>
         </top>";

        @From("//top/a")
        String a();

        @From("//top/b")
        int b();

        @From("//top/c")
        URI c();

        void d();

        @From("dis' ain't xpath")
        void e();
    }
}

Saturday, June 28, 2014

Java 8 predicate for tee

A "tee" method did not make it into Java 8 streams, but you can make your own easily enough:

public final class Tee<T>
        implements Predicate<T> {
    private final Predicate<T> test;
    private final Consumer<T> reject;

    @Nonnull
    public static <T> Predicate<T> tee(
            @Nonnull final Predicate<T> test,
            @Nonnull final Consumer<T> reject) {
        return new Tee<>(test, reject);
    }

    private Tee(final Predicate<T> test,
            final Consumer<T> reject) {
        this.test = test;
        this.reject = reject;
    }

    @Override
    public boolean test(final T t) {
        if (test.test(t))
            return true;
        reject.accept(t);
        return false;
    }

    public static void main(final String... args) {
        asList("a", "bb", "ccc", "dddd").stream().
                filter(tee(Tee::even, err::println)).
                forEach(out::println);
    }

    private static boolean even(final String s) {
        return 0 == s.length() % 2;
    }
}

Monday, May 26, 2014

Help for Java parameterized tests

JUnit has many great features. One I especially like is parameterized tests.

The problem

One itch for me however is setting up the parameters and getting failed tests to name themselves.

Modern JUnit has help on the second itch with the name parameter to @Parameters. You still need to get the test name passed into the constructor.

But what about setting up the test data?

I write clunky code like this:

@Parameters(name = "{index}: {0}")
public static Collection<Object[]> data() {
    return asList(
            new Object[]{"Some test", 1, "a string"},
            new Object[]{"Another test", 2, "another string"}
            /* and so on for each test case */);
}

This is not the worst code, but with more complex inputs or data it grows into a long list of eye-glazing use cases. For me this is hard to maintain and document.

The solution

As an alternative I scouted around and settled on the venerable Windows INI file format. It has several virtues in this context:

  • The format is simple and well known
  • Section titles can be use to name tests
  • Comments are supported

Looking at libraries for INI support in Java, one stands out: ini4j. In particular:

  • Is stable and acceptably documented
  • Supports C-style escape sequences, e.g. \t for TAB—import for my particular tests
  • Accepts reasonable ways to input the INI (URL, input stream, et al)

Integrating this into @Parameters is straight-forward. One approach is this:

@Nonnull
public static List<Object[]> parametersFrom(
        @Nonnull final Ini ini, final Key... keys) {
    final List<Object[]> parameters = new ArrayList<>();
    for (final Section section : ini.values()) {
        final Object[] array = new Object[1 + keys.length];
        array[0] = section.getName();
        for (int i = 0; i < keys.length; i++) {
            final Key key = keys[i];
            final String value = section.fetch(key.name);
            array[1 + i] = null == value ? null : key.get.apply(value);
        }
        parameters.add(array);
    }
    return parameters;
}

public static final class Key {
    public final String name;
    public final Function<String, ?> get;

    @Nonnull
    public static Key of(@Nonnull final String name, @Nonnull final Function<String, ?> get) {
        return new Key(name, get);
    }

    @Nonnull
    public static Key of(@Nonnull final String name) {
        return of(name, Function.identity());
    }

    private Key(final String name, final Function<String, ?> get) {
        this.name = name;
        this.get = get;
     }
}

A key might look like Key.of("amount", BigDecimal::new)

Example

I simplified tests of a money value object to a test name, a value to parse and a currency and amount to expect:

@RunWith(Parameterized.class)
public class MoneyTest {
    // Failed test messages look like:
    // 0: Missing currency: 1
    @Parameters(name = "{index}: {0}: {1}")
    public static Collection<Object[]> parameters()
            throws IOException {
        return parametersFrom(
                new Ini(MoneyTest.class.getResource("MoneyTest.ini")),
                Key.of("value"),
                Key.of("currency", Currency::getInstance),
                Key.of("amount", BigDecimal::new));
    }

    @Rule
    public final ExpectedException thrown = ExpectedException.none();

    private final String description;
    private final String value;
    private final Currency currency;
    private final BigDecimal amount;

    public MoneyTest(@Nonnull final String description,
            @Nonnull final String value,
            @Nullable final Currency currency,
            @Nullable final BigDecimal amount) {
        this.description = description;
        this.value = value;
        this.currency = currency;
        this.amount = amount;

        if (!((null == currency && null == amount)
                || (null != currency && null != amount)))
            throw new IllegalArgumentException(
                    format("%s: currency and amount must both be null or non-null",
                            description));
    }

    @Test
    public void shouldParse() {
        if (null == currency) {
            thrown.expect(MoneyFormatException.class);
            thrown.expectMessage(value);
        }

        final Money money = Money.parse(value);

        assertThat(money.getCurrency(), is(equalTo(currency)));
        assertThat(money.getAmount(), is(equalTo(amount)));
    }
}
; Format - INI file
; Section title is description of test, used in reporting failure
; value is the input to `Money.parse(String)`
; IFF parsing should fail:
;   - Do not provide currency/amount
; IFF parsing should pass:
;   - currency is input to `Currency.getInstance(String)`
;   - amount is input is `new BigDecimal(String)`

[Missing currency]
value = 1

[Single dollar, no whitespace]
value = USD1
currency = USD
amount = 1.00

I like being able to add new tests by text editing of the INI file.

Wednesday, March 05, 2014

World's smallest DAO

The world's smallest Java DAO (with heavy lifting provided by Spring Framework):

public class SimpleDAO {
    private final DataSourceTransactionManager transactionManager;

    public SimpleDAO(DataSourceTransactionManager transactionManager) {
        this.transactionManager = transactionManager;
    }

    public <T> T dao(Dao<T> dao) {
        return dao.using(transactionManager);
    }

    public interface Dao<T> {
        default T using(DataSourceTransactionManager transactionManager) {
            return new TransactionTemplate(transactionManager).execute(
                    status -> on(new JdbcTemplate(transactionManager.getDataSource()), status));
        }

        T on(JdbcTemplate jdbcTemplate, TransactionStatus status);
    }
}

Usage:

class InviteesDAO {
    private final SimpleDAO transact;

    InviteesDAO(DatabaseTransactionManager transactionManager) {
        transact = new SimpleDAO(tranactionManager);
    }

    List<String> getInvitees() {
        return transact.dao((jdbcTemplate, status) -> jdbcTemplate.queryForList(
            "SELECT username FROM invitees", String.class));
    }

    void invite(String username) {
        transact.dao((jdbcTemplate, status) -> jdbcTemplate.update(
            "INSERT INTO invitees (username) VALUES (?)", username));
    }
}

UPDATE: Demonstrate with composition rather than inheritance.

Friday, February 07, 2014

ServiceBinder

I've released ServiceBinder 0.2 to Maven Central (details below; soon to be 0.3 with more documentation). I wrote this to fix a common problem for my projects.

I like using the JDK ServiceLoader for discovery: it is simple to use and understand and always available. And Kawaguchi's META-INF/services generator makes use as simple as an annotation.

However, ServiceLoader is missing one key feature for me. It requires a default constructor for implementations, and I am a fan of constructor injection.

ServiceBinder fills this gap. It discovers service classes as ServiceLoader does, and injects them with Guice or Spring. See the GitHub site for examples and source. A taste:

Guice

public final class SampleModule
        extends AbstractModule {
    @Override
    protected void configure() {
        ServiceBinder.with(binder()).bind(Bob.class);
    }
}

Spring Framework

AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext();
context.register(AppConfig.class);
ServiceBinder.with(context).bind(Bob.class);
context.refresh();

Saturday, February 01, 2014

The Java fake return trick

Sometimes we have a method that does nothing but throw an exception (after munging arguments a bit). For example:

static void fail(Bob bob, Fred fred, Exception cause) {
    throw new SadError(format(
            "Sorry, but {} and {} did not meet today",
            bob, fred), cause);
}

Great when used as:

void friends(Bob bob, Fred fred) {
    try {
        meetAgain(bob, fred);
    } catch (MissedTrainException e) {
        fail(bob, fred, e);
    }
}

But what about this?

PhoneNumber exchange(Bob bob, Fred fred {
    try {
        return beamBusinessCards(bob, fred);
    } catch (LostPhoneException e) {
        fail(bob, fred, e);
        return null; // Never reached
    }
}

There's a neat generics trick to help:

static <R> R fail(Bob bob, Fred fred, Exception cause) {
    // Code unchanged - return ignored
}

Now we can write:

PhoneNumber exchange(Bob bob, Fred fred {
    try {
        return beamBusinessCards(bob, fred);
    } catch (LostPhone e) {
        return fail(bob, fred, e);
    }
}

The main downside is readability for the casual reviewer. Unless expecting this technique, he may think the new catch body returns a value. It may help to limit use to Exception, forcing handling in the caller method, but not with RuntimeException.