Thursday, May 02, 2013

Agile is Money

Fascinating read by Daniel Greening: Agile Capitalization, found thanks to Jeff Sutherland. As Jeff describes:

In many companies, agile software development is misunderstood and misreported, causing taxation increases, higher volatility in Profit and Loss (P&L) statements and manual tracking of programmer hours. One large company’s confused finance department expenses all agile software development and capitalizes waterfall development; projects in this company that go agile see their headcounts cut by 50%. This discourages projects from going agile.

Scrum’s production experiment framework can align well with the principles of financial reporting. In this article, the author explains the basics of capitalization and expensing, and offers a financial framework for capitalizing agile projects that can be understood by both accountants and agile teams.

I'll take double headcount, please. Daniel's introduction:

In many companies, agile software development is misunderstood and misreported, causing taxation increases, higher volatility in Profit and Loss (P&L) statements and manual tracking of programmer hours. I claim Scrum teams create production cost data that are more verifiable, better documented, and more closely aligned with known customer value than most waterfall implementations. Better reporting can mean significant tax savings and greater investor interest. Agile companies should change their financial reporting practices to exploit Scrum’s advantages. It might not be easy, but I did it.

Scrum’s production experiment framework aligns perfectly with the principles of financial reporting.

When I restructured software capitalization according to the principles here during an all-company Scrum transition at a 900-person software company, we delighted auditors, gave more insight to upper management and raised more internal money to hire developers. We gained millions in tax savings, by using Scrum Product Backlog Items to more accurately document and capitalize our software work.

I hope to arm you with perspectives and resources to make the accounting argument for agile capitalization, potentially reducing your company’s tax burden, increasing available funds for engineers, and making your auditors happy

Agile is real money.

Monday, April 29, 2013

Moving a git repository to the subdirectory of another repository

How do I move a git repository to be the subdirectory of another repository? is a question answered umpteen times, and all answers vary somewhat. For my needs, this suffices:

#!/bin/bash

case $# in
  3 ) old_repo=$1 new_repo=$2 new_name=$3 ;;
  * ) echo "$0: old_repo new_repo new_name" ; exit 2 ;;
esac

set -e

trap 'rm -rf $old_work $new_work' EXIT
old_work=$(mktemp -d)
new_work=$(mktemp -d)
git clone $old_repo $old_work
git clone $new_repo $new_work

cd $old_work
# If new_name contains commas, edit the sed command accordingly
git filter-branch --index-filter 'git ls-files -s | sed "s,\t\"*,&'$new_name'/," | GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info && mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE' HEAD

cd $new_work
git remote add $new_name $old_work
git pull $new_name master
git push

-XshowSettings

A java flag I hadn't noticed before but wish I had: -XshowSettings. It dumps out the system properties and a guess at which VM to use (server v. client), among other things. Look here for more magic -X flags.

Monday, April 15, 2013

More good Java 8, living with null

Edwin Dalorzo posts wonderfully on Java 8 Optional Objects: what is the problem, how it is handled in other languages, what Java 8 does about it. It all boils down to the "billion dollar mistake": null.

Thursday, April 11, 2013

Moving, but not the blog

Monday morning, April 1st, my family and I relocate for work to Manila from Houston. My employer, Macquarie Bank, is a conservative, successful investment bank from Australia. 2008 brought difficulties for many firms in the banking industry; for Macquarie it brought opportunity.

I encourage anyone looking for the right work environment to consider Macquarie. For many of my mates this is their first and only job out of college, writing building impressive systems from solid code. It is my only job as satisfying as ThoughtWorks.

Only my family and me are moving. binkley's BLOG remains where it is.

Map view of list key-value pairs in Java

Long time no code.

Example use

import ListMap;

import javax.annotation.Nonnull;
import java.util.ArrayList;

import static java.util.Arrays.asList;

public class AttributesMap
        extends ListMap<FooItem, String, String> {
    public AttributesMap(final FooOperations foos, final String cookie) {
        super(new FooItems(foos, cookie), new FooItemConverter());
    }

    private static class FooItems
            extends ArrayList<FooItem> {
        private final FooOperations foos;
        private final String cookie;

        private FooItems(final FooOperations foos, final String cookie) {
            super(asList(foos.getAttributes(cookie)));
            this.foos = foos;
            this.cookie = cookie;
        }

        @Override
        public FooItem set(final int index, final FooItem element) {
            final FooItem oldElement = super.set(index, element);
            save();
            return oldElement;
        }

        @Override
        public void add(final int index, final FooItem element) {
            super.add(index, element);
            save();
        }

        @Override
        public FooItem remove(final int index) {
            final FooItem oldElement = super.remove(index);
            save();
            return oldElement;
        }

        private void save() {
            try {
                foos.setAttributes(cookie, toArray(new FooItem[size()]));
            } catch (final BadDataRef e) {
                throw new UserRuntimeException(e);
            }
        }
    }

    private static final class FooItemConverter
            implements Converter<FooItem, String, String> {
        @Nonnull
        @Override
        public FooItem toElement(@Nonnull final String key, final String value) {
            return new FooItem(key, value);
        }

        @Nonnull
        @Override
        public Entry<String, String> toEntry(@Nonnull final FooItem element) {
            return new SimpleEntry<>(element.tag, element.value);
        }
    }
}

ListMap utility class

import javax.annotation.Nonnull;
import javax.annotation.concurrent.NotThreadSafe;
import java.util.AbstractMap;
import java.util.AbstractSet;
import java.util.List;
import java.util.ListIterator;

/**
 * {@code ListMap} provides a read-write map view over a list of elements.  The elements must
 * provide a key-value pair structure; keys must {@link #equals(Object)} and {@link #hashCode()}.
 * <p/>
 * All map mutating methods also mutate the underlying list.  Mutating the underlying list also
 * mutates the map view.
 * <p/>
 * There is no synchronization; instances are not thread safe.
 *
 * @param <T> the type of list elements
 * @param <K> the type of map keys
 * @param <V> the type of map values
 *
 * @author <a href="mailto:binkley@alumni.rice.edu">B. K. Oxley (binkley)</a>
 */
@NotThreadSafe
public class ListMap<T, K, V>
        extends AbstractMap<K, V> {
    private final List<T> elements;
    private final Converter<T, K, V> converter;

    /**
     * Creates a new {@code ListMap} for the given <var>elements</var> and <var>converter</var>.
     *
     * @param elements the list of elements underlying the map, never missing
     * @param converter the converter between elements and entry objects, never missing
     */
    public ListMap(@Nonnull final List<T> elements, @Nonnull final Converter<T, K, V> converter) {
        this.elements = elements;
        this.converter = converter;
    }

    /**
     * {@inheritDoc}
     * <p/>
     * Updates the underlying list of elements.
     */
    @Override
    public V put(final K key, final V value) {
        final ListIterator<T> lit = elements.listIterator();
        while (lit.hasNext()) {
            final T element = lit.next();
            final Entry<K, V> entry = converter.toEntry(element);
            if (entry.getKey().equals(key)) {
                elements.set(lit.nextIndex() - 1, converter.toElement(key, value));
                return entry.getValue();
            }
        }
        lit.add(converter.toElement(key, value));
        return null;
    }

    /**
     * {@inheritDoc}
     *
     * @return a specialized set view of the entries over the element list
     */
    @Override
    @Nonnull
    public ListMapSet entrySet() {
        return new ListMapSet();
    }

    /**
     * Exposes the underlying list of elements backing the map view.
     *
     * @return the element list, never missing
     */
    @Nonnull
    public List<T> elements() {
        return elements;
    }

    /**
     * Converts between elements and entry objects.
     *
     * @param <T> the type of list elements
     * @param <K> the type of map keys
     * @param <V> the type of map values
     */
    public static interface Converter<T, K, V> {
        /**
         * Converts to an element instance from a map <var>key</var>-<var>value</var> pair.
         *
         * @param key the map key, never missing
         * @param value the map value, optional
         *
         * @return the corresponding list element, never missing
         */
        @Nonnull
        T toElement(@Nonnull final K key, final V value);

        /**
         * Converts to a map entry instance from a list <var>element</var>.
         *
         * @param element the list element, never missing
         *
         * @return the corresponding map entry, never missing
         */
        @Nonnull
        Entry<K, V> toEntry(@Nonnull final T element);
    }

    /**
     * Backing set for {@link ListMap} exposed as a separate type so callers may access a {@link
     * #iterator() list iterator} and a list iterator {@link #iterator(int) offset by an index}.
     */
    public final class ListMapSet
            extends AbstractSet<Entry<K, V>> {
        /**
         * {@inheritDoc}
         *
         * @see java.util.List#listIterator()
         */
        @Override
        @Nonnull
        public ListMapIterator iterator() {
            return new ListMapIterator();
        }

        @Override
        public int size() {
            return elements.size();
        }

        @Override
        public boolean add(final Entry<K, V> entry) {
            return elements.add(converter.toElement(entry.getKey(), entry.getValue()));
        }

        /** @see List#listIterator(int) */
        @Nonnull
        public ListMapIterator iterator(final int index) {
            return new ListMapIterator(index);
        }

    }

    /**
     * Entry set iterator for {@link ListMapSet} exposed as a separate type.
     *
     * @see ListMapSet#iterator() list iterator
     * @see ListMapSet#iterator(int) offset by an index
     */
    public final class ListMapIterator
            implements ListIterator<Entry<K, V>> {
        private final ListIterator<T> it;

        public ListMapIterator() {
            it = elements.listIterator();
        }

        public ListMapIterator(final int index) {
            it = elements.listIterator(index);
        }

        @Override
        public boolean hasNext() {
            return it.hasNext();
        }

        @Override
        public Entry<K, V> next() {
            return new ListMapEntry(it.next());
        }

        @Override
        public boolean hasPrevious() {
            return it.hasPrevious();
        }

        @Override
        public Entry<K, V> previous() {
            return new ListMapEntry(it.previous());
        }

        @Override
        public int nextIndex() {
            return it.nextIndex();
        }

        @Override
        public int previousIndex() {
            return it.previousIndex();
        }

        @Override
        public void remove() {
            it.remove();
        }

        @Override
        public void set(final Entry<K, V> entry) {
            it.set(converter.toElement(entry.getKey(), entry.getValue()));
        }

        @Override
        public void add(final Entry<K, V> entry) {
            it.add(converter.toElement(entry.getKey(), entry.getValue()));
        }

        private class ListMapEntry
                implements Entry<K, V> {
            private final T element;

            ListMapEntry(final T element) {
                this.element = element;
            }

            @Override
            public K getKey() {
                return converter.toEntry(element).getKey();
            }

            @Override
            public V getValue() {
                return converter.toEntry(element).getValue();
            }

            @Override
            public V setValue(final V value) {
                final Entry<K, V> oldEntry = converter.toEntry(element);
                elements.set(nextIndex() - 1, converter.toElement(oldEntry.getKey(), value));
                return oldEntry.getValue();
            }
        }
    }
}

Monday, March 11, 2013

More Bob: Sat ci sat bene

More of the indisputable Uncle Bob, The Frenzied Panic of Rushing:

James and I had just ordered Gin and Tonics from the airport bar and returned to the waiting area, when we saw a group of the TWers frantically working on their laptops. We asked someone what was going on, and were told that they were racing to see who could implement Langton's Ant quickest.

James and I looked at each other, nodded, opened a laptop, and quietly began to pair-program. We followed our disciplines, wrote test-first, kept the code clean, got it working after a few minutes, and then closed the laptop. We looked up, and the young twenty-thirty-something TWers were still expending copious energies in a fit of frantic coding. The frenzied intensity of their effort, driven by the panic of potential loss, was evident in their gestures and expressions. We, the two fifty-something programmers, smiled, clinked our Gin and Tonics, and sat back to enjoy the show being put on by all those young turks who never even guessed that they had been calmly bested by two of their seniors who had the experience and wisdom to know that the only way to go fast is to go well.

I am happy I joined ThoughtWorks after the story took place lest I have been one of the frantic foragers.

Thursday, March 07, 2013

The JUnit forest

Edmund Kirwan damns the ubiquitous JUnit in JUnit's evolving structure. What's the problem? — galloping interdependencies. As a bonus, a spiffy graphical timeline of JUnit's structure.

Wednesday, March 06, 2013

Skipping TDD

Uncle Bob, as usual, is a pleasure to read, to wit, The Pragmatics of TDD. Highlights:

  • I don't write tests for getters and setters. Doing so is usually foolish. Those getters and setters will be indirectly tested by the tests of the other methods; so there's no point in testing them directly.
  • I don't write tests for member variables. They too will be tested indirectly.
  • I don't write tests for one line functions or functions that are obviously trivial. Again, they'll be tested indirectly.
  • I don't write tests for GUIs. GUIs require fiddling. You have to nudge them into place by changing a font size here, an RGB value there, an XY position here, a field width there. Doing that test-first is silly and wasteful.
  • In general I don't write tests for any code that I have to "fiddle" into place by trial and error. But I separate that "fiddling" code from code that I am surer of and can write tests for.
  • A few months ago I wrote an entire 100 line program without any tests. (GASP!)
  • I usually don't write tests for frameworks, databases, web-servers, or other third-party software that is supposed to work. I mock these things out, and test my code, not theirs.

Tuesday, March 05, 2013

Plugging a leaky ship

Plumbr's Nikita Salnikov-Tarnovski writes Hunting down memory leaks: a case study. Quoting:

A week ago I was asked to fix a problematic webapp suffering from memory leaks. How hard can it be, I thought – considering that I have both seen and fixed hundreds of leaks over the past two years or so.

But this one proved to be a challenge. 12 hours later I had discovered no less than five leaks in the application and had managed to fix four of them. I figured it would be an experience worth sharing. For the impatient ones – all in all I found leaks from

Of course, there's more.

Monday, February 25, 2013

TCP traceroute

Peteris Krumins posts on TCP Traceroute, a nifty trick to "see" through firewalls. His pudding:

# traceroute -T -p 80 www.microsoft.com
traceroute to www.microsoft.com (65.55.57.27), 30 hops max, 60 byte packets
 1  50.57.125.2 (50.57.125.2)  0.540 ms  0.629 ms  0.709 ms
 2  core1-aggr701a-3.ord1.rackspace.net (184.106.126.50)  0.486 ms  0.604 ms  0.691 ms
 3  corea.ord1.rackspace.net (184.106.126.128)  0.511 ms corea.ord1.rackspace.net (184.106.126.124)  0.564 ms  0.810 ms
 4  bbr1.ord1.rackspace.net (184.106.126.147)  1.339 ms  1.310 ms bbr1.ord1.rackspace.net (184.106.126.145)  1.307 ms
 5  chi-8075.msn.net (206.223.119.27)  3.619 ms  2.560 ms  2.528 ms
 6  * 204.152.140.35 (204.152.140.35)  3.640 ms *
 7  ge-7-0-0-0.co1-64c-1a.ntwk.msn.net (207.46.40.94)  52.523 ms xe-0-2-0-0.ch1-96c-2b.ntwk.msn.net (207.46.46.49)  3.825 ms xe-1-2-0-0.ch1-96c-2b.ntwk.msn.net (207.46.46.53)  3.355 ms
 8  xe-0-1-0-0.co1-96c-1a.ntwk.msn.net (207.46.33.177)  61.042 ms  61.032 ms  60.457 ms
 9  * * xe-5-2-0-0.co1-96c-1b.ntwk.msn.net (207.46.40.165)  100.069 ms
10  65.55.57.27 (65.55.57.27)  53.868 ms  53.038 ms  52.097 ms

Friday, February 15, 2013

On being the bearer of bad news

Elena Yatzeck writes 6 Steps To Success With Your Executive Boss, or "On being the bearer of bad news" (my title), another good post on the timeless theme, Do The Right Thing.

Thursday, February 07, 2013

The Java enum factory

This comes up in interviews with intermediate programmers (and some seniors), the "enum factory" in Java:

public final class Money {
    // Infrequent that new currencies created,
    // old currencies never vanish, just fade away
    public enum Currency {
        USD, PHP; // And many more

        // Additional fields as needed, e.g., locale

        public Money print(final BigDecimal amount) {
            return new Money(this, amount);
        }
    }

    private final Currency currency;
    private final BigDecimal amount;

    private Money(final Currency currency, final BigDecimal amount) {
        this.currency = currency;
        this.amount = amount;
    }

    // Appropriate methods
}

And elsewhere with static imports:

Money pocket = asList(USD.print(ONE), PHP.print(TEN));

The key observations:

  • Amounts of money are intimately attached to a particular currency
  • Currency has few qualities not attached to money
  • Ok to add more currencies through code
  • Code reads better with factory than without

Tuesday, February 05, 2013

Mana on leadership

Mana of Geek Diva posts on leadership. Key point (emphasis mine):

When I see people who I consider good leaders, be they of any age, gender, height or ethnicity, I see one thing that they do very well. They make decisions quickly. Quickly does not mean recklessly. It just means that they make a decision knowing that if it isn't right then they can correct at a later time.

Java 8 lambdas and all that

Lovely post from Anton Arhipov at JRebel, Java 8: The First Taste of Lambdas. Kudos for javap decompilation for an "under the hood" explanation. His conclusion:

We can definitely say that lambdas and the accompanied features (defender methods, collections library improvements) will have a great impact on Java very soon. The syntax is quite nice and once developers realize that these features provide value to their productivity, we will see a lot of code that leverages these features.

It was quite interesting for me to see what lambdas are compiled to and I was very happy when I saw the total utilization of the invokedynamic instruction in action without any anonymous inner classes involved at all.

Wednesday, January 16, 2013

Superior teams

Venkatesh Krishnamurthy writes The 4 Ingredients of Building Hyper-Productive Teams. What he means are agile teams, quoting Jeff Sutherland:

We define Hyper-Productivity here at 400% higher velocity than average waterfall team velocity with correspondingly higher quality. The best Scrum teams in the world average 75% gains over the velocity of waterfall teams with much higher quality, customer satisfaction, and developer experience.

Agile for me, though worded awkwardly: how do the best scrum teams compare to the best waterfall teams?

Friday, January 11, 2013

GapList, an alternative to ArrayList and LinkedList

Thomas Mauch presents GapList, an alternative to ArrayList and LinkedList in Java, with the best performance behaviors of both: fast random reads and compact, cache-friendly storage like ArrayList, fast random updates like LinkedList.

Monday, January 07, 2013

Marz on Lambda Architecture

Nathan Marz writes on Big Data Lambda Architecture, promoting his upcoming book from Manning, Big Data. It's a comprehensible take on big data architectures. (See Chapter 1 [pdf].)

In a nutshell:

Computing arbitrary functions on an arbitrary dataset in real time is a daunting problem. There is no single tool that provides a complete solution. Instead, you have to use a variety of tools and techniques to build a complete Big Data system.

The lambda architecture solves the problem of computing arbitrary functions on arbitrary data in real time by decomposing the problem into three layers: the batch layer, the serving layer, and the speed layer.

Thursday, December 13, 2012

Passing of an era: No more i386

Thus spake Linus Torvalds:

Pull "Nuke 386-DX/SX support" from Ingo Molnar:
 "This tree removes ancient-386-CPUs support and thus zaps quite a bit
  of complexity:

    24 files changed, 56 insertions(+), 425 deletions(-)

  ... which complexity has plagued us with extra work whenever we wanted
  to change SMP primitives, for years.

  Unfortunately there's a nostalgic cost: your old original 386 DX33
  system from early 1991 won't be able to boot modern Linux kernels
  anymore.  Sniff."

I'm not sentimental.  Good riddance.

Tuesday, December 04, 2012

Guicing your jars, part 2

In Guicing your jars I mention using Java's service loader feature to automate finding modules for Guice, along these lines:

public class MetaInfServicesModule
        extends AbstractModule {
    @Override
    protected void configure() {
        for (final Module module : ServiceLoader.load(Module.class))
            install(module);
    }
}

What I did not mention was the secret sauce, Kohsuke's MetaInfServices annotation processor which creates the META-INF/services file for you:

@MetaInfServices(Module.class)
public class FooModule
        extends AbstractModule {
    @Override
    protected void configure() {
        // Awesome foo-ness here
    }
}

Now using your Maven runtime dependencies to control the classpath, Guice automatically installs your modules:

Guice.createInjector(new MetaInfServicesModule());

(Dependencies between modules is beyond the scope of this post!)