Push early, push often, push on green

(This post follows up on Frequent commits, a post about git command line help for TDD. I assume you are already following good TDD practices. Also, please recall git normally requires pulling before pushing if your repo is behind on commits from the remote.)

Prologue

I'm chatting with a colleague who is new in her role as an Agile Coach (she comes from a PM background). We were talking about ways to organize a team's story board (card wall), and turned to Desk Checks and when they fit into a story's life.

An interesting remark by her: a team had agreed to share work (push) only after the desk check was successful; that is, they did not push code until a story was almost done: work lay fallow on individuals' machines for potentially days at a stretch.

I was surprised. Why would they wait days to push—what did they do about merge conflicts, complex refactorings, integration failures in the pipeline, et al?

Lecture

Entropy

To me this was clearly a smell. Martin Fowler specifically addresses this in Everyone Commits To the Mainline Every Day, and I would go further: Push commits at the earliest responsible moment. This is opposite the advice for refactoring, or especially emergent design, where the "Rule of 3" and last responsible moment cautions waiting for more information before committing to a course of action.

And you can see why early pushes differ from the other two: waiting will not get you more information. On the contrary, waiting will only increase the entropy of the code base! Commits lie fallow in the local repo, increasing the size of potential merge conflicts for others.

	Benefit from more information?	Principle	Entropy from waiting
Early push	None available	Earliest responsible moment	Rises from fallow commits
Refactoring	Get more code examples	Rule of 3	Falls after refactoring
Architecture decision	Learn more about system	Last responsible moment	Falls if responsible

(The "information" in the case of pushes are the pulled commits themselves.)

I've definitely experienced this firsthand, when I'd eventually discard my local commits after waiting too long, and letting them grow too much in a different direction from how the rest of the team progressed. Waste!

Complexity

Consider this work cycle:

1. Local commit
2. Fetch commits from remote
3. Merge, if needed
4. Local commit again, if needed
5. Push commits to remote

I've grouped these to emphasize what is local to you (the first four) and what is global to your team (the last one).

Considered only locally, you minimize entropy with frequent pulls for yourself, and likewise for your teammates individually, so you can catch merge conflicts early and resolve them when they are small. But considered globally, you need frequent pushes so those local pulls have only small changes in them. The longer you wait to push, the more work for those who pull.

Early push

You	Rest of team	Work for others
You commit
You push
	They pull	Less complexity of merge (1 commit)
You commit
You push
	They pull	Less complexity of merge (1 commit)

Each single push can be treated on it's own. There are two opportunities for merge conflict, but each is a small amount of work.

Late push^[1]

You	Rest of team	Work for others
You commit
	They pull	No changes to merge
You commit
You push
	They pull	Greater complexity of merge (2 commits)

In each scenario, there are two commits for others to contend with. The larger, combined push has a greater opportunity for merge conflict, and a greater chance for a large amount of work, because of the combined interactions of the two commits.

And as teams work in parallel, there are more opportunities for merge conflicts.

Push early, push often, push on green

From the above discussion, the safest course is to push early rather than wait as commits pile up locally. But when to push—what is the "earliest responsible moment"?

If your codebase is well-tested, and answer presents itself: Push when tests are green and changes alter the complexity.

The goal is to avoid complex commit interactions that lead to merge conflicts. Tests are the safety net. Further, if all else fails and a commit is bad, it is easy to throw away the last commit until things are right again: only a small amount of work is lost, not days worth.

Understanding what kind of changes alter complexity takes skill: skills improve with experience and coaching. The cost of early pushes is low, and the occassional penalty of late pushes high, so this would be a good topic for a "team norms" ("dev practices") discussion.

For example, the team might agree that changes to comments are not in themselves worth a push. At the other end, your refactorings which impact more than one source file almost certainly should be pushed early: discover their impact on others before you add more refactorings.

A good work cycle:

1. Pull
2. Build, run tests
3. Edit sources
4. Build, run tests
5. Commit
6. Pull and push

After a preliminary sanity check (#1 and #2), get in the the cycle of #3 to #6.

Epilogue

I checked with other teams: it is minority practice to wait until a successful desk check to push changes. That's a relief. Hopefully this practice can be made more rare.

One rational reason—itself a smell—is when tests take too long to run frequently. When I design a pipeline, I recommend breaking out "unit" tests from "integration" tests for this exact reason: even when integration tests run long, the initial CI stage with just unit tests should be fast enough to give quick feedback on frequent pushes, and encourage Push early, Push often, Push on (local) green.

Further reading

• Seth Robertson posted a wonderful list of git best practices.^[2]

Footnotes

^[1] The simple statement, "a greater chance for a large amount of work", has rather complex reasoning behind it, beyond the scope of this post.

For example, any particular commit can be viewed as applying an exponent to the overall complexity of a program. A neutral change (say, correcting a typo in a comment) has the exponent 1: it does not change the overall complexity; a positive change (say, removing code duplication) has an exponent between 0 and 1: it lowers the overall complexity; a negative change (say, adding a new dependency) has an exponent greater than 1: it raises the overall complexity.

Consider then that these complexity changes are not simple numbers, but distributions ("odds"), and change with time ("bitrot"), and involve more than the code (people or requirments changes).

^[2] In Seth's post, do not confuse "publish once" with "wait to push": it means "don't publish the same commit twice" (which does sometimes happen accidentally, even for experts, from merging or rebasing).

Update

Sure enough, right after posting I read an interesting discussion on the value of the statistical mean (average) relevant to the discussion on two commits taken separately or together.

Essentially, even when the merge conflict work averages out over time for pushing two commits separately versus pushing them together, the outliers for pushing them together is significantly worse than for pushing them separately because of interactions and complexity.

Frequent commits

Pair posting with guest Sarah Krueger!

A source control pattern for TDD

At work we recently revisited our commit practices. One issue spotted: we didn't commit often enough. To address we adopted the source control pattern in this post. There are lots of benefits; the one that mattered to me most: No more throwing the baby out with the bathwater, that is, no more two hour coding sessions only to start again and lose the good with the bad.

So we worked out this command-line pattern using single unit-of-work commits (without git rebase -i!):

# TDD cycle: edit code, run tests, rather-rinse-repeat until green
$ git pull --rebase --autostash && run tests && git commit -a --amend --no-edit

# Simple unit-of-work commit, push, begin TDD cycle again
$ git commit --amend && git push && git commit --allow-empty -m WIP

What is this?

1. Start with a pull. This ensures you are always current, and find conflicts as soon as possible.
2. Run your full tests. This depends on your project, for example, mvn verify or rake. If some tests are slow, split them out, and add a full test run before pushing.
3. Amend your work to the current commit. This gives you a safe fallback known to pass tests. Worst case you might lose some recent work, but not hours worth. (Hint: run tests often.)
4. When ready to push, update the commit message to the final message for public push.
5. Push. Share. Make the world better.
6. Restart the TDD cycle with an empty commit using a message that makes sense to you, for example "WIP" (work in progress); the message should be obvious not to push. Key: the TDD cycle command line only amends commits, so you need a first, empty commit to amend against.

Why?

They key feature of this source control pattern is: Always commit after reaching green on tests; never commit without testing. When tests fail, the commit fails (the && is short-circuit logical and).

In the math sense, this pattern makes testing and committing one-to-one and onto. Since TDD requires frequent running of tests, this means frequent commits when those tests pass. To avoid a long series of tiny commits when pushing, amend to collect a unit of work.

Bootstrapping

The TDD cycle depends on an initial, empty commit. The first time using this source control pattern:

# Do this after the most recent commit, before any edits
$ git commit --allow-empty -m WIP

Adding files

This pattern, though very useful, does not address new files. You do need to run git add with new files to include them in the commit. Automatically adding new files can be dangerous if gitignore isn't set up right.

It depends on your style

The exact command line depends on your style. You could include a script to run before tests, or before commit (though the latter might be better done with a git pre-commit hook). You might prefer merge pulls instead of rebase pulls. If your editor runs from the command line you might toss $EDITOR at the front of the TDD cycle.

The command lines assume git, but this source control pattern works with any system that supports similar functionality.

Fine-grained commits

An example of style choice. If you prefer fine-grained commits to unit-of-work single commits (depending on your taste or project; they're both good practice):

# TDD cycle: edit code, run tests, rather-rinse-repeat until green
$ git pull --rebase --autostash && run tests && git commit -a

# Fine-grained commits, push, begin TDD cycle again
$ git push

Improving your life

No matter your exact command line, it can be made friendlier for you. Yes, shell history can story your long chain of commands. What if they vary slightly between programmers sharing a project, or what if there is a common standard approach? Extend git. Let's call our example subcommand "tdd". Save this in a file named git-tdd in your $PATH:

#!/bin/sh
set -e
case $1 in
    test ) git pull --rebase --autostash && run tests && git commit -a --amend --no-edit ;;
    accept ) git commit --amend && git push && git commit --allow-empty -m WIP ;;
esac

Now your command line becomes:

$ git tdd test  # Repeat until unit of work is ready
$ git tdd accept

The source is in GitHub.

Updated:

An editing error left out the Why? section when initially posted.

Remember to autostash.

A follow up post: Push early, push often, push on green.

Gee wilickers

I have a common command-line pattern I grew tired of typing. An example

$ mvn verify | tee verify.out

I use this pattern so often as I want to both watch the build on screen, and have a save file to grep when something goes wrong. Sometimes I also find myself telling the computer:

$ mvn verify | tee verify.out
$ mv verify.out verify-old.out
$ $EDITOR pom.xml
$ mvn verify | tee verify.out
$ diff verify-old.out verify.out

I want to see what changed in my build. But ... too much typing! So I automated with gee, a mashup of git and tee. You can think of it as source control for <STDOUT>.

Now I can type:

$ gee -o verify.out mvn verify
$ gee -g git diff HEAD^  # Did build output change?

How does it work? gee keeps a git repository in a hidden directory (.gee), committing program output there using tee. It follows simple conventions for file name and commit message (changeable with flags), and accepts <STDIN>:

$ mvn verify | gee -o verify.out -m 'After foobar edit'

Automating github

I got tired of downloading my own scripts from Github when working among multiple projects. So I automated it, of course. The bitsh project reuses a test script from the shell project, and now the Makefile for bitsh is simply:

SHELL=/bin/bash

test:
	@[ -t 1 ] && flags=-c ; \
	./test-bitsh.sh -i -- $$flags t

When run-tests is updated in Github, bitsh automatically picks up the changes. And I learned the value of ETag.

By the way, why "bitsh"? I hunted around for project names combining "git" and "bash" and found most of them already taken. Beggars can't be choosers.

UPDATE: I found the <TAB> character got munged by Blogger to a single space. This is unfortunate as you cannot copy out a valid Makefile. One solution is to put in an HTML tab entity.

Overwhelmed: git flow, github, maven and central

I decided to publish code to Maven central providing extended version of this blog's sample code. And I have become fascinated with gitflow. So I am stirring the pot, mixing git, github, maven and the central repo. Wish me luck. I'll post a "howto" when it works well enough.

Moving a git repository to the subdirectory of another repository

How do I move a git repository to be the subdirectory of another repository? is a question answered umpteen times, and all answers vary somewhat. For my needs, this suffices:

#!/bin/bash

case $# in
  3 ) old_repo=$1 new_repo=$2 new_name=$3 ;;
  * ) echo "$0: old_repo new_repo new_name" ; exit 2 ;;
esac

set -e

trap 'rm -rf $old_work $new_work' EXIT
old_work=$(mktemp -d)
new_work=$(mktemp -d)
git clone $old_repo $old_work
git clone $new_repo $new_work

cd $old_work
# If new_name contains commas, edit the sed command accordingly
git filter-branch --index-filter 'git ls-files -s | sed "s,\t\"*,&'$new_name'/," | GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info && mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE' HEAD

cd $new_work
git remote add $new_name $old_work
git pull $new_name master
git push

More git: moving modules between repos

Not the coolest merge, EVER! but much more grokable, Greg Bayer provides Moving Files from one Git Repository to Another, Preserving History.

The problem: I want to move repo-A/module1 to repo-B/module1 while preserving history. Bayer to the rescue. Thanks, Greg!

Beautiful illustration of feature branching with git

Vincent Driessen posted a while back a rather beautiful illustration of feature branching in git. I wish I had not missed it when first posted (2010). And just what I needed — he includes git command line instructions for executing the model. Posts like this are what make the Internet so helpful in my day-to-day work life.