make for a simple shell project, to run tests before
committing. The check was trivial:
SHELL = bash test: @./run-tests t | grep 'Summary: 16 PASSED, 0 FAILED, 0 ERRORED' >/dev/null
This has the nice quality of Silence is Golden: say nothing when all is good. However, it loses the quality of Complain on Failure: it simply fails without saying why.
A better solution, preserving both qualities:
SHELL = bash test: @diff --color=auto \ <(./run-tests t | grep 'Summary: .* PASSED, .* FAILED, .* ERRORED') \ <(echo 'Summary: 16 PASSED, 0 FAILED, 0 ERRORED')
It still says nothing when all is good, but now shows on failure how many tests went awry. Bonus: color for programmers who like that sort of thing.
bash? I'm taking advantage of Process
Substitution. Essentially the command outputs inside the
subshells are turned into special kinds of files, and
likes to compare files. Ksh and Zsh also support process substitution, so
I'm going with the most widely available option.
Why are my arguments to
ordered like that? In usual testing language, I'm comparing "actual" vs
"expected", and more commonly you'll see programmers list "expected"
diff by default colors the left-hand input in RED, and the right-hand input in
GREEN. On failure, it makes more sense to
color "actual" in red and "expected" in green. Example output on failure:
$ make 1c1 < Summary: 17 PASSED, 1 FAILED, 0 ERRORED --- > Summary: 19 PASSED, 0 FAILED, 0 ERRORED make: *** [Makefile:4: test] Error 1