tag:blogger.com,1999:blog-56383722024-03-07T14:04:32.507-06:00binkley's BLOGI'm a Principal at ThoughtWorks, classical composer, and physics amateur.Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.comBlogger814125tag:blogger.com,1999:blog-5638372.post-22039285352927362222019-02-18T05:17:00.002-06:002019-02-18T05:30:02.853-06:00Java date-time timezone formats<p>Java has excellent date-time formatting with the arrival of
<code>java.time</code> (JSR310) in Java 8. I point out that release as it
came with a usable, safe API. (Let us never speak of <code>Calendar</code>
again).</p>
<p>However, I never recall how to format timezone. There are so many options,
and it is easy to get is "almost right", but not exactly right.</p>
<h3>Problem</h3>
<p>I'd like to append a "Z" character on the end of a UTC timestamp. OK, let's
look at
<a href="https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/time/format/DateTimeFormatter.html#patterns"
title="Patterns for Formatting and Parsing">the options</a>, showing
only those for timezone/offset:</p>
<table>
<thead>
<tr>
<th scope="col">Symbol</th>
<th scope="col">Meaning</th>
<th scope="col">Presentation</th>
<th scope="col">Examples</th>
</tr>
</thead>
<tbody>
<tr>
<th scope="row">V</th>
<td>time-zone ID</td>
<td>zone-id</td>
<td>America/Los_Angeles; Z; -08:30</td>
</tr>
<tr>
<th scope="row">v</th>
<td>generic time-zone name</td>
<td>zone-name</td>
<td>Pacific Time; PT</td>
</tr>
<tr>
<th scope="row">z</th>
<td>time-zone name</td>
<td>zone-name</td>
<td>Pacific Standard Time; PST</td>
</tr>
<tr>
<th scope="row">O</th>
<td>localized zone-offset</td>
<td>offset-O</td>
<td>GMT+8; GMT+08:00; UTC-08:00</td>
</tr>
<tr>
<th scope="row">X</th>
<td>zone-offset 'Z' for zero</td>
<td>offset-X</td>
<td>Z; -08; -0830; -08:30; -083015; -08:30:15</td>
</tr>
<tr>
<th scope="row">x</th>
<td>zone-offset</td>
<td>offset-x</td>
<td>+0000; -08; -0830; -08:30; -083015; -08:30:15</td>
</tr>
<tr>
<th scope="row">Z</th>
<td>zone-offset</td>
<td>offset-Z</td>
<td>+0000; -0800; -08:00</td>
</tr>
</tbody>
</table>
<p>One thing to be wary of: formatting characters can be doubled, tripled, or
quadrupled, and it changes the result. Further, some characters have
special rules on repeating (<em>eg</em>, "VV", and "O" vs "OOOO").</p>
<p>The best way to understand what to use is to try them all:</p>
<pre class="code">
final var when = ZonedDateTime.of(
LocalDate.of(2011, 2, 3),
LocalTime.of(14, 5, 6, 7_000_000),
ZoneId.of("UTC"))
.toInstant();
for (final String tzFormat
: List.of("VV", "v", "z", "zz", "zzz", "zzzz", "O", "OOOO", "X", "XX", "XXX",
"XXXX", "x", "xx", "xxx", "xxxx", "Z", "ZZ", "ZZZ", "ZZZZ")) {
System.out.println(
tzFormat + " - " + DateTimeFormatter
.ofPattern("yyyy-MM-dd'T'HH:mm:ss" + tzFormat)
.withZone(ZoneId.of("UTC"))
.format(when));
}
</pre>
<p>Producing:</p>
<pre>
VV - 2011-02-03T14:05:06UTC
v - 2011-02-03T14:05:06UTC
z - 2011-02-03T14:05:06UTC
zz - 2011-02-03T14:05:06UTC
zzz - 2011-02-03T14:05:06UTC
zzzz - 2011-02-03T14:05:06Coordinated Universal Time
O - 2011-02-03T14:05:06GMT
OOOO - 2011-02-03T14:05:06GMT
X - 2011-02-03T14:05:06Z
XX - 2011-02-03T14:05:06Z
XXX - 2011-02-03T14:05:06Z
XXXX - 2011-02-03T14:05:06Z
x - 2011-02-03T14:05:06+00
xx - 2011-02-03T14:05:06+0000
xxx - 2011-02-03T14:05:06+00:00
xxxx - 2011-02-03T14:05:06+0000
Z - 2011-02-03T14:05:06+0000
ZZ - 2011-02-03T14:05:06+0000
ZZZ - 2011-02-03T14:05:06+0000
ZZZZ - 2011-02-03T14:05:06GMT
</pre>
<p>What an exciting list! "zzzz" is rather wordy, and it's unclear what "ZZZZ"
is doing. Actually, the whole list is even more iteresting for timezones
<em>other</em> than UTC.</p>
<h3>Solution</h3>
<p>Since the goal is to append a "Z", the simplest choice is:
<code>yyyy-MM-dd'T'HH:mm:ssX</code>.</p>
<h3>Addendum</h3>
<p>Why didn't I just use <code>DateTimeFormatter.ISO_INSTANT</code>, which is
documented to produce the "Z"? I want a timestamp that is to only <em>seconds-precision</em>,
and the format for "ISO_INSTANT" includes milliseconds.</p>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-28759734204158344772019-01-18T05:53:00.002-06:002019-01-18T06:05:22.026-06:00Spring REST testing<p>After too much Internet searching, I was unable to find an easy
solution to repeated duplication in my Spring MockMVC tests of REST
controller endpoints. For years now, the endpoints we write have
typically sent or received JSON. This is what I mean:</p>
<pre class="code">
mockMvc.perform(post("/some/endpoint")
.contentType(APPLICATION_JSON_UTF8)
.accept(APPLICATION_JSON_UTF8)
.content(someRequestJson))
.andExpect(status().isCreated())
.andExpect(header().string(CONTENT_TYPE, APPLICATION_JSON_UTF8_VALUE))
.andExpect(header().string(LOCATION, "/some/endpoint/name-or-id"))
.andExpect(content().json(someResponseJson));
</pre>
<p>All the repeated "APPLICATION_JSON_UTF8"s, in every controller test!</p>
<p>If there is an existing Spring testing solution, I'd love to hear about
it. Rather than wait, I wrote up a small extension of
<code>@WebMvcTest</code> to default these values.</p>
<p>First, an annotation for Spring to use in setting up a
<code>MockMvc</code> (javadoc elided):</p>
<pre class="code">
@Documented
@Import(JsonMockMvcConfiguration.class)
@Retention(RUNTIME)
@Target(TYPE)
@WebMvcTest
public @interface JsonWebMvcTest {
@AliasFor(annotation = WebMvcTest.class)
String[] properties() default {};
@AliasFor(annotation = WebMvcTest.class)
Class<?>[] value() default {};
@AliasFor(annotation = WebMvcTest.class)
Class<?>[] controllers() default {};
@AliasFor(annotation = WebMvcTest.class)
boolean useDefaultFilters() default true;
@AliasFor(annotation = WebMvcTest.class)
ComponentScan.Filter[] includeFilters() default {};
@AliasFor(annotation = WebMvcTest.class)
ComponentScan.Filter[] excludeFilters() default {};
@AliasFor(annotation = WebMvcTest.class)
Class<?>[] excludeAutoConfiguration() default {};
}
</pre>
<p>Note it is a near exact lookalike of <code>@WebMvcTest</code> (minus
the deprecated parameter). The important bits are:</p>
<ol>
<li>Marking this annotation with <code>@WebMvcTest</code>, a kind of
extension through composition.
</li>
<li>Adding <code>@Import</code> to bind custom configuration to this
annotation.
</li>
<li>Tying the same-named annotation parameters to
<code>@WebMvcTest</code>, so this annotation is a drop-in
replacement of that one.
</li>
</ol>
<p>Next a configuration class, imported by the annotation, to customize
<code>MockMvc</code>:</p>
<pre class="code">
@Configuration
public class JsonMockMvcConfiguration {
@Bean
@Primary
public MockMvc jsonMockMvc(final WebApplicationContext ctx) {
return webAppContextSetup(ctx)
.defaultRequest(post("/")
.contentType(APPLICATION_JSON_UTF8)
.accept(APPLICATION_JSON_UTF8_VALUE))
.alwaysExpect(header().string(
CONTENT_TYPE, APPLICATION_JSON_UTF8_VALUE))
.build();
}
}
</pre>
<p>Some points about this class:</p>
<ul>
<li><code>@Primary</code> is not necessary for Spring, but helped
IntelliJ — perhaps I got lucky with Spring without
<code>@Primary</code>, and IntelliJ highlighted a real problem.
</li>
<li>It took quite a while to get <code>defaultRequest(...)</code>
working. I was unable to (re)implement the relevant interfaces, and
eventually found that passing <em>any</em>
<code>MockHttpServletRequestBuilder</code> sufficed. Spring
"merges" (overlays) the actual request builder from the test over
this default, replacing POST and <code>"/"</code> with whichever
HTTP method and path the test uses (<em>eg</em>,
<code>GET "/bob"</code>). Only the header customization is used.
</li>
</ul>
<p>Example usage:</p>
<pre class="code">
@JsonWebMvcTest(SomeController.class)
class SomeControllerTest {
@Autowired
private MockMvc jsonMockMvc;
@Test
void shouldCheckSomething()
throws Exception {
jsonMockMvc.perform(post("/some/endpoint")
.content(someRequestJson))
.andExpect(status().isCreated())
.andExpect(header()
.string(LOCATION, "/some/endpoint/new-name"))
.andExpect(content().json(someResponseJson));
}
}
</pre>
<p>See the <a href="https://github.com/binkley/basilisk">Basilisk</a> project
for source code and sample usage. (Basilisk is a demonstration project
for my team illustrating Spring usage and conventions.)</p>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-68319957576878563002019-01-09T06:09:00.002-06:002019-01-09T06:09:59.063-06:00Magic Bus returns<p>During my first stint at ThoughtWorks, I paired with Gregor Hohpe on
implementing messaging patterns while he worked with Bobby Woolf on
<a href="https://www.amazon.com/o/asin/0321200683"
title="Enterprise Integration Patterns"><cite>Enterprise
Integration Patterns</cite></a> (EIP). To this day, this remains
one of my favorite technical books. In conversation I was always struck by
Gregor's meticulous "napkin diagrams" as he illustrated the point he
was making.</p>
<p>One output from that pairing was to experiment with using messaging
patterns <em>within a single program</em>, not just between programs.
So I wrote the "Magic Bus" library in Java, using reflection, to connect
publishing and subscribing components within a web services backend.</p>
<p>While working a new project, I find myself diagramming one of our
backend services using EIP's notations for messaging patterns. And I
recalled "Magic Bus".</p>
<p>I thought I had long ago lost the source code, but found some
JVM <code>.class</code> files in a forgotten directory. IntelliJ to
the rescue! Using <a
href="https://github.com/JetBrains/intellij-community/tree/master/plugins/java-decompiler/engine"
title="intellij-community/plugins/java-decompiler/engine at master · JetBrains/intellij-community">JetBrain's
excellent Fernflower decompiler</a>, I recovered a later stage of
"Magic Bus" after I had converted it to typesafer generics and dropped
reflection.</p>
<p>That code is now <a href="https://github.com/binkley/magic-bus"
title="binkley/magic-bus: Bringing back the magic">in
public GitHub</a>, brought up to Java 11, and cleaned up.</p>
<p>If I recall correctly, I originally dropped "Magic Bus"
after Guava's Event Bus came along. What makes "Magic Bus" different from
<a href="https://github.com/google/guava/wiki/EventBusExplained"
title="EventBusExplained · google/guava Wiki">Event
Bus</a>? Not too much, actually. The main feature in "Magic Bus"
lacking in Event Bus is subscribing to message handler exceptions: in
Guava one instead <a
href="https://google.github.io/guava/releases/23.0/api/docs/com/google/common/eventbus/EventBus.html#EventBus-com.google.common.eventbus.SubscriberExceptionHandler-"
title="EventBus (Guava: Google Core Libraries for Java 23.0 API)">registers
a global callback</a> to handle exceptions.</p>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-79762483453811213722019-01-07T19:07:00.003-06:002019-01-07T19:08:56.832-06:00Hard-won JDK offset knowedge<p>It took far more research time than I expected. The goal:
<em>Output an <code>OffsetDateTime</code> with offset for Zulu
(OTC) timezone as <code>+00:00</code></em>.</p>
<p>I have a project where a 3<sup>rd</sup>-party JSON exchange expected
timestamps in the format <code>01-02-03T04:05:06+00:00</code>. We're
using Jackson in a Java project. All the default configuration I
could find, and trying all the "knobs" on Jackson I could find, led
to: <code>01-02-03T04:05:06Z</code>. Interesting, as any non-0 offset
for timezone produced: <code>01-02-03T04:05:06+07:00</code> rather
than a timezone abbreviation: Zero offset is special.</p>
<p>Finally, circling back to the JDK javadocs <em>yet again</em>, I
spotted what I had overlooked many times before:</p>
<blockquote>
<b>Offset X and x</b>: This formats the offset based on the number of
pattern letters. One letter outputs just the hour, such as '+01',
unless the minute is non-zero in which case the minute is also output,
such as '+0130'. Two letters outputs the hour and minute, without a
colon, such as '+0130'. Three letters outputs the hour and minute, with
a colon, such as '+01:30'. Four letters outputs the hour and minute and
optional second, without a colon, such as '+013015'. Five letters
outputs the hour and minute and optional second, with a colon, such as
'+01:30:15'. Six or more letters throws
<code>IllegalArgumentException</code>. Pattern letter 'X' (upper case)
will output 'Z' when the offset to be output would be zero, whereas
pattern letter 'x' (lower case) will output '+00', '+0000', or '+00:00'.
</blockquote>
<p>The key is to use <em>lowercase</em> 'x' in the format specification.
So my problem with Jackson became:</p>
<pre class="code">
@JsonFormat(pattern = "yyyy-MM-dd'T'HH:mm:ssxxx")
private final OffsetDateTime someOffsetDateTime;
</pre>
<p>And the result is the desired, <code>01-02-03T04:05:06+00:00</code>.</p>
<p>Now I can return to more interesting problems.</p>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-43141214615511927932018-11-09T08:38:00.002-06:002018-11-09T08:38:38.102-06:00What are exceptions?<article>
<section id="main">
<h3>What are exceptions?</h3>
<p>Essentially, exceptions are a form of structured, non-local
<code>goto</code> with <a
href="http://wiki.c2.com/?UnwindingTheStack"
title="Unwinding The Stack">stack unwinding</a>.
"Structured" here means <a
href="http://wiki.c2.com/?StructuredProgramming"
title="Structured Programming">"higher level than the
machine"</a> (a matter of taste/opinion), and "non-local"
means "beyond a single function/method".</p>
<p>What this means is that you can replace code like (in "C"):</p>
<pre class="code">int bottom()
{
int things_go_wrong = -1; // For illustration
if (things_go_wrong) goto error;
return 0;
error:
return -1;
}
int
middle()
{
if (-1 == bottom()) goto error;
return 0;
error:
return -1;
}
void
top()
{
if (-1 == middle()) {
handle_failure();
}
}</pre>
<p>With code like (in Java):</p>
<pre class="code">public class A {
void botton() {
boolean thingsGoWrong = true; // For illustration
if (thingsGoWrong) throw new ThingsWentWrong("So wrong!");
}
void middle() {
bottom();
}
void top() {
try {
middle();
} catch (ThingsWentWrong e) {
handleFailure();
}
}
}</pre>
<p>(<a href="https://stackoverflow.com/a/16493261/250924"
title="How to implement a try-catch block in scheme?">An
example in Scheme</a>.)</p>
<p>"Unwinding" here means the compiler or runtime treats
intermediate calls (the stack) the same as if returning
normally (for example, the stack pointer is moved back;
and in a language like C++, destructors are executed), and
program execution resumes in the <code>catch</code> block.</p>
<p>It is "structured" in the sense that it is not the same as a
direct <code>goto</code> to the resume point. This is not
possible in standard "C" or C++ (or in Java), which only
suppport local labels within functions. The equivalent in "C"
and C++ is to use <a
href="https://en.wikipedia.org/wiki/Setjmp.h"
title="setjmp.h"><code>setjmp</code> and
<code>longjmp</code></a> for a non-local goto, and forget
all about deallocating memory or calling destructors. (As odd as
this sounds, it is needed for <a
href="http://www.madore.org/~david/computers/callcc.html"
title="A page about call/cc"><code>call-with-continuation</code></a>,
an important feature in
LISP-like languages).
</p>
</section>
<section id="takeaway">
<h3>Takeaway</h3>
<p>All human endeavors build on the work of those who came
previous. <em>Exceptions</em> are no exception. They are the
result of <a
href="https://en.wikipedia.org/wiki/Exception_handling"
title="Exception handling">1960s and 1970s programmers</a>
wanting an easier way to deal with
<em>exceptional conditions</em>. This is not completely identical
with "errors", which may sometimes be better represented with
a simple boolean <code>true</code>/<code>false</code> return.
Like any sharp instrument, <a
href="https://dalibornasevic.com/posts/52-don-t-overuse-exceptions"
title="Don't Overuse Exceptions">do not abuse it</a>.</p>
</section>
</article>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-39074495973124491632018-10-19T10:36:00.000-05:002018-10-19T12:50:00.784-05:00Avoid JIRA<p>I filed this issue with Atlassian about JIRA:</p> <pre>You make it incredibly difficult to report issues about JIRA itself:
* I cannot use Markdown in the editor. You are the only tool I use which does
not support markdown. This is one of the top reasons I recommend against
using JIRA to clients Ex: quoting code with single backticks, or code blocks
with tripple backticks
* Finding the issue tracker for JIRA is a PITA. Even after finding it, when
creating a new issue, it offers a dialog/link that takes me back to the
beginning
* The web pages for a team project does not have any easy way to report JIRA
issues to Atlassian
* How do I find out the version of JIRA software in use when I report problems?
* Reporting to you, you _require__ a component, and severity. Which component
should I pick? I don't know how your product is architected, so I guessed
at one. And "affected version"? Heck if I know. Really, you can't
provide a link on a cloud JIRA board which fills this in automatically?
* In a dropdown for picking what Atlassian product to report against, the text
describing them is cut off. So when it says "XXX (includi)" I have no idea
what it is including</pre> <p>Only .. I didn't. Their publicly accessible issue tracker does not let me file issues.</p>Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-13793110190803562142018-09-23T12:30:00.002-05:002018-09-24T05:07:49.297-05:00Removing Joda from Spring Boot<!DOCTYPE html>
<html lang=en>
<head>
<meta charset=utf-8>
<title>Automated Acceptance Criteria</title>
<link href='https://fonts.googleapis.com/css?family=Noto%20Sans'
rel='stylesheet' type='text/css'>
<style>
body {
font-family: 'Noto Sans', sans-serif;
}
aside {
width: 25%;
float: right;
font-size: smaller;
}
</style>
</head>
<body>
<header>
<h1>Removing Joda from Spring Boot</h1>
</header>
<article>
<section id="the-problem">
<h2>The problem</h2>
<p>We recently migrated a medium-sized Java project to Spring Boot 2
from version 1. One of the challenges was migrating to the JDK
date-time library from Joda. It turns out that Spring Boot 2 has
excellent native support for JDK date-times, as does Jackson (JSON)
and Hibernate (database), the default technologies offered by Spring
Boot 2 for these features.</p>
<p>The migration itself went smoothly, which is unsurprising given the
fantastic work of <a href="https://blog.joda.org/"
title="Stephen Colebourne's blog">Stephen
Colebourne</a> in designing JDK date-time support based on
his authorship of Joda.</p>
<aside>
<h3>Gradle configurations</h3>
<p>For our purposes, we are treating
<a href="https://docs.gradle.org/current/dsl/org.gradle.api.artifacts.Configuration.html"
title="Configuration - Gradle DSL Version 4.10.2">a
Gradle configuration</a> as a <em>classpath</em>. More on
<a href="https://docs.gradle.org/current/userguide/java_library_plugin.html"
title="The Java Library Plugin">recent changes to
configurations in Gradle</a>.</p>
</aside>
<p>So we looked at disabling Joda completely in our Gradle build.
The most concise approach we found was:</p>
<pre class="code">configurations {
compile.exclude group: 'joda-time'
}</pre>
<p>This removed Joda completely from configurations (classpaths) related
to Java. However, this had unintended side effects:</p>
<ul>
<li>During tests, we needed Joda in the runtime classpath for a
3<sup>rd</sup>-party library, <a
href="https://wiki.shibboleth.net/confluence/display/OpenSAML/Home"
title="Home - OpenSAML 2 - Shibboleth Wiki">OpenSAML</a>
</li>
<li>During boot run (running the app), we needed Joda in the
classpath for another 3<sup>rd</sup>-party library, <a
href="http://springfox.github.io/springfox/"
title="SpringFox by springfox">SpringFox</a>
</li>
</ul>
<p>We easily found a workaround for SpringFox, but not for OpenSAML.</p>
<p>(If you're curious, yes, we do intent to migrate from OpenSAML 2
(desupported in 2016) to OpenSAML 3; however, we would like
<code>spring-security-saml2-core</code> support first.)</p>
</section>
<section id="a-solution-in-progress">
<h2>A solution in progress</h2>
<p>The exclusion in the <code>compile</code> configuration does
exactly what we need: <em>Joda disappears!</em> But what to do
about SpringFox and OpenSAML?</p>
<p>For the Spring Boot runtime classpath, there is another concise
solution, though finding it was rather troublesome, and it is
not well-documented by Pivotal or in Stack Overflow.</p>
<p>First, we setup another classpath of our own making named
<var>bootRuntime</var>:</p>
<pre class="code">configurations {
// Other parts of "configurations", including the Joda exclusion from above
bootRuntime // Synthetic configuration for deps needed *only* to launch app
}</pre>
<p>Then we added Joda to that synthetic classpath relying on Spring Boot
plugin's definition for the version of Joda to use:</p>
<pre class="code">dependencies {
// Other parts of "dependencies"
bootRuntime 'joda-time:joda-time'
}</pre>
<aside>
<h3>Finding the partial solution</h3>
<p>For the nitty-gritty on <code>bootJar</code> and
<code>bootInfo</code>, see <a
href="https://docs.spring.io/spring-boot/docs/current/gradle-plugin/api/org/springframework/boot/gradle/tasks/bundling/BootJar.html#bootInf-org.gradle.api.Action-"
title="BootJar (Spring Boot Gradle Plugin 2.0.5.RELEASE API)">the
javadocs</a>. Further searching led to <a
href="https://github.com/spring-projects/spring-boot/issues/13000"
title="Provide access to the CopySpec for BOOT-INF of BootJar as Gradle's War task does for WEB-INF">this
recently closed issue</a> and <a
href="https://docs.gradle.org/current/userguide/war_plugin.html"
title="The War Plugin">this vague hint</a>.</p>
</aside>
<p>Lastly, we taught Spring Boot to include this synthetic classpath
when launching our app (this was the trickiest part):</p>
<pre class="code">bootJar {
bootInf {
from configurations.bootRuntime
into 'lib'
}
}
bootRun {
classpath += configurations.bootRuntime
}</pre>
<p>This adds Joda to the runtime classpath for both the single "fat
jar" built by Spring (<code>bootJar</code>), and when launching
the app on the command line with gradle (<code>bootRun</code>).</p>
<p>Unless you are a heavy Gradle user, <code>from ... into ...</code>
syntax may be unfamiliar: this copies the jars in the synthetic
configuration into the fat jar at the location Boot expects to find
them. The "'lib'" is literally a directory location within the
jar. Useful magic, but a bit obtuse. The outcome:</p>
<pre class="code">$ jar tf build/libs/the-project.jar | grep joda-time
BOOT-INF/lib/joda-time-2.9.9.jar</pre>
<p>As a matter of fact, Joda is the very last file in the boot jar,
a suggestion that it was added by our <code>bootInf</code> section
after the Boot plugin built the jar.</p>
<p>(Our workaround is intentionally small. If we're unable to make
it work, we'll switch to brute-force library exclusions in our
dependencies lists. The goal is to prevent accidental import
from Joda, for example, of <code>LocalDate</code>.)</p>
</section>
<section id="remaining-work">
<h2>Remaining work</h2>
<p>For running the boot app, this solution is great: it is small,
readable, easy to maintain, and <em>it works</em>. However, for
tests which exercise our user authentication with OpenSAML, it
fails. Joda is not in the test classpath, and we cannot use or
mock OpenSAML methods which use Joda types.</p>
<p>Barring another magical solution like <code>bootRuntime</code>,
we'll fall back on manually excluding Joda from each dependency,
and adding it back in to the test classpath. A pity given how
pithy the solution is with exclusion from the
<code>compile</code> configuration.</p>
</section>
</article>
</body>
</html>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-52391292867152415532018-07-29T13:07:00.003-05:002018-07-30T05:51:39.344-05:00Kotlin JPA patterns<header>
<h1>Kotlin JPA pattern</h1>
</header>
<article>
<section id="The problem">
<h2>The problem</h2>
<p>Kotlin does many wonderful things for you. For example, <a
href="https://kotlinlang.org/docs/reference/data-classes.html"
title="Data Classes">the <code>data</code> classes</a>
create helpful constructors, and automatically implement
<code>equals</code> and <code>hashCode</code> in a reasonable
way.</p>
<p>Similarly, JPA works magic—expecially in <a
href="https://spring.io/guides/gs/accessing-data-jpa/"
title="Accessing Data with JPA">the context of Spring
Data</a>.</p>
<p>So how do I test that my Kotlin entity is correctly annotated
for JPA? The simplest thing would be a "round trip" test:
create an entity, save it to a database, read it back, and
confirm the object has the same values. Let's start with a
simple entity, and the simplest possible test:</p>
<pre class="code">@Entity
data class Greeting(
val content: String,
@Id @GeneratedValue
val: Int id = 0)</pre>
<pre class="code">@DataJpaTest
@ExtendWith(SpringExtension::class)
internal class GreetingRepositoryIT(
@Autowired val repository: GreetingRepository,
@Autowired val entityManager: EntityManager) {
@DisplayName("WHEN saving a greeting properly annotated")
@Nested
inner class Roundtrip {
@Test
fun `THEN is can be read back`() {
val greeting = Greeting("Hello, world!")
repository.saveAndFlush(greeting.copy())
entityManager.clear()
assertThat(repository.findOne(Example.of(greeting)).get())
.isEqualTo(greeting)
}
}
}</pre>
<p>Some things to note:</p>
<ol>
<li>To ensure we truly read from the database, and not the
entity manager's in-memory cache, flush the object and
clear the cache.
</li>
<li>As saving also updates the entity's <var>id</var> field,
save a copy, so our original is untouched.
</li>
<li>Be careful to use <code>saveAndFlush</code> on the Spring
repository, rather than <code>entityManager.flush()</code>,
which requires a transaction, and would add unneeded
complexity to the test.
</li>
</ol>
<p>But this test fails! Why?</p>
<p>The unsaved entity (remember, we made a copy to keep the
original pristine) does not have a value for <var>id</var>,
and the entity read back does. Hence, the automatically
generated <code>equals</code> method says the two objects
differ because of <var>id</var> (<code>null</code> in the
original <i>vs</i> some value from the database).</p>
<p>Further, <a
href="https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#query-by-example"
title="Query by Example">the Spring Data QBE</a> (QBE) search
for our entity includes <var>id</var> in the search criteria. Even
changing <code>equals</code> would not address this.</p>
<p>What to do?</p>
</section>
<section id="The solution">
<h2>The solution</h2>
<p>It turns out we need to address two issues:</p>
<ol>
<li>The generated <code>equals</code> takes <var>id</var> into
account, but we are only interested in the data values, not
the database administrivia.
</li>
<li>The test look up in the database includes the SQL
<var>id</var> column. Although we could try
<code>repository.getOne(saved.id)</code>, I'd prefer to
keep using QBE, if the code is reasonable.
</li>
</ol>
<p>To address <code>equals</code>, we can rely on <a
href="https://discuss.kotlinlang.org/t/ignoring-certain-properties-when-generating-equals-hashcode-etc/2715/3"
title="Ignoring certain properties when generating equals(), hashCode(), etc">
an interesting fact</a> about Kotlin data classes: <em>only
default constructor parameters are used, not properties in the
class body</em>, when generating <code>equals</code> and
<code>hashCode</code>. Hence, I write the entity like this,
and <code>equals</code> does not include <var>id</var>, while
JPA is stil happy as it relies on getter reflection:</p>
<pre class="code">@Entity
data class Greeting(
val content: String) {
@Id
@GeneratedValue
val id = 0
}</pre>
<p>To address the test, we can ask QBE to ignore <var>id</var> when
fetching our saved entity back from the database:</p>
<pre class="code">@DataJpaTest
@ExtendWith(SpringExtension::class)
internal class GreetingRepositoryIT(
@Autowired val repository: GreetingRepository,
@Autowired val entityManager: EntityManager) {
@DisplayName("WHEN saving a greeting properly annotated")
@Nested
inner class Roundtrip {
@Test
fun `THEN is can be read back`() {
val greeting = Greeting("Hello, world!")
repository.saveAndFlush(greeting.copy())
entityManager.clear()
val matcher = ExampleMatcher.matching()
.withIgnoreNullValues()
.withIgnorePaths("id")
val example = Example.of(greeting, matcher);
assertThat(repository.findOne(example).get()).isEqualTo(greeting)
}
}
}</pre>
<p>In a larger database, I'd look into providing an
<code>entity.asExample()</code> to avoid duplicating
<code>ExampleMatcher</code> in each test.</p>
</section>
<section id="Java approach">
<h2>Java approach</h2>
<p>The closest to Kotlin's data classes for JPA entities is Lombok's
<code>@Data</code> annotation, together with
<code>@EqualsAndHashCode(exclude = "id")</code> and
<code>@Builder(toBuilder = true)</code>, however the
expressiveness is lower, and clutter higher.</p>
<p>The test would
be largely the same <i>modulo</i> language, replacing
<code>greeting.copy()</code> with
<code>greeting.toBuilder().build()</code>. Alternatively,
rather than a QBE matcher, one could write
<code>greeting.toBuilder().id(null).build()</code>.</p>
<p>This last fact leads to an alternative with Kotlin: include
<var>id</var> in the data class' default constructor, and in
the test compare the QBE result as
<code>findOne(example).get().copy(id = null)</code> without
a matcher.</p>
</section>
<section id="Conclusion">
<h2>Conclusion</h2>
<p>What Kotlin JPA patterns have you discovered?</p>
</section>
</article>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-39469489043773550902018-05-01T10:39:00.001-05:002018-05-02T14:46:02.684-05:00JaCoCo, Gradle, and exclusions<style>
article {
counter-reset: footnotes;
}
a[aria-describedby="footnote-label"] {
counter-increment: footnotes;
text-decoration: none;
color: inherit;
cursor: default;
outline: none;
}
a[aria-describedby="footnote-label"]::after {
content: counter(footnotes);
vertical-align: super;
font-size: smaller;
margin-left: 2px; /* TODO: Scalable? Percentage? And following... */
color: #0000EE;
text-decoration: underline;
cursor: pointer;
}
a[aria-describedby="footnote-label"]:focus::after {
outline: thin dotted;
outline-offset: 2px;
}
</style>
<article>
<section id="the-setup">
<h3>The setup</h3>
<p>My team is working on a Java server, as part of a larger project
project, using <a href="https://gradle.org/"
title="Gradle Build Tool">Gradle</a> to build and <a
href="http://www.eclemma.org/jacoco/"
title="JaCoCo Java Code Coverage Library">JaCoCo</a>
to measure testing code coverage. The build fails if coverage
drops below fixed limits (branch, instruction, and
line)—"verification" in JaCoCo-speak.</p>
<p>We follow the strategy of <a
href="http://robertgreiner.com/2012/01/continuous-code-improvement-using-ratcheting/"
title="Continuous Code Improvement Using Ratcheting"><em>The
Ratchet</em></a>: as dev pairs push commits into the project, code
coverage may not drop without group agreement, and if coverage
rises, the verification limits rise to match. This ensures we
have ever-rising coverage, and avoid new code which lacks
adequate testing<a aria-describedby="footnote-label"
href="#fn1"></a>.</p>
</section>
<section id="the-problem">
<h3>The problem</h3>
<p>At a work project, we're struggling to get JaCoCo to ignore some
new, configuration-only Java classes. These classes have no
"real" implementation code to test, are used to setup
communication with an external resource, yet are high
line-count (static configuration via code). So they drag down
our code coverage limits, and there is no effective way to
unit test them sensibly. (They are best tested as system
tests within our CI pipeline using live programs and remote
resources.)</p>
<p>JaCoCo has what seems at first blush a sensible way to exclude
these configuration classes from testing:</p>
<pre class="code">jacocoTestVerificationCoverage {
violationRules {
rule {
excludes ['hm.binkley.labs.saml.SomeConfig']
limit {
counter = 'LINE'
minimum = 0.90
}
}
}
}</pre>
<p>Unfortunately, this does nothing. There is no warning or error,
and coverage continues to include the whole code base.</p>
</section>
<section id="a-solution">
<h3>A solution</h3>
<p>After a lot of experimenting and StackOverflow research, <a
href="http://jivimberg.io/blog/2018/04/26/gradle-verify-coverage-with-exclusions/"
title="Jacoco & Gradle - How to Verify Coverage With Exclusions">this
answer from Juan Vimberg</a> worked exactly as we needed.
Following his approach:</p>
<pre class="code">final def excludedClasses = ['hm.binkley.labs.saml.SomeConfig']
jacocoTestVerificationCoverage {
violationRules {
rule {
limit {
counter = 'LINE'
minimum = 0.90
}
}
}
afterEvaluate {
classDirectories = files(classDirectories.files.collect {
fileTree(dir: it, excludes: excludedClasses.collect {
it.replace('.', '/') + '.class'
})
})
}
}</pre>
<p>The list of excluded classes is extracted so the same trick can
be used in the generated reports:</p>
<pre class="code">jacocoTestReport {
executionData test, databaseTest
reports {
html.enabled = true
xml.enabled = true
csv.enabled = false
}
afterEvaluate {
classDirectories = files(classDirectories.files.collect {
fileTree(dir: it, excludes: excludedClasses.collect {
it.replace('.', '/') + '.class'
})
})
}
}</pre>
<p>Something to consider: using wildcards
(<code>hm.binkley.labs.saml.*</code>) may take additional work.
</p>
</section>
<section id="why">
<h3>Why?</h3>
<p>Why does this work, and the "obvious" way does not?</p>
<p>JaCoCo has more than one notion of scoping. The clearest one is <a
href="http://www.eclemma.org/jacoco/trunk/doc/counters.html">the
<em>counters</em></a>: branches, classes, instructions, lines,
and methods.</p>
<p>Not as well documented is <a
href="http://www.eclemma.org/jacoco/trunk/doc/check-mojo.html">the
scope of checks</a>: bundles, classes, methods, packages, and
source files. These are <em>not</em> mix-and-match. For
example, exclusions apply to classes. Lyudmil Latinov has <a
href="https://automationrhapsody.com/automated-code-coverage-of-unit-tests-with-jacoco-and-maven/">the
best hints I've found</a> on how this works.</p>
</section>
</article>
<footer id="footnotes">
<h3 id="footnote-label">Footnotes</h3>
<ol>
<li id="fn1">Paul Friedman discusses why <a
href="http://www.agiletrailblazers.com/blog/high-unit-test-code-coverage-means-you-have-quality-tests-right-not-so-fast"
title="High Code Coverage Means You Have Quality Tests, Right? Wrong!"><em>high
coverage</em> is <strong>not</strong> <em>synonymous with
quality</em></a>.
</li>
</ol>
</footer>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-23374872864352274602018-03-31T08:41:00.001-05:002018-03-31T08:52:43.744-05:00Workaround for jenv on Cygwin<p>I'd like to use <code>jenv</code> on my Cygwin setup at home. Oracle has
moved to a 6-month release pace, and so I find myself dealing with
multiple Java major verions. However, my tool of choice, <code>jenv</code>,
does not play well with Cygwin.</p>
<p>(Note: There are <em>two</em> <code>jenv</code>s out there. I am talking
about <a href="http://www.jenv.be/">jenv.be</a>, not <a
href="http://jenv.io">jenv.io</a>. Apologies that neither does
HTTP<em>S</em> well.)</p>
<p>As a workaround, I wrote a straight-forward shell function to provide the
minimum I need: switch between versions in the current shell:</p>
<pre class="code"># Until jenv.be supports Cygwin
function set-java {
local -a java_v
local jdk v OPTIND
for jdk in /cygdrive/c/Program\ Files/Java/jdk*
do
jdk="${jdk/\/cygdrive\/c\/Program\ Files\/Java\/jdk/}"
v=${jdk#-}
v=${v#1.}
v=${v%%.*}
java_v[$v]=$jdk
done
local verbose=false
while getopts :hv opt
do
case $opt in
h ) cat <<EOH
Usage: $FUNCNAME [-hv] VERSION
Options:
-h Print help and exit
-v Verbose output
Arguments:
VERSION One of ${!java_v[@]}
EOH
return 0 ;;
v ) verbose=true ;;
* ) echo "Usage: $FUNCNAME [-hv] VERSION" >&2 ; return 2 ;;
esac
done
shift $((OPTIND - 1))
case $# in
1 ) ;;
* ) echo "Usage: $FUNCNAME [-hv] VERSION" >&2 ; return 2 ;;
esac
if ! [[ ${java_v[$1]+foo} ]]
then
echo "$FUNCNAME: No such Java version: $1. Try $FUNCNAME -h" >&2
return 2
fi
export JAVA_HOME='C:\Program Files\Java\jdk'${java_v[$1]}
for v in ${!java_v[@]}
do
case $v in
$1 ) ;;
* ) export PATH="${PATH//${java_v[$v]}/${java_v[$1]}}" ;;
esac
done
if $verbose
then
echo "$FUNCNAME: Updated JAVA_HOME and PATH for JDK to $v at $JAVA_HOME"
fi
}</pre>
<p>Try the <code>-h</code> flag (help).</p>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-67571122477089766752018-01-11T14:51:00.002-06:002018-01-15T07:52:10.301-06:00Automated acceptance criteria<!DOCTYPE html>
<html lang=en>
<head>
<meta charset=utf-8>
<title>Automated Acceptance Criteria</title>
<link href='https://fonts.googleapis.com/css?family=Noto%20Sans'
rel='stylesheet' type='text/css'>
<style>
body {
font-family: 'Noto Sans', sans-serif;
}
aside {
width: 25%;
font-size: smaller;
}
dt {
font-weight: bolder;
}
img {
box-shadow: none;
border: none;
}
.story-syntax {
font-weight: bolder;
}
.message {
font-style: oblique;
}
.trouble {
color: red !important;
}
.missing .ac:before {
content: '\1F636';
font-size: larger;
font-style: normal;
}
.missing {
color: gray;
}
.untested .ac:before {
content: '\1F637';
font-size: larger;
font-style: normal;
}
.untested {
}
.running .ac:before {
content: '\1F914';
font-size: larger;
font-style: normal;
}
.running {
color: blue;
font-style: oblique;
}
.passed .ac:before {
content: '\1F600';
font-size: larger;
font-style: normal;
}
.passed {
color: green;
}
.failed .ac:before {
content: '\1F61E';
font-size: larger;
font-style: normal;
}
.failed {
color: red;
}
.errored .ac:before {
content: '\1F644';
font-size: larger;
font-style: normal;
}
.errored {
color: orange;
}
.disabled .ac:before {
content: '\1F634';
font-size: larger;
font-style: normal;
}
.disabled {
color: gray;
text-decoration: line-through;
}
.flex-container {
display: flex;
flex-flow: row wrap;
justify-content: flex-start;
}
.flex-container form {
display: inline;
}
.header {
flex: 100%;
}
.flex-item {
margin: 1em;
}
</style>
</head>
<body>
<header>
<h1>Automated Acceptance Criteria</h1>
</header>
<article>
<aside style="float: right">
<img src="https://i.imgur.com/j8WZOOo.jpg"
alt="Thoughtful Ralph Wiggum" style="float: right; width: 10vw;
height: auto; padding-left: 1vw; border: none; box-shadow: none"/>
<blockquote>
A well written user story helps put me into a thoughtful frame
of mind.
</blockquote>
</aside>
<section id="dream-story-card">
<h2>I have a dream (Story Card)</h2>
<p>What is my dream story card? I don't mean: <em>What's the story
I'd most like to work on</em>! I mean: <em>What should a
virtual story card look like</em> (as opposed to card stock on
a wall)? This may be a trivial question. But for me the
<strong>user experience</strong> working with stories is very
important: I work with them daily, write them, discuss them,
work on them, accept them, etc. I want the feel of the card to be
<em>thoughtful</em>, like that fellow to the right.</p>
<p>And more than that. I am lazy, impatient, hubristic. The
<strong>acceptance criteria</strong>, I want them testable,
literally testable in that each has a matching test I can
execute. Given my laziness, I don't want to switch systems
and run tests; I'd like to execute acceptance criteria directly
from the story card.</p>
<p>So is there a system like that today? No. There are bits and
pieces though.</p>
</section>
<aside style="float:left">
<img src="https://i.imgur.com/ndiChvc.gif" alt="Scared Milhouse"
style="float:left; width: 10vw; height: auto; padding-right:
1vw; border: none; box-shadow: none"/>
<blockquote>
Difficult to use systems, waiting on tests, ugly interfaces,
counter-intuitive controls: these are a few of my least
favorite things.
</blockquote>
</aside>
<section id="nightmare-story-card">
<h2>Not all story card systems are equal</h2>
<p>Some story card systems are particularly awkward to read,
understand or use. Special demerits for:</p>
<ul>
<li>A hard-coded workflow that the team cannot be change to fit
how they work: the team is expected to fit the tool
</li>
<li>Workflow state transition buttons are nice, but not so
nice is unconfigurable labels, especially when the button
labels are misleading
</li>
<li>A hard-coded or limited hierarchy of stories, so if a
team uses <em>epics</em> or <em>features</em> or
<em>themes</em> or whatever to organize stories, and there
are more than one level to this, the team is out of luck
</li>
<li>Lack of quality RESTful support, in particular, no simple
identifier for story cards, so linking directly to cards is
opaque, useless or completely absent
</li>
</ul>
</section>
<section id="scenario">
<h2>A scenario</h2>
<p>Post-development testing on this team is a fairly ordinary role.
The developers say: a new web page is ready. Testers then
validate the same page features each time for each new page,
simple things:</p>
<ul>
<li>Can an account with security role <samp>X</samp> log in?</li>
<li>Can <samp>X</samp> submit the form? (Or not submit if
forbidden?)
</li>
<li>What form defaults appear for role <samp>X</samp>? Do they
reflect role <samp>X</samp>?
</li>
</ul>
<p>(Yes, I know — what about developer testing? Bear with
me.)</p>
<p>On it goes, the same work each time. Redundant, repetitive,
error-prone, fiddly. And worst of all—<em>boring</em>,
<strong>BORING</strong>! This is traditional, manual "user
testing" at its worst.</p>
<aside style="float: right">
<img src="https://i.imgur.com/uJGSibd.png" alt="Empowered Lisa"
style="float: right; width: 10vw; height: auto; padding-left:
1vw; border: none; box-shadow: none"/>
<blockquote>
The QA is the <em>Advocate for Value</em>. Strive to make
them awesome!
</blockquote>
</aside>
<p><em>What's to be done?</em> <em>Can we fix this?</em></p>
<p>After all, the testing is valuable: nobody wants broken web pages;
everybody wants to log in. But the tester is valuable, too,
more valuable even: is this the <em>most</em> valuable testing
a human could do? Surely people are more clever, more
insightful than this. And what about all the other page
features not tested because of time spent on the basics?</p>
<p>Well, people are more clever than this.</p>
</section>
<aside style="float: left">
<img src="https://i.imgur.com/NzyAvNt.png"
alt="Groundskeeper Willie" style="float: left; width: 10vw;
height: auto; padding-right: 1vw; border: none; box-shadow: none"/>
<blockquote>
A great topic for <em>Retrospective</em>, as the team finds
means to overcoming whatever slows them down.
</blockquote>
</aside>
<section id="clearing-the-path">
<h2>Clearing the path</h2>
<p>What guides testing? If you're using nearly any form of modern user
story writing, this includes something like "Acceptance Criteria".
These are the gates that permit story development to be called
successful: the testers are gatekeepers <a
href="https://www.thoughtworks.com/insights/blog/how-we-eliminated-gatekeeper-quality">in
the manual testing world</a>. In the manual world these
criteria might be congregated into a single "Requirements
Document" or similar (think: <a
href="https://en.wikipedia.org/wiki/Big_Design_Up_Front">big,
upfront design</a>).</p>
<p>We can do better! Gatekeeper-style testing assumes a linear path
from requirements to implementation to testing, just as waterfall
considers these activities as distinct phases. But we know
agile approaches do better than waterfall in most cases. Why
should we build our teams to mirror waterfall? Of course the
answer is to structure teams to look agile, just as the team
itself practices agile values.</p>
<p>So how do we make <em>Acceptance Criteria</em> more agile?</p>
</section>
<section id="three-amigos">
<h2>Enter the Three Amigos</h2>
<p>In current agile practice, a story card is not a ready to play
until approved by the <em>The Three Amigos</em>: <strong>BA, Dev,
QA</strong>. Each plays their part, brings their
perspective, contributes to meeting team-agreed "Definition of
Ready".</p>
<aside style="float: right">
<img src="https://i.imgur.com/lFBEXHp.jpg"
alt="Carl, Homer, and Lenny" style="float: right; width:
15vw; height: auto; padding-left: 1vw; border: none;
box-shadow: none"/>
<blockquote>
BA, Dev, QA: Who is which? Let agilists try out any role,
or all the roles.
</blockquote>
</aside>
<p>A key component of playable cards are the
<em>Acceptance Criteria</em> — answering the question,
"What does success look like?" when a story is finished.</p>
<p>The perspectives include:</p>
<ul>
<li><strong>BA</strong>: Is the story told right? — What
is the way to describe the work?
</li>
<li><strong>Dev</strong>: Is the story the right size? —
What is the complexity of the work?
</li>
<li><strong>QA</strong>: Is it the right story to tell? —
What is the value of the work?
</li>
</ul>
</section>
<section id="acceptance-criteria">
<h2>What are Acceptance Criteria?</h2>
<p>But where does this simple testing come from? Any software delivery
process beyond "winging it" has some requirements</p>
<p>Well-written agile stories have Acceptance Criteria. What are
these? An Acceptance Criteria (AC) is a statement in a story that
a tester (QA) can use to validate that all or a portion of the
story is complete. The formulaic phrasing for ACs I like best
is:</p>
<blockquote>
<span class="story-syntax">GIVEN</span> <em>some state of the
world</em><br>
<span class="story-syntax">WHEN</span> <em>some change
happens</em><br>
<span class="story-syntax">THEN</span> <em>some observable
outcome results</em>
</blockquote>
<p>Generally for web applications this means when I do something with
a web page in the application, then the page changes in some
particular way or submits a request and the response page has
some particular quality or property (or the negative case that
is does not have that quality or property).</p>
<p>In some cases it is even simpler: just check that a particular web
address fully loads, for example, when testing login access to
pages.</p>
<img
src="https://martinfowler.com/bliki/images/testPyramid/test-pyramid.png"
alt="Martin Fowler's Test Pyramid"
style="float:right; width:35%; height:auto; border: none; box-shadow: none">
<h2>So what's the question?</h2>
<p>Wherever possible we want to automate. If something can be done for
us by a computing machine, we don't want to spend human time on
it. Humans can move on to more interesting, valuable work when
existing tasks can be automated.</p>
<p>Consider the Test Pyramid (<em>image right</em>): automating
lower-value tests focuses people on higher-value ones. You get
more human attention and insight on the kinds of tests which best
improve the value of software. You win more.</p>
</section>
<section id="the-story">
<h2>The Story</h2>
<p>This is a sample story card with a simplistic implementation of
AACs. Live buttons call back into the story system, to be
mapped to calls in the testing system. (Another
implementation might have the buttons directly call to the
testing system, avoiding an extra call into the story system
but showing details in the page source about the testing
system.)</p>
<blockquote>
<section>
<h3>Title</h3>
<h4></h4>
<h3>Narrative</h3>
<p><span class="story-syntax">AS A</span> AAC author<br>
<span class="story-syntax">I WANT</span> a mock executable
story<br>
<span class="story-syntax">SO THAT</span> others can see
the value</p>
</section>
<section>
<h3>Details</h3>
<p>No actual criteria were validated in the execution of
these tests. This is only a mock.</p>
</section>
<section class="flex-container">
<h3 class="header">
<form action="/test-all">
<button>Test All</button>
</form>
Acceptance criteria
</h3>
<div class="header"><strong>Summary:</strong>
<span class="missing">1 missing</span>,
<span class="untested">1 untested</span>,
<span class="running">1 running</span>,
<span class="passed">1 passed</span>,
<span class="failed">1 failed</span>,
<span class="errored">1 errored</span>,
<span class="disabled">1 disabled</span></div>
<div class="flex-item missing">
<div>
<form
action="/create-test/ThisProduct/ThisStory/AC1?given=magical+thinking&when=in+Missingland&then=there's+no+test"
method="PUT">
<button><em>Create</em></button>
</form>
<span class="ac">
<span class="story-syntax">GIVEN</span> magical thinking<br>
<span class="story-syntax">WHEN</span> in Missingland<br>
<span class="story-syntax">THEN</span> there's no test
</span>
</div>
<div class="message trouble">No test (yet) - <em>create
one!</em></div>
</div>
<div class="flex-item untested">
<div>
<form
action="/run-test/ThisProduct/ThisStory/AC2">
<button>Test</button>
</form>
<span class="ac">
<span class="story-syntax">GIVEN</span> magical thinking<br>
<span class="story-syntax">WHEN</span> in Newland<br>
<span class="story-syntax">THEN</span> nothing has happened yet
</span>
</div>
<div class="message trouble">Test never run - <em>be the
first!</em></div>
</div>
<div class="flex-item running">
<div>
<form
action="/tests-in-progress/ThisProduct/ThisStory/AC3"
method="DELETE">
<button>Cancel</button>
</form>
<span class="ac">
<span class="story-syntax">GIVEN</span> magical thinking<br>
<span class="story-syntax">WHEN</span> in Fastland<br>
<span class="story-syntax">THEN</span> tests run quickly
</span>
</div>
<div class="message">15% done (3s)</div>
</div>
<div class="flex-item passed">
<div>
<form
action="/run-test/ThisProduct/ThisStory/AC4">
<button>Test</button>
</form>
<span class="ac">
<span class="story-syntax">GIVEN</span> magical thinking<br>
<span class="story-syntax">WHEN</span> in Happyland<br>
<span class="story-syntax">THEN</span> Unicorns
</span>
</div>
</div>
<div class="flex-item failed">
<div>
<form
action="/run-test/ThisProduct/ThisStory/AC5">
<button>Test</button>
</form>
<span class="ac">
<span class="story-syntax">GIVEN</span> magical thinking<br>
<span class="story-syntax">WHEN</span> in Sadland<br>
<span class="story-syntax">THEN</span> there be Dragons
</span>
</div>
<div class="message trouble">Expected:
<em>Dragons</em>, got: <em>Puppies</em></div>
</div>
<div class="flex-item errored">
<div>
<form
action="/run-test/ThisProduct/ThisStory/AC6">
<button>Test</button>
</form>
<span class="ac">
<span class="story-syntax">GIVEN</span> magical thinking<br>
<span class="story-syntax">WHEN</span> in Crazyland<br>
<span class="story-syntax">THEN</span> nothing works right
</span>
</div>
<div class="message trouble">Test timed out
after 90 seconds
</div>
</div>
<div class="flex-item disabled">
<div>
<button disabled>Disabled</button>
<span class="ac">
<span class="story-syntax">GIVEN</span> magical thinking<br>
<span class="story-syntax">WHEN</span> in Slowland<br>
<span class="story-syntax">THEN</span> tests are disabled
</span>
</div>
<div class="message trouble">@dev1 @qa2: BLOCKED on
widget spanner
</div>
</div>
</section>
</blockquote>
</section>
<section id="ac-states">
<h2>Acceptance Criteria states</h2>
<p>Every AC potentially has a message from the testing system
giving more detail on state. These are noted below.</p>
<dl>
<dt>Missing</dt>
<dd>
<p>This AC has no matching test in the testing system. Use
the <em>Create</em> button to create a new test. This does
not run the test.</p>
<p>The <em>message</em> is boilerplate to remind users to
create tests.</p>
</dd>
<dt>Untested</dt>
<dd>
<p>The AC has a matching test in the testing system, but the
test has never been run. Use the <em>Test</em> button to
run the test.</p>
<p>Typically there is no <em>message</em> for this state.</p>
</dd>
<dt>Running</dt>
<dd>
<p>The matching test for the AC is running in the testing
system. Use the <em>Cancel</em> button to stop the test,
or wait for it to complete.</p>
<p>The <em>message</em>, is supported by the testing
system, should give a notion of progress. See <a
href="http://farazdagi.com/blog/2014/rest-long-running-jobs/"><cite>REST
and long-running jobs</cite></a> for how to do this.
</p>
</dd>
<dt>Passed</dt>
<dd>
<p>The matching test for the AC passed last time it ran.
Use the <em>Test</em> button to run the test again.</p>
<p>Typically there is no <em>message</em> for this state.</p>
</dd>
<dt>Failed</dt>
<dd>
<p>The matching test for the AC failed last time it ran.
Use the <em>Test</em> button to run the test again.</p>
<p>The <em>message</em> must give a notion of why the test
failed.</p>
</dd>
<dt>Errored</dt>
<dd>
<p>The matching test for the AC errored last time it ran.
Use the <em>Test</em> button to run the test again.</p>
<p>The <em>message</em> must give a notion of why the test
errored.</p>
</dd>
<dt>Disabled</dt>
<dd>
<p>The matching test for the AC is disabled in the testing
system. Update the testing system to reenable.</p>
<p>The <em>message</em>, if supported, should give a
reason the test is disabled when available.</p>
</dd>
</dl>
</section>
<section id="problems">
<h2>Potential problems</h2>
<p>Nothing is free. Potential problems with AACs include:</p>
<dl>
<dt>Integrations</dt>
<dd>
<p>There are no existing
integrations along these lines. You need to build your
own against JIRA, Mingle, FitNesse, Cucumber, etc. Whether
the story system drives the interaction, or the test
system does, may depend on the exact combination. Best
would be if both systems can call the other.</p></dd>
<dt>Scaling</dt>
<dd>
<p>As more AACs run, the complete suite takes longer. For
example, adding 1 minute of AAC/story, and 5
stories/iteration, in a 12 iteration project takes 60
minutes to run. This is not specific to AACs but a
general problem with acceptance tests. It's still much
cheaper than the manual steps for each test, but
prohibitive for a developer to run the whole suite
locally.</p>
<p><em>Best practice</em> is for the 3 amigos to run only
the tests specific to a story before calling that story
<em>Ready to Accept</em>.</p></dd>
</dl>
</section>
<section id="update">
<h2>Update</h2>
<p>Until I published this post on Blogger, I really wasn't certain how
it would look on that platform. I manually tested the visuals from
Chrome with <a
href="https://github.com/binkley/html/blob/master/blog/automated-acceptance-criteria.html">the
local post</a>, and it looked good. After seeing it in
Blogger, however, the "aside" sections are laid out poorly,
overlapping the text. I won't relay the page: mistakes are the
best way to improve, and a subtext of this post is <a
href="http://modernagile.org/#guildingPrinciples">Experiment
& Learn Rapidly</a>. Public experiements are the most
faithful kind: no opportunity to fudge and pretend all was well on
the first try.</p>
</section>
</article>
</body>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-48432316828799574692018-01-09T13:33:00.000-06:002018-01-09T14:18:02.199-06:00Sproingk lives!<h3><em>Sproingk lives!</em></h3>
<p>After months of instability, I again have <a
href="https://circleci.com/gh/binkley/sproingk">a fully working
pipeline</a> and <a
href="http://sproingk-binkley.boxfuse.io:8080/swagger-ui.html">a live
web page</a> for Sproingk, <a
href="https://github.com/binkley/sproingk">my demo project</a>
combining the latest public betas in:</p>
<ul>
<li><a href="https://kotlinlang.org/">Kotlin</a> (1.2.10, soon 1.2.20)
</li>
<li><a href="https://projects.spring.io/spring-boot/">Spring Boot</a>
(2.0.0 M7, soon RC1)
</li>
<li><a href="http://springfox.github.io/springfox/">Springfox</a> (2.7.0)
</li>
<li>Hygienic build tooling
(<a href="https://github.com/shyiko/ktlint">ktlint</a> 0.14.0,
<a href="http://www.jacoco.org/jacoco/trunk/index.html">jacoco</a>
0.7.10 SNAPSHOT)
</li>
<li><a href="https://circleci.com/">Circle CI</a> (2.0)</li>
<li><a href="https://boxfuse.com/">Boxfuse</a> (1.30.*.*, updates
frequently)
</li>
</ul>
<p>If you're interested in any combination of these, please take a gander.</p>
<h3>About some of the items</h3>
<p>Kotlin (<em>It's not just for Android!</em>) is really where JVM
programming is headed. And it's fun.</p>
<p>Springfox gives you a lovely UI for your REST API. (In the demo, try
the "greeting controller".)</p>
<p>Boxfuse make minimal shrink-wrapped Linux images of your software, and
handles blue/green AWS deployments.</p>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-33637656907157335872018-01-03T19:48:00.002-06:002018-01-04T08:23:49.260-06:00A language stack for 2018<style>
article {
counter-reset: footnotes;
}
a[aria-describedby="footnote-label"] {
counter-increment: footnotes;
text-decoration: none;
color: inherit;
cursor: default;
outline: none;
}
a[aria-describedby="footnote-label"]::after {
content: counter(footnotes);
vertical-align: super;
font-size: smaller;
margin-left: 2px; /* TODO: Scalable? Percentage? And following... */
color: #0000EE;
text-decoration: underline;
cursor: pointer;
}
a[aria-describedby="footnote-label"]:focus::after {
outline: thin dotted;
outline-offset: 2px;
}
</style>
<article>
<section id="caution">
<p>(I talk about myself in the post more than usual. I'm expressing
opinions more than usual, rather than observations and advice.
<em>Caveat lector</em>. Also, this post is <em>link-rich</em>.)
</p>
</section>
<section id="the-stack">
<h3>The stack</h3>
<p>After reading Eric S. Raymond (<em>ESR</em>)'s posts on the
post-"C" world, I realized that I, too, live in that world. What
would my ideal language stack look like for 2018?</p>
<p>For context, here are <em>ESR</em>'s posts I have in mind:</p>
<ol>
<li><a href="http://esr.ibiblio.org/?p=7711"
title="The long goodbye to C"><cite>The long goodbye to
C</cite></a></li>
<li><a href="http://esr.ibiblio.org/?p=7724"
title="The big break in computer languages"><cite>The big
break in computer languages</cite></a></li>
<li><a href="http://esr.ibiblio.org/?p=7745"
title="Language engineering for great justice"><cite>Language
engineering for great justice</cite></a></li>
<li><a href="http://esr.ibiblio.org/?p=7804"
title="C, Python, Go, and the Generalized Greenspun Law"><cite>C,
Python, Go, and the Generalized Greenspun Law</cite></a></li>
</ol>
<p>Read them? Good. So this is the language stack I have in mind:</p>
<ul>
<li id="python"><a href="https://www.python.org/"
title="Python is a programming language that lets you work quickly and integrate systems more effectively">Python</a><a
aria-describedby="footnote-label"
href="#fn1"></a> — By default
</li>
<li id="kotlin-jvm"><a href="https://kotlinlang.org/"
title="Statically typed programming language for modern multiplatform applications">Kotlin</a>
(JVM)<a aria-describedby="footnote-label"
href="#fn2"></a>
— When you need it
</li>
<li id="go"><a href="https://golang.org/"
title="Go is an open source programming language that makes it easy to build simple, reliable, and efficient software">Go</a>
— When you must
</li>
</ul>
<p>Each of these languages hits a sweet spot, and displaces an earlier
language which was itself a sweet spot of its time:</p>
<ul>
<li>Bash and Perl<a aria-describedby="footnote-label"
href="#fn3"></a> →
<strong>Python</strong></li>
<li>Java → <strong>Python and Kotlin</strong></li>
<li>"C" and C++ → <strong>Kotlin and Go</strong></li>
</ul>
<p>An interesting general trend here: Not just replace a language with
a more modern equivalent, but also move programming further away
from the hardware. As <em>ESR</em> points out, Moore's law and
improving language engineering have raised the bar.</p>
<p>(<em>ESR</em>'s thinking has <a
href="http://www.catb.org/esr/writings/taoup/html/ch14s04.html"
title="The Art of Unix Programming, Chapter 14, Section 4, Language Evaluations">evolved
over time</a>, a sign of someone who has given deep and sustained
thought to the subject.)</p>
</section>
<section id="about-my-experience-with-these-languages">
<h3>About my experience with these languages</h3>
<h4>Python</h4>
<p>I have moderate experience in Python spread out since the
mid-90s. At that time, I was undecided between Python, Ruby
and Perl. Over my career I worked heavily in Perl (it paid
well, then), some in Ruby (mostly at ThoughtWorks), and
gravitated strongly to Python, especially after using it
at Macquarie commodities trading where it was central to their
business.</p>
<h4>Kotlin</h4>
<p>I've been a Kotlin fan since it was announced. It scratches itches
that Java persistently gives, and JetBrains is far more
pleasant a <a
href="https://en.wikipedia.org/wiki/Benevolent_dictator_for_life">"benevolent
dictator"</a> than Oracle: JetBrains continually sought input
and feedback from the community in designing the language, for
example. Java has been my primary language since the late 90s,
bread and butter in most projects. If JetBrains keeps Kotlin in
its current directions, I expect it to displace Java, and
deservedly so.</p>
<h4>Go</h4>
<p>This is where I am relying on the advice of others more than
personal experience. I started programming with "C" and LISP
(Emacs), and quickly became an expert (things being relative) in
C++. Among other C++ projects, I implemented for INSO (Microsoft
Word multilingual spell checker) a then new specification for
UNICODE/CJKV support in C++ (<a
href="http://www.cplusplus.com/reference/string/wstring/"><code>wstring</code></a>
and friends). I still love "C" but could do without C++. Go seems
to be the right way to head, especially with garbage collection.
</p>
<p>With <a
href="https://www.amazon.com/Lions-Commentary-Unix-John/dp/1573980137"
title="How I learned the kernel">Ken Thompson</a> and <a
href="https://www.amazon.com/Unix-Programming-Environment-Prentice-Hall-Software/dp/013937681X"
title="How I learned the command line">Rob Pike</a> behind
it, intelligent luminaries like <em>ESR</em> pitching for it, and
colleagues at ThoughtWorks excited to start new Go projects, it's
high time I make up this gap.</p>
</section>
<section id="what-makes-a-modern-language">
<h3>What makes a "modern" language?</h3>
<p>I'm looking for specific things in a "modern" language<a
aria-describedby="footnote-label" href="#fn4"></a>,
chiefly:</p>
<section id="good-community">
<h4>Good community</h4>
<p>Varying and strong:</p>
<ul>
<li><a href="https://www.python.org/community/">Python</a>
</li>
<li><a href="https://kotlinlang.org/community/">Kotlin</a>
</li>
<li><a href="https://golang.org/help/">Go</a></li>
</ul>
<p>You can explore the figures <a
href="https://www.tiobe.com/tiobe-index/"
title="TIOBE Index for November 2017">at TIOBE</a> and
<a
href="https://insights.stackoverflow.com/survey/2017#technology-programming-languages"
title="Stack Overflow Developer Survey 2017">StackOverflow</a>.
</p>
<p>Kotlin is the interesting case. Though low in the rankings,
because of 100% interoperability running on the JVM, it's easy
to <a
href="https://kotlinlang.org/docs/reference/java-interop.html"
title="Calling Java code from Kotlin">call Java from
Kotlin</a> and <a
href="https://kotlinlang.org/docs/reference/java-to-kotlin-interop.html"
title="Calling Kotlin from Java">call Kotlin from
Java</a>, so the whole Java ecosystem is available to
Kotlin natively.</p>
<p>Further, Google <a
href="https://developer.android.com/kotlin/"
title="Kotlin and Android">fully supports Kotlin on
Android</a> as a first-class language, so there is a wealth of
interoperability there. (The story for iOS is more nuanced,
with Kotlin/Native including that platform as a target but in
progress.)</p>
<p>Lastly, Kotlin/Native is bringing <a
href="https://github.com/JetBrains/kotlin-native/blob/master/LIBRARIES.md"
title="Kotlin/Native libraries">similar
interoperability</a> between Kotlin and Go.</p>
<p>(This is related to <a href="#rich-ecosystem">having a rich
ecosystem</a>.)</p>
</section>
<section id="garbage-collection">
<h4>Garbage collection</h4>
<p>(A <a href="http://www.toves.org/books/gc/"
title="Garbage collection">quick primer (2011)</a> on
garbage collection.)</p>
<p>Among <strong>Go</strong>'s advantages over "C" and C++ is <a
href="https://groups.google.com/d/msg/golang-nuts/KJiyv2mV2pU/wdBUH1mHCAAJ"
title="Why golang garbage-collector not implement Generational and Compact gc?">solid
garbage collection</a> out of the box<a
aria-describedby="footnote-label" href="#fn5"></a>,
though Boehm <a href="http://www.hboehm.info/gc/"
title="A garbage collector for C and C++">a
valiant effort</a>. It's only been since <a
href="https://www.seas.harvard.edu/courses/cs252/2016fa/16.pdf"
title="[PDF] Origins of Garbage Collection"><em>1959</em></a>.
No modern programmer—short of special cases—should
manually manage memory.</p>
<p><strong>Kotlin</strong> gets a head start here. It's built
on the JVM (when targeting that environment), which has
arguably the world's greatest GC (or at least most tested).
It definitely gives you <a
href="https://blog.takipi.com/garbage-collectors-serial-vs-parallel-vs-cms-vs-the-g1-and-whats-new-in-java-8/"
title="Garbage Collectors - Serial vs. Parallel vs. CMS vs. G1 (and what’s new in Java 8)">a
choice of garbage collectors</a>. There's too much to
say about GC on the JVM for this post except that it is
first-rate.</p>
<p><strong>Python</strong> has garbage collection. Though not
as strong as the JVM, it <a
href="https://www.python.org/dev/peps/pep-0556/"
title="PEP 556 -- Threaded garbage collection">continues
to improve</a>. It is unusual for GC to become a limiting
factor in a Python program; <a
href="https://engineering.instagram.com/dismissing-python-garbage-collection-at-instagram-4dca40b29172"
title="Dismissing Python Garbage Collection at Instagram">you
will know if it does</a>.</p>
<p>(If you are in "C" or C++, and want some of the benefits of
GC, do consider
<a href="http://www.hboehm.info/gc/">the Boehm-Demers
-Weiser garbage collector</a>. No, you won't get the
fullest benefits of a language built for GC, but you'll get
enough to help a lot, even if just for leak detection.)</p>
</section>
<section id="static-type-inference">
<h4>Static type inference</h4>
<p><strong>Kotlin</strong> really demonstrates where Java could
improve by leaps rather than <a
href="http://openjdk.java.net/jeps/286"
title="JEP 286: Local-Variable Type Inference">baby
steps</a>: automatic type inference. At least Java is
heading in the right direction. Don't tell the computer <em>how</em>
to run your program when it can figure this our for itself!
Keep your focus on <em>what</em> the program is for.</p>
<p>Interestingly, Kotlin and <strong>Go</strong> have strong
typing out of the box, but not Python. <strong>Python</strong>
has built-in support for typing, and <a
href="http://mypy-lang.org/"
title='Mypy is an experimental optional static type checker for Python that aims to combine the benefits of dynamic (or "duck") typing and static typing'>an
excellent optional implementation</a>, <code>mypy</code>,
however this is <em>type declaration</em> not <em>type
inference</em>, so it loses a bit there. And none of the
common alternatives (Ruby, Perl, PHP, etc.) have type
inference either. I'll need to check again in a few years, and
possibly update this choice.</p>
<p>(Paul Chiusana writes <a
href="https://pchiusano.github.io/2016-09-15/static-vs-dynamic.html"
title="The advantages of static typing, simply stated">on
the value</a> of static type checking.)</p>
</section>
<section id="rich-ecosystem">
<h4>Rich ecosystem</h4>
<p>Of the languages, <strong>Kotlin</strong> is a standout here.
Because it is a JVM language, it is 100% compatible with
Java, and can fully use the rich world of Java libraries
and frameworks. The main weakness for Kotlin is lack of
tooling: because more advanced tools may make assumptions
about bytecode, Kotlin's particular choices of emitted
bytecode sometimes confuse them.</p>
<p>(JetBrains has surveyed on <a
href="https://www.jetbrains.com/research/devecosystem-2017/"
title="The State of Developer Ecosystem in 2017">the state
of ecosystems</a> for programming languages, related to <a
href="#good-community">having a good community</a>.)</p>
<p>Close behind is <strong>Python</strong>, <a
href="https://docs.python.org/3/tutorial/stdlib.html#batteries-included"
title="Brief Tour of the Standard Library">"batteries
included"</a> and all, and it has a better organized and
documented standard library than Java. For some problem
domains, Python has a richer ecosystem, for example, <a
href="https://www.scipy.org/">SciPy</a> and <a
href="http://www.numpy.org/">NumPy</a> is the best
math environment available in any language. (Some
specialty languages like <a
href="https://www.mathworks.com/products/matlab.html">MATLAB</a>
deserve mention—an early employer of mine.) I may need
to reconsider my ranking Kotlin over Python here.</p>
<p><strong>Go</strong> is, frankly, too new to have developed an
equivalent ecosystem, and full-blown package management is
still <a
href="https://github.com/golang/go/wiki/PackageManagementTools"
title="PackageManagementTools · golang/go Wiki">a
work in progress</a>.</p>
</section>
<section id="consision-and-convenience">
<h4>Concision and convenience</h4>
<p>A common thread in recent language development is <em>lower
ceremony</em>: fewer punctuation marks and boilerplate; make
the machine do more work, you do less. Kotlin provides the
most obvious example compared to Java. Go is know for
cleanness and brevity. And Python ranks high here as well.</p>
<p>(Donnie Berkholz writes <a
href="http://redmonk.com/dberkholz/2013/03/25/programming-languages-ranked-by-expressiveness/"
title="Programming languages ranked by expressiveness">an
interesting post</a> on ranking language
<em>expressiveness</em>.)</p>
</section>
</section>
<section id="code-samples">
<h3>Code samples</h3>
<p>The classic "Hello, World!" showing three things:</p>
<ul>
<li>Writing a "main" callable from the command line</li>
<li>Using standard output to print to console</li>
<li>String formatting to build the output message</li>
</ul>
<p>This doesn't, of course, give a sense of how these languages in
their full spectrum, but does give a first taste.</p>
<section id="java-sample">
<h4>Java</h4>
<p>Java in a file named <code>MyStuff.java:</code></p>
<pre class="code">package my.stuff;
public final class MyStuff {
public static final String LANGUAGE = "Java";
public static void main(final String... args) {
System.out.println(String.format("Hello, World, from %s", language));
}
}</pre>
</section>
<section id="kotlin-sample">
<h4>Kotlin</h4>
<p>Kotlin in a file named <code>my-program.kt:</code></p>
<pre class="code">package my.stuff
const val LANGUAGE = "Kotlin"
fun main(args: Array<String>) = println("Hello, World, from $LANGUAGE")</pre>
</section>
<section id="go-sample">
<h4>Go</h4>
<p>But also compare Go to C++:</p>
<pre class="code">package main
import "fmt"
const Language = "Go"
func main() {
fmt.Println("Hello, World, from", Language)
}</pre>
</section>
<section id="cpp-sample">
<h4>C++</h4>
<p>And C++:</p>
<pre class="code">#include <iostream>
int
main()
{
std::cout << "Hello, World!" << std::endl;
return 0;
}</pre>
</section>
<section id="python-sample">
<h4>Python</h4>
<p>And for completeness, Python compared to Perl and BASH:</p>
<pre class="code">#!/usr/bin/python
language = 'Python'
print('Hello, World, from {}'.format(language))
</pre>
</section>
<section id="perl-sample">
<h4>Perl</h4>
<p>Any Perl:</p>
<pre class="code">#!/usr/bin/perl
use strict;
use warnings;
my $language = "Perl";
printf("Hello, World, from %s\n", $language);</pre>
</section>
<section id="bash-example">
<h4>BASH</h4>
<p>Unfairly simple:</p>
<pre class="code">#!/bin/bash
language=BASH
echo "Hello World, from $language"</pre>
</section>
</section>
<section id="see-also">
<h3>See also</h3>
<ul>
<li><a href="mixing-kotlin-and-go.html"
title="Mixing Kotlin and Go"><cite>Mixing Kotlin and
Go</cite></a></li>
</ul>
</section>
<section id="update">
<h3>Update</h3>
<p>As usual, I never catch as many problems in my writing as I do
after reading it posted publically. Many small edits, and an
added, explicit mention of <code>wstring</code>.</p>
</section>
<footer id="footnotes">
<h3 id="footnote-label">Footnotes</h3>
<ol>
<li id="fn1"><a
href="https://hackernoon.com/javascript-vs-python-in-2017-d31efbb641b4"
title="JavaScript vs. Python in 2017">A surprising
take</a> on Python versus JavaScript from Michael Bolin.
And I do not feel <a
href="https://kotlinlang.org/docs/tutorials/javascript/kotlin-to-javascript/kotlin-to-javascript.html">Kotlin
(JS)</a> is ready yet for front-end work.
</li>
<li id="fn2">In contrast to <em>ESR</em>'s thoughtful posts, a
nice Steve Yegge <a
href="https://steve-yegge.blogspot.com/2017/05/why-kotlin-is-better-than-whatever-dumb.html"
title="Why Kotlin Is Better Than Whatever Dumb Language You're Using">rant
in favor of Kotlin</a>. (I enjoy both their writing
styles.)
</li>
<li id="fn3">YMMV — I use Perl as a example, but it
could
be Ruby or PHP or similar. And some might prefer Node.js to
Python (but don't: see <a href="#fn1">footnote 1</a>. The
exact
choice is a matter of preference: I prefer Python, and some
would keep Ruby (for example).
</li>
<li id="fn4">Mike Vanier wrote <a
href="http://users.cms.caltech.edu/~mvanier/hacking/rants/scalable_computer_programming_languages.html"
title="Scalable computer programming languages">a similar
list</a> for "scalable computer programming languages" in
2001,
at least for the technical elements.
</li>
<li id="fn5">Mike Hearn points out <a
href="https://blog.plan99.net/modern-garbage-collection-911ef4f8bd8e"
title="Modern garbage collection">potential
pitfalls</a> with Go GC.
</li>
</ol>
</footer>
</article>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-41158527384994952082017-12-28T14:18:00.003-06:002017-12-28T16:28:39.702-06:00Push early, push often, push on green<article>
<section id="reference">
<p>(This post follows up on <a
href="https://binkley.blogspot.com/2017/03/frequent-commits.html"
title="Frequent commits"><cite>Frequent commits</cite></a>, a
post about git command line help for TDD. I assume you are already
following <a
href="https://agileinaflash.blogspot.com/2009/03/tdd-process-smells.html"
title="TDD Process Smells">good TDD practices</a>. Also,
please recall git normally requires pulling before pushing if your
repo is behind on commits from the remote.)</p>
</section>
<section id="prologue">
<h3>Prologue</h3>
<p>I'm chatting with a colleague who is <a
href="https://www.thoughtworks.com/insights/blog/road-agile-practitioner-agile-coach"
title="The Road from an Agile Practitioner to an Agile Coach">new
in her role as an <em>Agile Coach</em></a> (she comes from a PM
background). We were talking about ways to organize a team's story
board (card wall), and turned to <a
href="https://www.thoughtworks.com/insights/blog/defect-prevention-using-agile-techniques"
title="Defect Prevention Using Agile Techniques"><em>Desk
Checks</em></a> and when they fit into a story's life.</p>
<p>An interesting remark by her: a team had agreed to share work
(push) only after the desk check was successful; that is, they did
not push code until a story was almost done: work lay fallow on
individuals' machines for potentially days at a stretch.</p>
<p>I was surprised. Why would they wait days to push—what did
they do about merge conflicts, complex refactorings, integration
failures in the pipeline, <i>et al</i>?
</p>
</section>
<section id="lecture">
<h3>Lecture</h3>
<h4>Entropy</h4>
<p>To me this was clearly a <a
href="https://martinfowler.com/bliki/CodeSmell.html"
title="Code Smell"><em>smell</em></a>. Martin Fowler
specifically addresses this in <a
href="https://martinfowler.com/articles/continuousIntegration.html#EveryoneCommitsToTheMainlineEveryDay"
title="Everyone Commits To the Mainline Every Day"><cite>Everyone
Commits To the Mainline Every Day</cite></a>, and I would go
further: <em>Push commits at <strong>the earliest responsible
moment</strong></em>. This is opposite the advice for
refactoring, or especially <a
href="https://www.slideshare.net/ThoughtWorks/neal-ford-emergent-design-and-evolutionary-architecture"
title="emergent design & evolutionary architecture"><em>emergent
design</em></a>, where the <a
href="https://en.wikipedia.org/wiki/Rule_of_three_(computer_programming)">"Rule
of 3"</a> and <a
href="https://blog.codinghorror.com/the-last-responsible-moment/"
title="The Last Responsible Moment"><cite>last responsible
moment</cite></a> cautions waiting for more information before
committing to a course of action.</p>
<p>And you can see why early pushes differ from the other two: waiting
will not get you more information. On the contrary, waiting will
only increase <a
href="https://en.wikipedia.org/wiki/Software_entropy"
title="Software entropy">the entropy of the code base</a>!
Commits <em>lie fallow</em> in the local repo, increasing the size
of potential merge conflicts for others.</p>
<table>
<thead>
<tr>
<th> </th>
<th>Benefit from more information?</th>
<th>Principle</th>
<th>Entropy from waiting</th>
</tr>
</thead>
<tbody>
<tr>
<td>Early push</td>
<td>None available</td>
<td>Earliest responsible moment</td>
<td>Rises from fallow commits</td>
</tr>
<tr>
<td>Refactoring</td>
<td>Get more code examples</td>
<td>Rule of 3</td>
<td>Falls after refactoring</td>
</tr>
<tr>
<td>Architecture decision</td>
<td>Learn more about system</td>
<td>Last responsible moment</td>
<td>Falls if responsible</td>
</tr>
</tbody>
</table>
<p>(The "information" in the case of pushes are the pulled commits
themselves.)</p>
<p>I've definitely experienced this firsthand, when I'd eventually
discard my local commits after waiting too long, and letting them
grow too much in a different direction from how the rest of the
team progressed. <a
href="http://www.allaboutlean.com/muda-mura-muri/"
title="Muda, Mura, Muri: The Three Evils of Manufacturing"><strong>Waste!</strong></a>
</p>
<h4>Complexity</h4>
<p>Consider this work cycle:</p>
<ol>
<li>Local commit</li>
<li>Fetch commits from remote</li>
<li>Merge, if needed</li>
<li>Local commit again, if needed</li>
<li style="margin-top: 1em">Push commits to remote</li>
</ol>
<p>I've grouped these to emphasize what is <em>local to you</em> (the
first four) and what is <em>global to your team</em> (the last
one).</p>
<p>Considered only <em>locally</em>, you minimize entropy with <em>frequent
pulls</em> for yourself, and likewise for your teammates
individually, so you can catch merge conflicts early and resolve
them when they are small. But considered <em>globally</em>, you
need <em>frequent pushes</em> so those local pulls have only small
changes in them. <em>The longer you wait to push, the more work
for those who pull.</em></p>
<table>
<caption>Early push</caption>
<thead>
<tr>
<th>You</th>
<th>Rest of team</th>
<th>Work for others</th>
</tr>
</thead>
<tbody>
<tr>
<td>You commit</td>
<td> </td>
<td> </td>
</tr>
<tr>
<td>You push</td>
<td> </td>
</tr>
<tr>
<td> </td>
<td>They pull</td>
<td>Less complexity of merge (1 commit)</td>
</tr>
<tr>
<td>You commit</td>
<td> </td>
</tr>
<tr>
<td>You push</td>
<td> </td>
</tr>
<tr>
<td> </td>
<td>They pull</td>
<td>Less complexity of merge (1 commit)</td>
</tr>
</tbody>
</table>
<p>Each single push can be treated on it's own. There are two
opportunities for merge conflict, but each is a small amount of
work.</p>
<table>
<caption>Late push<sup><a href="#fn1">[1]</a></sup></caption>
<thead>
<tr>
<th>You</th>
<th>Rest of team</th>
<th>Work for others</th>
</tr>
</thead>
<tbody>
<tr>
<td>You commit</td>
<td> </td>
<td> </td>
</tr>
<tr>
<td> </td>
<td>They pull</td>
<td>No changes to merge</td>
</tr>
<tr>
<td>You commit</td>
<td> </td>
</tr>
<tr>
<td>You push</td>
<td> </td>
</tr>
<tr>
<td> </td>
<td>They pull</td>
<td>Greater complexity of merge (2 commits)</td>
</tr>
</tbody>
</table>
<p>In each scenario, there are two commits for others to contend with.
The larger, combined push has a greater opportunity for merge
conflict, and a greater chance for a large amount of work, because
of the combined interactions of the two commits.</p>
<p>And as teams work in parallel, there are more opportunities for
merge conflicts.</p>
<h4>Push early, push often, push on green</h4>
<p>From the above discussion, the safest course is to push early
rather than wait as commits pile up locally. But when to push—what
is the "earliest responsible moment"?</p>
<p>If your codebase is well-tested, and answer presents itself: <em>Push
when <strong>tests are green</strong> and <strong>changes alter
the complexity</strong></em>.</p>
<p>The goal is to avoid complex commit interactions that lead to merge
conflicts. Tests are the safety net. Further, if all else fails
and a commit is bad, it is easy to throw away the last commit
until things are right again: only a small amount of work is lost,
not days worth.</p>
<p>Understanding what kind of changes alter complexity takes skill:
skills improve with experience and coaching. The cost of early
pushes is low, and the occassional penalty of late pushes high, so
this would be a good topic for a "team norms" ("dev practices")
discussion.</p>
<p>For example, the team might agree that changes to comments are not
in themselves worth a push. At the other end, your refactorings
which impact more than one source file almost certainly should be
pushed early: discover their impact on others before you add more
refactorings.</p>
<p>A good work cycle:</p>
<ol>
<li>Pull</li>
<li>Build, run tests</li>
<li style="margin-top: 1em">Edit sources</li>
<li>Build, run tests</li>
<li>Commit</li>
<li>Pull and push</li>
</ol>
<p>After a preliminary sanity check (#1 and #2), get in the the cycle
of #3 to #6.</p>
</section>
<section id="epilogue">
<h3>Epilogue</h3>
<p>I checked with other teams: it is minority practice to wait until a
successful desk check to push changes. That's a relief. Hopefully
this practice can be made more rare.</p>
<p>One rational reason—itself a smell—is when tests take
too long to run frequently. When I design a pipeline, I recommend
breaking out "unit" tests from "integration" tests for this exact
reason: even when integration tests run long, the initial CI stage
with just unit tests should be fast enough to give quick feedback
on frequent pushes, and encourage <em>Push early, Push often, Push
on (local) green</em>.</p>
</section>
<section id="further-reading">
<h3>Further reading</h3>
<ul>
<li>Seth Robertson posted <a
href="https://sethrobertson.github.io/GitBestPractices/"
title="Commit Often, Perfect Later, Publish Once: Git Best Practices">a
wonderful list of git best practices</a>.<sup><a href="#fn2">[2]</a></sup>
</li>
</ul>
</section>
<section id="footnotes">
<h3>Footnotes</h3>
<p id="fn1"><sup>[1]</sup> The simple statement, "a greater chance for
a large amount of work", has rather complex reasoning behind it,
beyond the scope of this post.</p>
<p>For example, any particular commit
can be viewed as applying an exponent to the overall complexity of
a program. A neutral change (say, correcting a typo in a comment)
has the exponent <em>1</em>: it does not change the overall
complexity; a positive change (say, removing code duplication) has
an exponent between <em>0</em> and <em>1</em>: it lowers the
overall complexity; a negative change (say, adding a new
dependency) has an exponent greater than <em>1</em>: it raises the
overall complexity.</p>
<p>Consider then that these complexity changes are not simple numbers,
but distributions ("odds"), and change with time ("bitrot"), and
involve more than the code (people or requirments changes).</p>
<p id="fn2"><sup>[2]</sup> In Seth's post, do not confuse "publish
once" with "wait to push": it means "don't publish the same commit
twice" (which does sometimes happen accidentally, even for
experts, from merging or rebasing).</p>
</section>
<section id="update">
<h3>Update</h3>
<p>Sure enough, right after posting I read an interesting discussion
on <a href="http://brooker.co.za/blog/2017/12/28/mean"
title="Is the Mean Really Useless?">the value of the <em>statistical
mean</em></a> (average) relevant to the discussion on two
commits taken separately or together.</p>
<p>Essentially, even when the merge conflict work averages out over
time for pushing two commits separately versus pushing them
together, the outliers for pushing them together is significantly
worse than for pushing them separately because of interactions and
complexity.</p>
</section>
</article>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-28739701333288247352017-11-01T17:41:00.002-05:002017-11-02T06:45:39.604-05:00Old scripting<h3>The landscape</h3>
<p>I'm helping someone with a scripting problem on an old system and an old
shell. How old? Try IBM AIX 6.1, first released in 2007, and ksh93
"e" released in ... 1993. At least the AIX is a version of 6.1 from 2014!
(Kudos to IBM for treating long-term support seriously.)</p>
<p>A second point to ponder. The goal is to improve remote
scripting—running scripts on a remote machine. In this
environment, <code>ssh</code> exists but is not used. The remote execution
tool chosen is <code>rexec</code>, <a
href="https://books.google.com/books?id=_-1jwRwNaEoC&lpg=PA198&ots=KxAUu0sVYq&dq=rsh%20rexec%20dangerous&pg=PA198#v=onepage&q=rsh%20rexec%20dangerous&f=false">considered
one of the most
dangerous tools possible</a>. But my remit is not to address the
insecurity, just to improve the scripting. (They know this is a bad,
and are actively working to eventually resolve.)</p>
<p>So, given these constraints, what problem am I solving?</p>
<h3>Example problem</h3>
<p>This environment makes extensive use of remotely executed scripts to
wire together a distributed, locally-hosted system. Current scripts
duplicate the same approach, each implemented as a one-off: Copy a
script to a remote machine with <code>rcp</code>; use
<code>rexec</code> to invoke the script, capturing the output to a
file on the remote host; copy the captured file back to the local host;
process the output file; sometimes clean up the remote host afterwards.
</p>
<p>Some gotchas to watch out for with <code>ksh93e</code> or
<code>rexec</code>:</p>
<ul>
<li>Function tracing - Using the standard <code>xtrace</code> setting
to trace script execution in <code>ksh93</code> has <a
href="https://stackoverflow.com/a/2274727/250924">problems
with tracing functions</a>, and requires using <em>old</em>-style
function syntax
</li>
<li>Variable scope - To keep variables local to a function in
<code>ksh93</code>, you must <a
href="https://stackoverflow.com/a/12004099/250924">use the
<em>new</em>-style function syntax</a> (note the conflict with
tracing)
</li>
<li>Exit broken with trap - When calling <code>exit</code> to quit a
remote script, <code>trap</code> does not get a correct
<code>$?</code> variable (it is always 0, as <code>exit</code>
succeeded in returning a non-0 exit status). Instead one must "set"
<code>$?</code> with the code of a failing command, and then leave
with a plain call to <code>exit</code></li>
<li>No pipefail - Release "e" of ksh93 just does not know anything
about <code>set -o pipefail</code>, and there is no uninstrusive
workaround. This now common feature showed up <a
href="http://www.kornshell.com/doc/faq.html">in release
"g"</a></li>
<li>No exit code - Would you believe <code>rexec</code> does not
itself <a
href="https://www.unix.com/unix-for-dummies-questions-and-answers/20824-exit-codes-rexec.html">exit
with the exit code of the remote command</a>, never has, and
never will? It always exits 0 if the remote command could be started.
</li>
<li>Buffered <code>stderr</code> - Empirically, <code>rexec</code> (at
least the version with this AIX) <em>buffers</em> the <code>stderr
</code> stream of remote commands, and only flushes when <code>
rexec</code> exits, so the sense of ordering between <code>
stdout</code>, <code>stderr</code> and the command-line prompt is
even worse than usual (the actual handling <a
href="http://www.manpagez.com/man/8/rexecd/">is
unspecified</a>)
</li>
</ul>
<p>This problem and environment triggers a memory: The last time I worked
on AIX was in 1994, and it was almost the same problem! I really
thought I had escaped those days.</p>
<h3>A solution</h3>
<p>So I refactored. I couldn't change the use of <code>rexec</code>—this
environment is not ready for SSH key management—, I couldn't
replace KSH93 with BASH or replace AIX with Linux, but I could do
something about the imperfect duplication and random detritus files.</p>
<h4>The solution</h4>
<p>Note the need to call a <code>fail</code> function instead of
<code>exit</code> directly because of poor interaction with
<code>trap</code>.</p>
<p>Assuming some help, such as a global <code>progname</code> variable
(which could simply be <code>$0</code>), and avoiding remote
temporary files:</p>
<pre class="code">_transfer_exit_code() {
while read line
do
case $line in
^[0-9] | ^[1-9][0-9] | ^11[0-9] | ^12[0-7] ) return ${line#^} ;;
* ) printf '%s\n' "$line" ;;
esac
done
return 1 # ksh93e lacks pipefail; we get here when 'rscript' failed
}
rscript() {
case $# in
0 | 1 )
echo "$progname: BUG: Usage: rexec SCRIPT-NAME HOSTNAME [ARGS]..." >&2 ;;
* ) script_name=$1 ; shift
hostname=$1 ; shift ;;
esac
# Trace callers script if we ourselves are being traced
case $- in
*x* ) _set_x='set -x' ;;
esac
rexec $hostname /usr/bin/ksh93 -s "$@" <<EOS | _transfer_exit_code
set - "$@" # Only reasonable way to pass through function arguments
# Work around AIX ksh93 return code of exit ignored by trap
fail() {
return \$1
}
# Our hook to capture the exit code for rexec who dumbly swallows it
trap 'rc=\$?; echo ^\$rc; exit \$rc' EXIT
PS4='+$script_name:\$(( LINENO - 14 )) (\$SECONDS) '
$_set_x
# The callers script
$(cat)
EOS
}</pre>
<h4>Example use</h4>
<pre class="">#!/usr/bin/ksh93
progname=${0##*/}
PS4='+$progname:$LINENO ($SECONDS) '
usage() {
echo "Usage: $0 [-d] HOSTNAME"
}
. rexec.ksh
debug=false
while getopts :d opt
do
case $opt in
d ) debug=true ;;
* ) usage >&2 ; exit 2 ;;
esac
done
shift $(( OPTIND - 1 ))
case $# in
1 ) hostname=$1 ;;
* ) usage >&2 ; exit 2 ;;
esac
$debug && set -x
script_name=My-Remote-Script
tmp=${TMPDIR-/tmp}/$progname.$RANDOM
trap 'rm -f $tmp' EXIT
rscript $script_name $hostname Katy <<'EOS' >$tmp
echo $#: $1
fail 3
EOS
case $? in
3 ) ;;
* ) echo "$0: Did not pass through exit code" >&2 ; exit 1 ;;
esac
case "$(<$tmp)" in
'1: Katy' ) ;;
* ) echo "$0: Did not pass through arguments" >&2 ; exit 1 ;;
esac</pre>
<h4>Source</h4>
<p>The code is <a
href="https://github.com/binkley/shell/tree/master/ksh">in GitHub</a>.
</p>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-25648830536707406592017-09-19T07:23:00.002-05:002017-09-19T07:34:59.090-05:00Help for JDBC with Java streams<p>We wanted to use JDBC with Java Streams, but encountered several
difficulties. Fortunately we found solutions with rather small bits of
code.</p>
<h3 id="checked-exceptions">Checked exceptions</h3>
<p>The main obstacle was the JDBC API throwing <code>SQLException</code> for
all API methods used in our code. <code>SQLException</code> is a <em>checked
exception</em>, so must be declared in our method signatures, or
caught otherwise. However the Streams API only accepts methods which
declare to throw <em>no</em> checked exceptions, so something simple like
this will not compile:</p>
<pre class="code">stream(results).
map(row -> row.getString("label")). // checked exception
forEach(this::processLabel);</pre>
<p>The call to <code>ResultSet.getString(String)</code> throws a checked
exception. The usual approach is to wrap the call, and handle the
exception in the wrapping method:</p>
<pre class="code">method String streamGetLabel(final ResultSet results) {
try {
return results.getString("label");
} catch (final SQLException e) {
throw new UncheckedIOException(e);
}
}</pre>
<p>(Here <code>UncheckedIOException</code> is an unchecked exception wrapper
we wrote for <code>SQLException</code>, similar to <code>UncheckedIOException</code>
in the JDK for <code>IOException</code>.) </p>
<p>Then the stream becomes:</p>
<pre class="code">stream(results).
map(this::streamGetLabel).
forEach(this::processLabel);</pre>
<p>This is OK, however needing to write a wrapper method for each time we
wanted to use JDBC in a stream became tedious.</p>
<h4 id="checked-exceptions-solution">Solution</h4>
<p>First we wrote a SAM (more on <a
href="https://zeroturnaround.com/rebellabs/java-8-the-first-taste-of-lambdas/"
title="Java 8: The First Taste of Lambdas">SAM interfaces/classes</a>)
interface as a lookalike for the JDK <code>Function</code> interface: this
is what <code>Stream.map(Function)</code> wants.</code>The lookalike is
different in that it throws <code>SQLException</code>: </p>
<pre class="code">@FunctionalInterface
public interface SQLFunction<T, R> {
R apply(final T t) throws SQLException;
// Other default methods - no more abstract methods
}</pre>
<p>Then we used this in a closed <code>Function</code> implementation to wrap
and delegate to the lookalike, and throw
<code>UncheckedSQLException</code> if the lookalike throws <code>SQLException</code>:
</p>
<pre class="code">@RequiredArgsConstructor(staticName = "applyUnchecked")
public final class UncheckedSQLFunction<T, R>
implements Function<T, R> {
private final SQLFunction<T, R> wrapped;
@Override
public R apply(final T t) {
try {
return wrapped.apply(t);
} catch (final SQLException e) {
throw new UncheckedSQLException(e);
}
}
}</pre>
<p>(Here we use the excellent <a
href="https://projectlombok.org/features/constructor"
title="@NoArgsConstructor, @RequiredArgsConstructor, @AllArgsConstructor">Lombok
library</a> to generate our constructor, and give us a static convenience
method, "applyUnchecked".)</p>
<p>Finally some static importing, and our example streams use becomes:</p>
<pre class="code">stream(results).
map(applyUnchecked(row -> row.getString("label"))).
forEach(this::processLabel);</pre>
<p>Or with more help:</p>
<pre class="code">stream(results).
map(getString("label")).
forEach(this::processLabel);</pre>
<p>We wrote similar lookalikes and wrappers for <code>Predicate</code> and
<code>Consumer</code>. It would be easy enough to write them for other
Java functional interfaces, such as <code>BiFunction</code>.</p>
<h3 id="streaming-result-sets">Streaming result sets</h3>
<p>The next difficulty we tackled was how to loop over <code>ResultSet</code>,
and use them with Streams.</p>
<p>(A side note: <code>ResultSet</code> is not a <em>set</em> but a
<em>list</em>: rows are ordered, and they can duplicate each other in
their column data. However, they were named after the SQL concept of sets,
not the Java one.)</p>
<p>Fundamentally, a <code>ResultSet</code> is not an <a
href="https://docs.oracle.com/javase/8/docs/api/java/util/Iterator.html"><em>iterator</em></a>,
but is close:</p>
<table>
<tr>
<th>Iterator</th>
<th>ResultSet</th>
<th>Returns</th>
<th> </th>
</tr>
<tr>
<td><code>hasNext()</code></td>
<td><code>next()</code></td>
<td><code>boolean</code></td>
<td> </td>
</tr>
<tr>
<td><code>next()</code></td>
<td><code>this</code></td>
<td><code>ResultSet</code></td>
<td>(Yes, the <code>ResultSet</code> itself is the equivalent)</td>
</tr>
</table>
<h4 id="streaming-result-sets-solution">Solution</h4>
<p>To provide a <code>ResultSet</code> as an <em>iterator</em>:</p>
<pre class="code">final List<String> values = new ArrayList<>();
for (final ResultSet row : iterable(results)) {
values.add(row.getString("value"));
}</pre>
<p>Moreso, to provide one as a <em>stream</em>:</p>
<pre class="code">final List<String> values = stream(results).
map(getString("value")).
collect(toList());</pre>
<p>Any SQL failures are thrown as <em>unchecked exceptions</em>. The stream
has
<a href="https://docs.oracle.com/javase/8/docs/api/java/util/Spliterator.html#characteristics--">the
characteristics</a>: immutable, nonnull, and ordered.</p>
<h3 id="transactions">Transactions</h3>
<p>We found JDBC transactions to be tricky. Fundamentally they are tied to a
<em>connection</em>; there is no proper nesting. (To simulate nesting, use
separate connections. Even then, there is no guarantee of ordering from
the database engine.) And they have a baroque API, relying on diddling of
the "auto-commit" setting with care needed to restore its setting after
the transaction concludes. Several bugs ensued before we switched to using
small helper interfaces and methods.</p>
<p>Further, some programming languages do not distinguish <code>void</code>
from other return types (e.g., <code>Unit</code> type): Java is not one of
them. Likewise for user vs primitive types (<code>Boolean</code> vs <code>boolean</code>).
Hence, there are separate transaction blocks for consumers, functions, and
predicates.</p>
<h4 id="transactions-solution">Solution</h4>
<p>One example explains them all. Consider functions and the
<code>SQLFunction</code> lookalike interface:</p>
<pre class="code">@RequiredArgsConstructor(staticName = "applyTransacted")
public final class TransactedFunction<T, R>
implements SQLFunction<T, R> {
private final Connection connection;
private final SQLFunction<T, R> wrapped;
@Override
public R apply(final T in)
throws SQLException {
connection.setAutoCommit(false);
try {
final R out = wrapped.apply(in);
connection.commit();
return out;
} catch (final SQLException e) {
connection.rollback();
throw e;
} finally {
connection.setAutoCommit(true);
}
}
}</pre>
<p>(The pattern is the same for other collection operations.)</p>
<p>With a helper and static importing:</p>
<pre class="code">final Integer value = applyTransacted(connection, in -> 0).apply("string");</pre>
<p>Or when the transaction fails:</p>
<pre class="code">applyTransacted(connection, in -> {
throw new SQLException("Something went wrong");
}).apply("string");</pre>
<h3 id="some-convenience">Some convenience</h3>
<p>Many places in these examples are improved with helper functions, or for
transactions, with <a href="https://en.wikipedia.org/wiki/Currying">currying</a>
(similar to <a href="https://en.wikipedia.org/wiki/Builder_pattern">the
builder pattern</a>). Hence, the wide use of Lombok static
constructors. Transactions are another example as they need a <code>Connection</code>
for begin/commit/rollback.</p>
<h4 id="some-convenience-solution">Solution</h4>
<p>A simple helper curries connection for transactions:</p>
<pre class="code">@RequiredArgsConstructor(staticName = "with")
public final class WithConnection {
private final Connection connection;
public <T> Predicate<T> testTransacted(final SQLPredicate<T> wrapped) {
return UncheckedSQLPredicate.testUnchecked(
TransactedPredicate.<T>testTransacted(connection, wrapped));
}
public <T, R> Function<T, R> applyTransacted(
final SQLFunction<T, R> wrapped) {
return UncheckedSQLFunction.applyUnchecked(
TransactedFunction.<T, R>applyTransacted(connection,
wrapped));
}
public <T> Consumer<T> acceptTransacted(final SQLConsumer<T> wrapped) {
return UncheckedSQLConsumer.acceptUnchecked(
TransactedConsumer.<T>acceptTransacted(connection, wrapped));
}
}</pre>
<p>Example use:</p>
<pre class="code">final Optional<Integer> value = Stream.of(0).
filter(with(connection).testTransacted(in -> true)).
findFirst();</pre>
<p>(Yes, we might also describe the code as <a
href="https://en.wikipedia.org/wiki/Partial_application"><em>partial
application</em></a>. The "object-oriented" implementation confuses
matters with the hidden <code>this</code> reference.)</p>
<h3>Conclusion</h3>
<p>There is nothing we did that was difficult or complex: simple one-liner
interfaces, simple wrapper implementations of Java funcional interfaces,
some rote JDBC best practices. The main difficulty was conceptual: seeing
the duplication of many, small wrapper methods, and pulling out their
commonality. This is a good pattern to keep in mind throughout your
code.</p>
<h3><strong>UPDATE:</strong></h3>
<p>And the source: <a href="https://github.com/binkley/java-sql-stream">Java
helpers for JDBC and Streams</a>.</p>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-14151874964276610352017-08-11T12:45:00.002-05:002017-08-12T07:29:25.437-05:00How to write clean Java<p>I cannot tell you how to write <em>good</em> Java, but I can help you write
<strong>clean</strong> Java. As usual, automation is key. It's all about
the tooling.</p>
<p>The best thing about good tooling is that they work together: each covers a
different area, does not impeded another tool, and fixing one complaint a
tool reveals often fixes complaints from other tools.</p>
<p>This advice applies to any programming language, not just Java. Java,
having the most mature ecosystem, is instructive.</p>
<h3 id="the-tools">The tools</h3>
<dl>
<dt><em>Use good source control</em></dt>
<dd><a href="https://git-scm.com/" title="Git">Git</a> is your best
choice. Use either <a href="https://trunkbaseddevelopment.com/"
title="Trunk Based Development">trunk-based
development</a> with <a
href="https://trunkbaseddevelopment.com/feature-flags/"
title="Feature flags - Trunk Based Development">feature
toggles (feature flags)</a>, or <a
href="https://www.atlassian.com/git/tutorials/comparing-workflows#gitflow-workflow">gitflow</a>
patterns, depending on your needs and organization. Require full
builds and testing before pushing to a shared repository; <a
href="https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks"
title="Git - Git Hooks">hooks</a> can help together with your
build tool.
</dd>
<dt><em>Use good build automation</em></dt>
<dd><a href="https://maven.apache.org/" title="Welcome to Apache Maven">Maven</a>
or <a href="https://gradle.org/" title="Gradle Build Tool">Gradle</a>
are your best choices; either is excellent. Teach the tool to be
strict and fail builds if anything is not just right. Treat your build
configuration as code, part of the same repository and held to the
same hygiene standards. Keep your local build fast.
</dd>
<dt><em>Use a good editor</em></dt>
<dd><a href="https://www.jetbrains.com/idea/"
title="IntelliJ IDEA: The Java IDE for Professional Developers by JetBrains">IntelliJ</a>
is the best choice. Use <a
href="https://www.jetbrains.com/help/idea/code-inspection.html"
title="Code Inspection">inspections</a>, <a
href="https://www.jetbrains.com/help/idea/intention-actions.html"
title="Intention Actions">intentions</a>, and <a
href="https://www.jetbrains.com/help/idea/reformatting-source-code.html"
title="Reformatting Source Code">reformatting</a> obsessively:
code diffs should only show real changes, not whitespace or
formatting. Use <a href="https://plugins.jetbrains.com/"
title="JetBrains Plugin Repository">plugins</a> to
ease integrating IntelliJ with other tooling.
</dd>
<dt><em>Keep your style consistent</em></dt>
<dd><a href="http://checkstyle.sourceforge.net/" title="Checkstyle">Checkstyle</a>
is your best choice; other choices are needed for non-Java JVM
languages. Fail your build if checkstyle complains. Keep your IntelliJ
formatting and Checkstyle rules consistent. Generally, choose either
<a href="http://checkstyle.sourceforge.net/sun_style.html"
title="checkstyle – Sun's Java Style">Sun</a> or <a
href="http://checkstyle.sourceforge.net/google_style.html"
title="checkstyle – Google's Style">Google</a> coding
standards, and be leary of deviating from them; if unsure, pick Sun.
</dd>
<dt><em>Fully test your code</em></dt>
<dd><a href="http://www.eclemma.org/jacoco/"
title="EclEmma - JaCoCo Java Code Coverage Library">JaCoCo</a>
works well. Fail your build if coverage drops below a threshhold;
rachet up the threshhold as coverage improves. Start low and aim for a
95%+ threshhold. Follow <a
href="https://testing.googleblog.com/2015/04/just-say-no-to-more-end-to-end-tests.html"
title="Google Testing Blog: Just Say No to More End-to-End Tests">the
<em>Test Pyramid</em></a>; save your end-to-end tests for CI, or
run locally only once before pushing commits.
</dd>
<dt><em>Be zealous in looking for problems</em></dt>
<dd><a href="http://findbugs.sourceforge.net/"
title="FindBugs™ - Find Bugs in Java Programs">FindBugs</a>,
<a href="https://sourceforge.net/projects/pmd/" title="PMD">PMD</a>,
or <a href="http://errorprone.info/" title="Error Prone">Error
Prone</a> are your best choices (pick one); other choices may work
better for non-Java JVM languages. Fail your build if your choice
complains. Be judicious in disabling complaints (for example, FindBugs
"experimental" checks, should likely be disabled).
</dd>
<dt><em>Use code generation</em></dt>
<dd><a href="https://projectlombok.org/" title="Project Lombok">Lombok</a>
is your first choice. <a
href="https://medium.com/@iammert/annotation-processing-dont-repeat-yourself-generate-your-code-8425e60c6657"
title="Annotation Processing : Don’t Repeat Yourself, Generate Your Code.">Add
others as needed</a> (or build domain-specific code generators).
Generated code does not need test coverage, style checks, or bug
detection if the generator is clean and well-tested: trust it.
</dd>
</dl>
<h3>Update</h3>
<p><a href="https://plus.google.com/+DanWallach">Dan Wallach</a> reminded me
of Error Prone, added above.</p>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-82959700327579119072017-08-11T08:20:00.003-05:002017-08-11T09:07:05.073-05:00Cygwin terminal in IntelliJ<p>IntelliJ sports an excellent terminal emulator (the "Terminal" tab at
bottom of the editor). By default it brings up a terminal native to your
Operating System: <code>CMD.EXE</code> on Windows, <code>$SHELL</code> on
Linux and Mac.</p>
<p>However I prefer Cygwin when I work on Windows. WSL is incredible, but
there are still interoperability issues between its filesystem and
Windows-native programs, and IntelliJ (which relies on
<code>java.exe</code>, a Windows-native program) <a
href="https://youtrack.jetbrains.com/issue/IDEA-171457"
title="Support for WSL project ">is still working on it</a>.</p>
<p>So, how to open a Cygwin terminal in IntelliJ? Setting the program to start
in <code>Settings|Tools|Terminal|Shell path</code>, the most obvious thing
to do, does not quite work:</p>
<pre class="code">C:\cygwin64\bin\bash.exe</pre>
<p>This is a non-interactive shell, and does not source your profile. The next
try is:</p>
<pre class="code">C:\cygwin64\bin\bash.exe --login -i</pre>
<p>This produces an error from IntelliJ that it cannot start the program
correctly. A little checking says the leading command needs to be quoted,
else IntelliJ treats the entire line as the name of the command, not as a
command followed by flags. OK:</p>
<pre class="code">"C:\cygwin64\bin\bash.exe" --login -i</pre>
<p>Hey, I have a shell! Unfortunately, it starts in my home directory, not in
my project root. Starting in the project root is one of the nice features
of the terminal in IntelliJ. Finally, <a
title="Configuring IntelliJ's terminal to run Cygwin"
href="https://gist.github.com/mpicker0/a6a3f10e6b9278074f93">two
changes</a>. First the IntelliJ setting:</p>
<pre class="code">"C:\cygwin64\bin\bash" -c "exec /usr/bin/env INTELLIJ=true $SHELL --login -i"</pre>
<p>And an addition to my <code>~/.bashrc</code>:</p>
<pre class="code">${INTELLIJ-false} && cd ${OLDPWD-.}</pre>
<p>Ipso presto!</p>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-1984558311695645782017-06-12T11:53:00.001-05:002017-06-13T07:53:03.037-05:00Example JVM agent in Kotlin<p>Oleg Shelajev wrote an <a
href="https://zeroturnaround.com/rebellabs/how-to-inspect-classes-in-your-jvm/"
title="How-to guide to writing a javaagent">excellent tutorial
post</a> on writing JVM agents. These are bits of code which run <em>before</em>
your <code>main()</code> method. Why do this? It permits <a
href="https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/Instrumentation.html">some
interesting tricks</a>, chiefly modifying classes as they are loaded,
but also estimating the actual memory used by Java objects.</p>
<p>I gave this a try myself, but rather than writing my agent in Java, I wrote
it in Kotlin. It was straight-forward, with only one gotcha.</p>
<h3>AgentX.kt</h3>
<pre class="code">@file:JvmName("AgentX")
package hm.binkley.labs.skratch.jvmagent
import java.lang.instrument.Instrumentation
fun premain(arguments: String?, instrumentation: Instrumentation) {
println("Hello from AgentX 'premain'!")
}
fun main(args: Array<String>) {
println("Hello from AgentX 'main'!")
}</pre>
<p>OK, the non-gotcha. You can declare functions at the <em>package</em>
level. This acts just like static methods in Java, with simpler syntax (no
potentially artificial wrapper class to hold the static method). The two
obvious examples in the above code are <code>main()</code> and <code>premain()</code>.
</p>
<p>But <a
href="https://kotlinlang.org/docs/reference/java-to-kotlin-interop.html#package-level-functions"
title="Calling Kotlin from Java">when calling Kotlin from Java</a>,
you use a wrapping class name in the the fully-qualified method name. My
Kotlin file is named "AgentX.kt", so the default class name for Java is
"AgentXKt". I'm lazy, wanted to save some typing, so I used a Kotlin
package-level annotation to name the wrapping class just "AgentX".</p>
<h3>Output</h3>
<p>The JVM requires an absolute path to any agent jar, and I'm running Cygwin,
so a little help to get a full path. Similarly, I used <a
href="https://maven.apache.org/plugins/maven-shade-plugin/"
title="Apache Maven Shade Plugin">the Maven shade plugin</a> to
build a single uber-jar holding my own classes, and those of my
dependencies (the Kotlin standard library).</p>
<pre class="code" style="white-space: pre-wrap">$ java -javaagent:$(cygpath -m $PWD/target/skratch-0-SNAPSHOT.jar) -jar target/skratch-0-SNAPSHOT.jar
Hello from AgentX 'premain'!
Hello from AgentX 'main'!</pre>
<p>Project is here: <a href="https://github.com/binkley/skratch">https://github.com/binkley/skratch</a>.
</p>
<h3>Gotcha</h3>
<p>Enough preamble, now the gotcha. Unlike Java, Kotlin helps you <a
href="https://kotlinlang.org/docs/reference/null-safety.html"
title="Null Safety">protect yourself from <code>null</code>s</a>
without boilerplate code. So in <code>premain()</code> for the "arguments"
parameter, you need to use <strong><code>String?</code></strong> rather
than <code>String</code> as the parameter type as the JVM may pass you a
<code>null</code>. The first time I tried the code, I didn't realize this
and it blew up:</p>
<pre class="code" style="white-space: pre-wrap">$ java -javaagent:$(cygpath -m $PWD/target/skratch-0-SNAPSHOT.jar) -cp target/skratch-0-SNAPSHOT.jar hm.binkley.labs.skratch.jvmagent.AgentX
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.instrument.InstrumentationImpl.loadClassAndStartAgent(InstrumentationImpl.java:386)
at sun.instrument.InstrumentationImpl.loadClassAndCallPremain(InstrumentationImpl.java:401)
Caused by: java.lang.IllegalArgumentException: Parameter specified as non-null is null: method hm.binkley.labs.skratch.jvmagent.AgentX.premain, parameter arguments
at hm.binkley.labs.skratch.jvmagent.AgentX.premain(AgentX.kt)
... 6 more
FATAL ERROR in native method: processing of -javaagent failed</pre>
<p>Interesting! Kotlin found the issue at runtime. It can't find it at compile
time as the JVM API for "premain" is pure convention without an interface
or class to inspect.</p>
<p>Let's try running the agent a different way. The command-line lets us pass
options, and these become the "arguments" parameter:</p>
<pre class="code" style="white-space: pre-wrap">$ java -javaagent:$(cygpath -m $PWD/target/skratch-0-SNAPSHOT.jar)= -cp target/skratch-0-SNAPSHOT.jar hm.binkley.labs.skratch.jvmagent.AgentX
Hello from AgentX 'premain'!
Hello from AgentX 'main'!</pre>
<p>Sneaky. The mere presence of the "=" on the command line turns the
"arguments" parameter from <code>null</code> to an empty string.</p>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-28511278763681532992017-05-17T10:59:00.003-05:002017-05-17T10:59:54.383-05:00SVG takeaways<p>I started playing with SVG for another blog post I'm working on. I wanted
to learn more about SVG, an XML application. Some of my takeaways:</p>
<h3>Z ordering</h3>
<p>Elements are drawn in the order they appear in the XML. So if you want
element <var>B</var> to overlay element <var>A</var>, then write
<var>A</var> earlier in XML, and write <var>B</var> later. Then
<var>B</var> will be drawn after <var>A</var>, and lay on top of it (if
they overlap), hiding bits of <var>A</var> if <var>B</var> isn't
transparent.</p>
<h3>Scaling</h3>
<p>You define your own coordinate system, and X/Y coordinates are all relative
to this drawing area, called the "view box". So if you declare the view
box to have an <var>X</var> ranging 0-20 "units" and a <var>Y</var>
ranging 0-100 "units", then your elements will use an <var>X</var> between
0 and 20 and a <var>Y</var> between 0 and 100.</p>
<p>Recall the "S" in "SVG" stands for <em>scalable</em>. So when I say
"units", these are relative units in context of the view box, and so in
the example, "50%" of the X coordinate is 10 "units". You don't write down
"units" in the XML, just provide the <var>X</var> and <var>Y</var> ranges.
</p>
<p>What size is this on screen? Your browser or device or PDF will render the
SVG view box to fit, and scale your elements accordingly. This is great
for diagrams and drawings. If you do nothing special to stretch or squeeze
the view box, your elements will retain perspective and always fit the
page.</p>
<p>Embedding SVG in HTML5, I found this produced nice standalone diagrams
within the vertical flow of text (using the X/Y from the example). The
"style" bit is important:</p>
<pre class="code">
<svg xmlns="http://www.w3.org/2000/svg" version="1.1" viewBox="0 0 100 20"
width="80%" style="margin: auto; display: block">
<!-- Write elements here -->
</svg>
</pre>
<h3>Styling</h3>
<p>Element attributes are consistent. So when I figured out that <code>stroke="black"</code>
drew black lines, I could use the same attribute on any element to assign
color. This was nice for learning the SVG language piecewise, picking up
bits as I went along.</p>
<p>Fractional values are OK in several contexts. So when I wanted skinnier
lines, I could write <code>stroke-width="0.5"</code>.</p>
<h3>A full example</h3>
<p>Here's the SVG for that post I mentioned at start. I provide it without
explanation; take your best guess what it's for!</p>
<svg xmlns="http://www.w3.org/2000/svg" version="1.1" viewBox="0 0 100 20"
width="80%" style="margin: auto; display: block">
<defs>
<marker id="arrow" viewBox="0 0 10 10" refX="1" refY="5" orient="auto"
stroke="green" fill="green">
<path d="M 0 0 L 10 5 L 0 10 z"></path>
</marker>
</defs>
<line x1="30" y1="0" x2="30" y2="30" stroke="lightgrey"
stroke-dasharray="1,1" stroke-linecap="round"
stroke-width="0.5"></line>
<line x1="30" y1="0" x2="30" y2="30" transform="rotate(7 30 10)"
stroke="grey" stroke-dasharray="1,1" stroke-linecap="round"
stroke-width="0.5"></line>
<line x1="50" y1="0" x2="50" y2="30" stroke="lightgrey"
stroke-dasharray="1,1" stroke-linecap="round"
stroke-width="0.5"></line>
<line x1="50" y1="0" x2="50" y2="30" transform="rotate(7 50 10)"
stroke="grey" stroke-dasharray="1,1" stroke-linecap="round"
stroke-width="0.5"></line>
<line x1="70" y1="0" x2="70" y2="30" stroke="lightgrey"
stroke-dasharray="1,1" stroke-linecap="round"
stroke-width="0.5"></line>
<line x1="70" y1="0" x2="70" y2="30" transform="rotate(7 70 10)"
stroke="grey" stroke-dasharray="1,1" stroke-linecap="round"
stroke-width="0.5"></line>
<line x1="10" y1="10" x2="90" y2="10" stroke="lightblue"
stroke-dasharray="2,1" stroke-linecap="round"
stroke-width="0.5"></line>
<line x1="10" y1="10" x2="90" y2="10" transform="rotate(-7 50 10)"
stroke="blue"></line>
<!-- TODO: How to calculate x/y for text? -->
<text x="13" y="9" font-size="7" font-style="italic">Work</text>
<line x1="15" y1="3" x2="80" y2="3" stroke="green"
marker-end="url(#arrow)"></line>
<text x="74" y="16" font-size="7" font-style="italic">Env</text>
<line x1="85" y1="17" x2="20" y2="17" stroke="green"
marker-end="url(#arrow)"></line>
<rect x="10" y="0" width="80" height="20" stroke="navy"
fill="transparent"></rect>
<text x="10" y="9" transform="rotate(-90 10 10)" font-size="10"
style="text-anchor: middle">Dev
</text>
<text x="90" y="9" transform="rotate(90 90 10)" font-size="10"
style="text-anchor: middle">Prod
</text>
</svg>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-77469253939851113032017-04-29T13:36:00.002-05:002017-06-23T05:24:16.409-05:00Maven color logging on Cygwin<p><strong>WORKAROUND:</strong> Thanks to comment by <a
href="https://plus.google.com/u/0/111012722058314768782">Zart
Colwing</a>, there is a workaround. Add
<code>-Djansi.passthrough=true</code> to <code>MAVEN_OPTS</code>. It is
not enough to add the flag to the command line; it needs to be seen by
JANSI before maven begins parsing the command line. See <a
href="https://issues.apache.org/jira/browse/MNG-6242">No color for
maven on Cygwin</a> to track the issue.</p>
<h3>Post</h3>
<p>I'm quite happy Maven 3.5.0 has added color logging!</p>
<p>With earlier versions of maven, I used Jean-Christophe Guy's <a
href="https://github.com/jcgay/maven-color">excellent maven-color
extension</a>. On Windows that involved some manual hacking of my maven
installation, but on Mac, it was trivial with homebrew.</p>
<p>So now I'm getting color output from maven out of the box. Except when I
don't.</p>
<p>You see, this new feature relies on <a
href="https://github.com/fusesource/jansi">the good JAnsi library</a>.
And JAnsi presently has an issue with color on Cygwin. When I'm at home,
Cygwin is my mainstay, used on my gaming-cum-programming rig, so this is
significant to me. What is the issue? No color—JAnsi detects I'm on
Windows, and uses the native Windows console color handling, which doesn't
work in Mintty or Xterm. Those use standard ANSI escape sequences, etc.
rather than an OS-specific library.</p>
<p>Digging through the source for JAnsi, I find <a
href="https://github.com/fusesource/jansi/blob/c6830ac1e54b1f8fd3985c5fe595a835ad56046a/jansi/src/main/java/org/fusesource/jansi/AnsiConsole.java#L46">the
trouble spot</a>:</p>
<pre class="code">private static final boolean IS_WINDOWS = System.getProperty("os.name").toLowerCase(Locale.ENGLISH).contains("win");</pre>
<p>Aha! The special Windows-handling code kicks in when the
<code>os.name</code> system property contains "win" (in any case). Using
the <code>jps</code> and <code>jinfo</code> tools that come with the JDK,
I double-checked against a long-running maven build (16848 just was the
PID used by the JVM for this maven build; use <code>jps</code> to list all
current PIDs):</p>
<pre class="code">$ jinfo -sysprops 16848 | grep os.name
os.name = Windows 10</pre>
<p>Some experimenting found a way to work around that:</p>
<pre class="code">MAVEN_OPTS=-Dos.name=Cygwin mvn clean</pre>
<p>(You need to use <code>MAVEN_OPTS</code> rather than passing <code>-Dos.name=Cygwin</code>
to maven; once maven starts the value is immutable.)</p>
<p>Color is back. It turns out, any value for <code>os.name</code> will work
(for example, "Bob") as long as it doesn't contain "win". I picked
"Cygwin" for explicitness.</p>
<p><strong>UPDATE</strong>: I have to rethink this. Sure I get color now, but
at the expense of <code>maven test</code> failing as maven no longer
believes I'm on Windows, so is unhappy at the lack of a
<code>/bin/sh</code> and general UNIX filesystem. One step forward, two
steps back.</p>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-59664535108294228782017-04-21T13:50:00.000-05:002017-04-21T13:50:15.810-05:00Quick Kotlin idiom: naming things<p>Kotlin is very much a better Java. A great example is this simple idiom for
naming things:</p>
<pre class="code">open class Named<in T>(val name: String, check: (T) -> Boolean)
: (T) -> Boolean by check {
override fun toString() = name
}</pre>
<p>So I can write something like this:</p>
<pre class="code">fun maybe_three(check: (Int) -> Boolean) {
if (check(3)) do_the_three_thing()
else println("\"$check says\", Do not pass go.")
}
maybe_three(Named("Is it three?") { i -> 3 == i })
maybe_three(Named("Is it four?") { i -> 4 == i })</pre>
<p>The first call of <code>maybe_three</code> prints: <samp>Breakfast, lunch,
dinner</samp>. The second call prints: <samp>"Is it four?" says, Do not
pass go.</samp></p>
<p>Many variations on this are possible, not just functions! What makes this
example work nicely is <a
href="https://kotlinlang.org/docs/reference/delegation.html">delegation</a>—the
magical <code>by</code> keyword— for the general feature of naming
things by overriding <code>toString()</code>; and for the function
delegated to, the elegant <a
href="https://kotlinlang.org/docs/reference/lambdas.html">lambda</a>
(anonymous function) syntax for the last parameter. You can delegate
anything, not just functions, so you could make named maps, named business
objects, <i>et al</i>, by using delegation on existing types without
needing to change them.</p>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-3037157744588540842017-04-13T17:12:00.001-05:002017-04-15T08:50:00.174-05:00WSL, first experiences<p>I first tried Windows Subsystem for Linux in December 2016, but was not
successful in installing, so I held off.</p>
<p>After getting the Windows 10 Creators Edition update, I saw <a
href="https://blogs.msdn.microsoft.com/commandline/2017/04/11/windows-10-creators-update-whats-new-in-bashwsl-windows-console/"
title="Windows 10 Creators Update: What’s new in Bash/WSL & Windows Console">how
much work and love</a> went into improving WSL, and decided to try again.
I was rewarded.</p>
<p>On the whole, the installation was smooth, brief, you might even say <a
href="http://catb.org/jargon/html/T/trivial.html">trivial</a>. There
were Windows reboots to enable Developer Mode, and after installing WSL,
but much solid effort has gone into making Windows reboots quick and
painless, and with a regular Linux distro I'd have rebooted anyhow after
upgrading, so no disgruntlement.</p>
<p>And what did I get for my efforts? WSL bash is bash. Just bash. Really, it
is just plain old bash, with all the command line tools I've grown
accustomed to over 30 years. The best praise I can give a tool: <em>It
just works</em>. And WSL just works. (But see <a href="#almost-there"><cite>Almost
there</cite></a>, below.)</p>
<p>Out of the box WSL runs Ubuntu 16.04 (Xenial), the official LTS
distribution (long-term support). This is a sane choice for Microsoft.
It's stable, reliable, secure, tested, trusted. For anyone wanting a
working Linux command line, this is a go-to choice. Still, I updated
it.</p>
<h3 id="things-i-changed">Things I changed</h3>
<p>Even with all the goodness, there were some things I had to change:</p>
<dl>
<dt>The terminal</dt>
<dd>I immediately installed <a href="https://github.com/mintty/wsltty"
title="Mintty as a terminal for Bash on Ubuntu on Windows / WSL">Mintty
for WSL</a>. I've grown to love Mintty on Cygwin, trusting it as a
reliable and featureful terminal emulator without going overboard.
It's a tasteful balance, well executed. And <code>CMD.EXE</code>,
though much improved, still is not there (but may head there; we'll
see if PowerShell wins out).
</dd>
<dt>DBus</dt>
<dd>Not to get into flamewars, I just accept that Ubuntu uses DBus. By
default it doesn't run on WSL, but this was easy to fix, and it made
upgrading Ubuntu smoother. Using <code>sudo</code>, edit <code>/etc/dbus-1/session.conf</code>
as
<a href="https://www.reddit.com/r/Windows10/comments/4rsmzp/bash_on_windows_getting_dbus_and_x_server_working/">others
have suggested</a> (I did it by hand, not with <code>sed</code>).
You may have to repeat after upgrading Ubuntu.
</dd>
<dt>The Ubuntu version</dt>
<dd>It seems trivial, but I was unhappy that <code>diff --color</code>
didn't work. Am I shallow—color? Some of the scripts <a
href="https://github.com/binkley/shell"
title="Helpful shell scripts and programs">I write</a> for
open source provide colorized diff output, and I'd like to work on
them in WSL without disabling this feature. Microsoft made <a
href="https://blogs.msdn.microsoft.com/commandline/2016/09/22/24-bit-color-in-the-windows-console/"
title="24-bit Color in the Windows Console!">much hay</a> over
24-bit color support in <code>CMD.EXE</code>. So <a
href="#upgrading-wsl-ubuntu" title="Upgrading WSL Ubuntu">I
updated to Ubuntu 17</a>, which includes diffutils 3.5 (the
version in which <code>--color</code> was added). Microsoft does not
official support upgrading Ubuntu, but I ran into no real problems.
</dd>
</dl>
<h3 id="upgrading-wsl-ubuntu">Upgrading WSL Ubuntu</h3>
<p><em>Caveat coder</em> — there is a <em>reason</em> this is
unsupported by Microsoft at present. I just never ran into those reasons
myself. For example, I used DBus to make upgrading happier; I am not using
any Linux desktop (graphical) programs, so maybe this could be a reason.
</p>
<p>Researching <a
href="http://sourcedigit.com/22194-upgrade-ubuntu-17-04-upgrade-ubuntu-command-line/"
title="Upgrade Ubuntu 17.04 – Upgrade Ubuntu Via Command Line">several</a>
<a href="https://www.howtoforge.com/tutorial/ubuntu-lts-update-dist-upgrade/"
title="How to Update a Ubuntu LTS release to the next LTS Version (release upgrade)">helpful</a>
Internet sources, I:</p>
<ol>
<li>Edited <tt>/etc/update-manager/release-upgrades</tt> to use "normal"
releases, not just LTS
</li>
<li>Fixed <code>/etc/dbus-1/session.conf</code></li>
<li>Ran <code>sudo do-release-upgrade</code> to move to 16.10 from 16.04
</li>
<li>Re-fixed <code>/etc/dbus-1/session.conf</code></li>
<li>Ran <code>sudo do-release-upgrade -d</code> to move to 17.04 from
16.10
</li>
</ol>
<p>(Pay attention: there are many "yN" prompts were the default is to abort:
you <strong>must</strong> enter "y" to these!)</p>
<p>When I am prompted to reboot, I quit the upgrade, close all WSL terminals,
and start a fresh one. There is <a
href="https://blogs.msdn.microsoft.com/wsl/2016/04/22/windows-subsystem-for-linux-overview/"
title="Windows Subsystem for Linux Overview">no actual kernel</a>
to reboot: it remains 4.4.0-42-Microsoft throughout. The kernel is
emulated by Windows, not an actual file to boot, so upgrades only change
the packages bundled with the kernel, not the kernel itself. <a
href="https://www.microsoft.com/en-us/research/project/drawbridge/">The
underlying abstraction</a> is quite elegant.</p>
<h3 id="almost-there">Almost there</h3>
<p>Can I drop Cygwin and make WSL my daily development environment? Not quite
yet. For shell script work, WSL is excellent. But for my Kotlin, Java,
Ruby, et al, other projects, I rely on IntelliJ IDEA as my editor (though
Emacs might return into my life again). Filesystem interop between Windows
programs (such as <code>java.exe</code>) and WSL is good <a
href="https://youtrack.jetbrains.com/issue/IDEA-171457"
title="Git source control status wrong for project stored in WSL">but
not perfect</a>. </p>
<h3 id="other-options">Other options</h3>
<dl>
<dt>Cygwin on Windows</dt>
<dd>This is and has been my solution for bash on Windows for many years. I
will move to WSL when I'm ready, but I'm not ready yet. I need my
regular development cycle to work first. (See <a href="#almost-there"><cite>Almost
there</cite></a>.) There are downsides to Cygwin, for example,
coping with line endings, but it's been reliable for me.
</dd>
<dt>Homebrew on Mac</dt>
<dd>This is work. My company issues me a Mac laptop, and I use it. For the
most part, it is fine for work with colleagues and clients, though at
times the Mac is <a
href="http://appletoolbox.com/2016/07/macos-sierra-features-review/">a
little strange</a>, and much of the user experiences feels
counterintuitive. Still, the software mostly works, and the hardware
is incredibly good.
</dd>
</dl>
<p><em>But why not just use Linux?</em> Well, my daily machine at home is a
Windows box. Because it's my <em>gaming rig</em>, and <a
href="http://www.lotro.com/en"
title="The Lord of the Rings Online">games I play</a> don't run
well in Linux, and getting a Mac desktop is not currently <a
href="https://arstechnica.com/apple/2017/04/imagining-a-new-mac-pro-the-imac-pro-and-the-future-of-apples-desktops/"
title="Imagining a new Mac Pro, the “iMac Pro,” and the future of Apple’s desktops">a
pretty story</a>.</p>
<p><strong>UPDATE:</strong> More on <a
href="https://blogs.msdn.microsoft.com/wsl/2016/06/08/wsl-system-calls/">how
syscalls work</a>.</p>
<p><strong>UPDATE:</strong> Slightly dated (Microsoft is moving <em>very</em>
fast on WSL—kudos!), this is a good video presentation on <a
href="https://www.youtube.com/watch?v=_p3RtkwstNk">what happens
under the hood</a>.</p>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-61441081744923453252017-04-12T16:12:00.002-05:002017-04-12T16:24:48.195-05:00Quick diff tip, make, et al<p>I'm using <code>make</code> for a simple shell project, to run tests before
committing. The check was trivial:</p>
<pre class="code">SHELL = bash
test:
	@./run-tests t | grep 'Summary: 16 PASSED, 0 FAILED, 0 ERRORED' >/dev/null</pre>
<p>This has the nice quality of <em>Silence is Golden</em>: say nothing when
all is good. However, it loses the quality of <em>Complain on Failure</em>:
it simply fails without saying why.</p>
<p>A better solution, preserving both qualities:</p>
<pre class="code">SHELL = bash
test:
	@diff --color=auto \
	 <(./run-tests t | grep 'Summary: .* PASSED, .* FAILED, .* ERRORED') \
	 <(echo 'Summary: 16 PASSED, 0 FAILED, 0 ERRORED')</pre>
<p>It still says nothing when all is good, but now shows on failure how many
tests went awry. <em>Bonus</em>: color for programmers who like that sort
of thing.</p>
<p>Why set <code>SHELL</code> to <code>bash</code>? I'm taking advantage of <a
href="https://www.gnu.org/software/bash/manual/html_node/Process-Substitution.html#Process-Substitution"><cite>Process
Substitution</cite></a>. Essentially the command outputs inside the
subshells are turned into special kinds of files, and <code>diff</code>
likes to compare files. Ksh and Zsh also support process substitution, so
I'm going with the most widely available option.</p>
<p><strong>UPDATE:</strong></p> <p>Why are my arguments to <code>diff</code>
ordered like that? In usual testing language, I'm comparing "actual" vs
"expected", and more commonly you'll see programmers list "expected"
first.</p>
<p><code>diff</code> by default colors the <em>left-hand</em> input in <span
style="color:red">RED</span>, and the <em>right-hand</em> input in
<span style="color:green">GREEN</span>. On failure, it makes more sense to
color "actual" in red and "expected" in green. Example output on failure:
</p>
<pre class="code">$ make
<span style="color:cyan">1c1</span>
<span style="color:red">< Summary: 17 PASSED, 1 FAILED, 0 ERRORED</span>
---
<span style="color:green">> Summary: 19 PASSED, 0 FAILED, 0 ERRORED</span>
make: *** [Makefile:4: test] Error 1</pre>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0tag:blogger.com,1999:blog-5638372.post-34339812087763103032017-04-04T10:09:00.001-05:002017-04-04T10:19:08.912-05:00Maven logging and the command line<p>I usually set up my Maven-build projects to be as quiet as possible. My
preference is "<em>Silence is Golden</em>": if the command says nothing on
the command line, it worked; if it failed, it prints to STDERR.</p>
<p>However, sometimes I want to see some output while I'm tracking down a
problem. How best to reconcile these?</p>
<p><a href="https://maven.apache.org/docs/3.3.1/release-notes.html"
title="Release Notes – Maven 3.3.1">Maven 3.3.1</a> (maybe earlier)
introduced the <code>.mvn</code> directory for your project root (note
leading DOT). In here you can keep a <code>jvm.config</code> file which
has the flags to the <code>java</code> command used when running
<code>mvn</code>. Here's my usual <code>jvm.config</code>:</p>
<pre class="code">
-Dorg.slf4j.simpleLogger.defaultLogLevel=WARN
-Dorg.slf4j.simpleLogger.log.Sisu=WARN</pre>
<p>This quiets maven down quite a bit, using the properties to control Maven's
more recent logger, SLF4J. And I normally commit that into my project code
repository.</p>
<p>And for those times I'd like more output? I could edit the file, but I
don't trust myself enough not to accidentally commit those temporary
changes. So I use the command line:</p>
<pre class="code">$ MAVEN_OPTS='-Dorg.slf4j.simpleLogger.defaultLogLevel=INFO' mvn</pre>
<p>Ultimately <a
href="https://github.com/apache/maven/blob/master/apache-maven/src/bin/mvn"><code>mvn</code>
reads <code>.mvn/jvm.config</code></a>, putting the contents into the
variable <code>MAVEN_OPTS</code>, and uses <code>MAVEN_OPTS</code> in the
invocation of <code>java</code>, and you can override the variable
yourself on the command line.</p>
Brian Oxleyhttp://www.blogger.com/profile/06617364377560752378noreply@blogger.com0