Monday, September 10, 2007

Performing fixed amounts of work with blocking queues

A colleague of mine showed me a nice idiom with BlockingQueue for performing a fixed amount of work:

final List<T> work = new ArrayList<T>(N);

for (;;) {
    // Wait until there is some work
    // Get more work, if available
    queue.drainTo(work, N - 1);

    for (final T item : work)


This idiom works on 1 to N chunks at a time efficiently and succinctly.

An example use is batching SQL updates. At the top of the loop begin the batch transaction after take() and finish the batch after the processing loop.

UPDATE: This idiom is also more efficient when unbounded amounts of work:


Every call to take() is blocking and checks a lock — essentially giving up any remaining thread time slice. Using drainTo(work) avoids this lost thread work, cutting down on lock checks.

Post a Comment