The Emperor's New Prose: Duct Tape Blogging

I've been a fan of Joel Spolsky's for a long time. He has written some excellent, influential stuff. Unfortunately, his latest blog entry, The Duct Tape Programmer, falls short of the mark, and has sparked some debate throughout the web.

I wasn't going to respond until I saw supportive tweets such as 'thoughtful, provoking essay'. (Damn you, Twitter, now you've drawn me into this.)

Joel's piece isn't a thoughtful, provoking essay. This one is, but Duct Tape Programmer is just a quick rant. Like I said, I'm a fan of Joel, but let's not be sycophants here!

The Gist

I freely admit that I haven't read the book, Coders At Work, but here is Joel's article. Here's my summary of his points:

• Keep things as simple as possible, but no simpler.
• 'Simple as possible' is context dependent.
• It's the features, stupid.
  

The first item is a favourite quote of mine, often attributed to Einstein (though I have not verified that). An example in software: your webapp may not need GWT on a full blown J2EE stack, but you'll probably need a database. The first case is not simple as possible; yet without a DB, it is too simple.

His other point is that the definition of 'simple as possible' depends on circumstances: the context of a start-up company is much different than a mature app at a large enterprise. Naturally, the start-up will have a much more stringent definition of 'simple as possible'.

Finally, Joel admires those that concentrate on features, and who ship code.

The Surprise

I agree with Joel. His thesis isn't particularly original, but as I see it, it is virtually indisputable.

However, I take issue with some of the details. My point is that this just isn't a thoughtful post.

At Issue: What is Duct Tape?

Joel gives lots of examples of complex technologies: multi-threading, COM, and CORBA. Hard stuff, no doubt. He goes on to write:

... any kind of coding technique that’s even slightly complicated is going to doom your project.

However, against the backdrop of extremely complicated technologies, he doesn't define 'slightly complicated'. There's not even an example! From what I can tell there isn't even an example of duct tape!

Is OO slightly complicated? AOP? Functional programming? Transactions? Languages without garbage collection? with garbage collection? The notion of 'simple' is much more nuanced than Joel implies.

This rhetoric reminds me of the straw-man argument and definitely is the logical fallacy of false alternative. Joel assured me on Twitter that the COM example is real, and not a straw-man. I'm sure it is true (I didn't think Joel was being deceitful), but berating the most extreme case with no comment on the middle-ground does not make a thoughtful article.

At Issue: Design Patterns

Quick -- what is the most commonly used design pattern, using the vocabulary of the seminal work, Design Patterns?

I don't know, but I'd wager that it is the Iterator. In fact, if you work with Java, it is so common that it may not 'count' in your mind. And yet, there it is: a freaking commodity, no less.

Joel takes a shot at a 'Design Patterns meet-up'. It's true that people can go crazy with esoterica, but again, nuances are lost with broad strokes. It is easy to deride the architecture astronauts!

There may well be times when a design pattern is the right fit, and it is our job as professionals to be prepared. True -- we have to be intellectually honest and disciplined -- but that doesn't mean we shouldn't be informed.

At Issue: Unit Tests

Hoo-boy, Joel fired a shot across the bow of the agile ship. A brave man.

I'm a big fan of unit testing and am convinced that they helps us make better software. However, if the context is a start-up in an ultra-competitive space, and we are racing for the '50% good' mark, then I agree that unit tests would slow things down. If I were in that environment, I would shower every 2 hours just to get the smell off me.

The issue here is that most of us are not in that context, and the post implies that unit tests are used for 'endless polishing' to get to the '99% sparkling' mark. That's just bogus.

Unit tests are the first client of any software. They find bugs. They highlight problems in an API. They serve as executable documentation. They get us to the X% mark faster, where X is way higher than 50 and not as obsessive as 99. Most projects are shooting for X.

The Upshot

We all agree that simplicity and features are of paramount importance. We all agree that we shouldn't listen to architecture astronauts with high-falutin', ego-driven schemes that are not appropriate for the situation. As usual, the devil is in the definitions (what is appropriate?).

But to the Twitterverse: just because an excellent writer wrote a piece, it doesn't make it excellent. Call them out, when they have no prose, and write your own 'duct tape post' instead of broad, incendiary brush strokes.

STM and Clojure: An Article Recommendation

Your friend and mine, Mark Volkmann, has written an excellent, thorough article over at the OCI Java News Brief (I work for an affiliate of OCI).

The article first places Software Transactional Memory (STM) within the milieu of other concurrency techniques. I especially like the emphasis on transactional, as I found that as a stumbling block in the past. (It does not necessarily mean a database!)

The article goes on to work though some serious study of STM in Clojure. Great stuff....

PostgreSQL and Unicode

I ran into this recently and thought I would post it, just in case in helps someone.

In PostgreSQL 8.3, the chr() function accepts a Unicode codepoint. This is useful to insert (or in our case, correct) Unicode strings.

You might think that this feature has been around since... version 6, or 7, or surely 8.2 ?

No. Be careful: the documentation for 8.2 mentions only ASCII.

Vending Machine: A DSL in Groovy

In a previous post, I explained a modest example of the Vending Machine in Groovy. I've since taken the code and tried my hand at writing an internal DSL.

I'm a relative newbie to internal DSLs, but I've pushed my example up to GitHub (here). (This post is light on explaining the details, since the code is available.)

Observation #1

The best discovery is the magic of disappearing code. In this example, the main program simply evaporates and becomes this:

// load the DSL engine/rules
def dslEngine = new File("${args[0]}").text

// load the command input
def input = new File("${args[1]}").text.toLowerCase()

// dslEngine creates 'machine', which accepts the input:
def dslScript = " $dslEngine ; machine.accept { $input } "

// let Groovy do the rest!
new GroovyShell().evaluate(dslScript)

Amazing! There are 2 input arguments to the program. One is the DSL "engine" or context. It looks something like this:

class Machine {
   def machineState = new MachineState()

   def service(def coinList, def inventoryMap) {
      machineState.availableChange = new MoneyState(coinList)
      machineState.inventoryState = new InventoryState(inventoryMap)
   }

   def getN() { machineState.addInsertedMoney(MoneyState.NICKEL) }
   def getD() { machineState.addInsertedMoney(MoneyState.DIME) }
   def getQ() { machineState.addInsertedMoney(MoneyState.QUARTER) }

   def getCoin_return() { machineState.insertedMoney = MoneyState.ZERO }

   // snip
}

The other argument is the set of commands. For example:

SERVICE ([50, 50, 50, 50], [ [N:'A', P:'65', C:'10'] ])

VERIFY "[50, 50, 50, 50] [0, 0, 0, 0] [ [N:'A', P:'65', C:'10'] ]"

Note that the SERVICE command looks like a method call, with parentheses and a comma. That's because it is a method call. Similarly, VERIFY is as well, though no parentheses are necessary for the single string argument.

The other commands are simpler:

N ; D ; Q ; a$ ; COIN_RETURN

These are direct method/property calls as well (e.g. machine.getN()).

Observation #2

Articles on internal DSLs often talk about the contortions that one must go through to simplify the syntax for the end-user. Often, one uses techniques that would otherwise be considered poor style. (Venkat Subramaniam jokes that "designing the DSL" is "finding the right tricks").

I discovered that as well. In Groovy, it is relatively easy to have a decent DSL, but there is a never-ending desire to improve upon it. In the current version, I ran into a wall for using the dollar-sign as a token (see the compromise above: quoting that character in this post is giving Blogspot fits). Along with the parentheses on SERVICE, this pains me. I literally think about it while running on a treadmill.

Observation #3

In the first example, I adopted a file based approach for the input, over an interactive command-line. This has paid off in spades, because my suite of input files act as acceptance tests. Morphing the Java-esque example into a DSL was considerably easier with the existence of those files.

The upshot

With the right support and on the right scale, internal DSLs are terrific. For example, parts of (or, all of?) Grails and Gant are internal DSLs and I love it.

On a smaller scale, I'm not so sure. I'm still disturbed about the issue of Domain Specific Error messages. That is: can a domain expert (without development skills) really handle the power of an internal DSL (including the errors)?

Either way, internal DSLs are undeniably a fun exercise and a great way to learn more about a language.

Groovy Console App and WIDE

In April 2008, I blogged about the WIDE: web-enabled IDE. I wondered if web tools might augment some of the standard stuff available in the IDEs.

(I realize many have 'scratchpads' but I wanted something more. Also, by environment, I don't mean a single app).

In particular, I've thought about a website dedicated to string and regex utilities. Whenever I find myself writing a little script to parse a data string, or to toy with a regex, I always think "there must be a better way".

Well, we're one huge step closer: check out the Groovy Web Console, by Guillaume Laforge.

It laughs at basic string utilities:

def s = "does this string fit into a 32-char column?"
println s.size()

And provides a test-bed for regular expressions:

(Note, this is a Java-Groovy hybrid. It's only somewhat 'Groovy'. Making it Groovier is left to the reader)

import java.util.regex.Pattern

def s = "1 - 314-867- 5309"
def p = Pattern.compile(/.*1.*(\d\d\d).*(\d\d\d).*(\d\d\d\d).*/)
def m = p.matcher(s)
if (m.matches()) {
  println "area code = ${m[0][1]}"
  println "exchange = ${m[0][2]}"
  println "digits = ${m[0][3]}"
}

Count me in... I'm definitely going to have this site at hand in my environment.

The IDE is a Browser: Neuroscience and Language Design

Lizard Brain Web Design

Last winter, I saw a great talk by Scott Davis called "Lizard Brain Web Design". The theme was to apply psychology and neuro sci ideas to web sites, and to explain why simplicity and good design can really work. For example, we want the site to stay "out of the way" so that the users stay in a primal, "lizard" mode of consciousness with respect to the site. In this way, they can concentrate on what matters.

During the talk, I remember thinking that all of the principles discussed apply to more than surfing content on the web. They also apply to surfing code in an IDE. That is, topics such as:

• Whitespace is a critical aspect of design
• Group related items (locality of reference)
• Our minds can only stack N items (N = 7 ?)
• Principle of least surprise

apply just as well to our APIs, our code organization, and coding conventions.

For months now, I've wondered if there were studies that applied neuro sci to developers.

Subitizing

Let's play a game: as you add a parameter to a method, how many parameters triggers your sense of "this is too many -- I need to refactor this".

Seriously, go ahead, think of a number, N, for your threshold.

You probably said N = 3 or 4. True, that's what everyone says, but here is one reason why. The delightful book Mind Hacks discusses subitizing (item #35): given a set of N objects, where N is 4 or less, we process counting in a much faster way. The book claims 250 ms for the first 4 items and a full second (!) for every 4 items after that.

There is debate as to how this works (see Mind Hacks for academic references), but one conjecture is that when N <= 4, the "counting" is a side effect of visual processing: i.e. it is done by the lizard, reptilian level of the brain. When N goes past 4, we have to do some work.

Now, let's be clear: the book talks about counting shapes. Stars, circles, beads on an abacus. I have no idea if this applies to Java or C# parameters.

But I'm willing to bet money that it does.

Eye-Tracking and Variable Names

This article came across the transom recently, and dovetailed with the above ideas. The gist is that the researchers used scientific techniques (e.g. eye-tracking) to evaluate productivity of programming styles.

The claim in the paper is that the Scala style of using comprehensions is more productive than Java's iterative loops. Also, for small code blocks, well-named intermediate variables may not matter.

I didn't read the paper, and I have no idea of the validity of the science. However, I find the approach to be very fascinating. I'm sure scientific methods have been used for a long time with respect to lines of code, and productivity, but I wonder if neuroscience will have a future impact on language design?

It would be fascinating to see if researchers start hooking up developers to functional MRI machines, to see how the brain works while coding. (I know that my amygdala lights up when I see a 80-line method!)

The Upshot

Imagine a geek conference where a new language is unveiled: instead of its design being driven by a sense of tradition or aesthetic, what if its design was modeled on hard evidence from a neuro lab?

Neat stuff.

An Understated Feature in Groovy

Background

I have worked with Groovy for awhile, but have been averse to one feature: the implicit return of the last expression evaluated. For example:

def getFoo() {
    def foo = null

    if (something) {
        foo = new Foo()
    }

    foo // no 'return' necessary
}

In the above code, I would use return because it felt safer to me. I tended to agree with Eric's post (with respect to explicit returns).

Revelation

I get it now, and it's all thanks to the enhanced collection methods in Groovy.

Consider the collect method. It transforms a list by applying a function. In pseudocode:

[ a, b, c, d] => [ f(a), f(b), f(c), f(d) ]

With that in mind, here's another example:

class Composer {
    def name
    // def era, etc
}

def list = [ 'Bach', 'Beethoven', 'Brahms' ]

// the closure returns the new Composer object, as it is the last
// expression evaluated.

def composers = list.collect { item -> new Composer(name : item) }

assert 'Bach' == composers[0].name

The idea is simply to build a list of Composer objects from a list of strings. As noted in the comment, the closure passed to collect uses the implicit return to great effect. At one time, I would have taken 3-4 lines to express that idea.

But now, this code is not merely concise: it is truly elegant. Very reminiscent of other languages such as Python.

The Take-Home Message

There are many lists of excellent Groovy features. However we rarely see 'implicit return' listed. I'm a fan: it greases the wheels for other features.

I realize it is available in many languages: I just haven't used it in Groovy. I suspect in a few weeks I won't be able to live without it.