Erlang Pattern: The Router

There are three components to this pattern. The client, the router, and the target. The client begins the process by calling the router with a message destined for a target. The router uses the message, and possibly other metadata, to locate a desirable target. Then the router hands off the message to the selected target. The target replies directly to the client and completes the process.

I’ve written this code a number of times in several different languages. What makes the Erlang implementation so nice is it’s brevity and simplicity. Using the gen_server behavior I can implement the core logic in just a few lines:

1. Client calls the router with a message destined for a target:
   gen_server:call(?ROUTER, {invoke, Target, Msg})

2. The router looks up the target and forwards the message on:
   
handle_call({invoke, Target, Msg}, From, State) ->
%% Target selection code goes here
gen_server:cast(TargetPidOrName, {From, Msg}),
{noreply, State};


3. The target replies directly to the waiting client:
   
handle_cast({Originator, Msg}, State) ->
%% Server logic goes here
gen_server:reply(Originator, Reply),
{noreply, State};


The pretty bit is that all this plumbing can be hidden away inside a function. So the original call to the router in the client winds up looking like this: router:invoke_target(Target, Msg). The rest happens behind the scenes and appears as just another gen_server call.

I’m pretty sure there are no bugs in using this approach. I’ve benchmarked a recent implementation of this pattern at over 6000 requests/sec without a hiccup.

CodeWriMo

NaNoWriMo is an event run over the Internet to help aspiring authors write a 50,000 word novel in a month. I was *this* (imagine two fingers held closely together) close to being an English major in college so I’ve got a built in soft spot for authors. I’ve wanted to participate in NaNoWriMo for the past few years but the timing has never quite worked out.

It’s almost NaNoWriMo time this year and it got me thinking about having a similar event for developers. For as long as I can remember code has been the way I’ve expressed my creativity and I know that’s true for lots of others. But it’s also the harsh reality for most that the “day job” is rarely your creative outlet. So you’re stuck trying to cram in a few hours a week of creative hacking in amongst life’s other demands.

What if you took a month and added a sense of purpose to your after-hours hacking? What if you set a few goals and had a group of like-minded people to lend moral support when you needed it?

My idea is this: Start a new open source project from scratch and go to initial release in a month. Working code isn’t enough, though. The release would also include docs and examples. The key is to pick an idea that’s reasonably do-able in 3 weeks’ of hacking leaving an additional week for polish and docs.

Is this something worth doing? Ping me on twitter (@kevsmith) or leave a comment and let me know what you think.

Tomorrow Night

I’ll be speaking tomorrow night at a joint meeting of DC ALT.NET, Novalang, and Arlington Erlang Users’ Group. I’ve got two full hours to fill so I’m planning on covering a lot of ground. I’ll be leading off with a gentle but speedy introduction to Erlang followed by blatant evangelism on why webmachine is the finest framework for building REST resources EVAH.

If you’re in the area, please consider coming out. You can reserve a spot here.

Thanks to Matt, Luc, and Chris, for organizing the meeting and having me up to speak. I’m really looking forward to meeting everyone and spending an evening talking about one of my favorite subjects.

CouchDB Benchmarks: gcc vs. clang

Since the release of Snow Leopard, I’ve been itching to run some benchmarks. Jan Lehnhardt suggested I give the benchmarking script from this bug a few runs. So that’s what I did.

At first glance testing seemed pretty simple. Compile one version of Erlang with gcc and another with clang. Then use each of those versions to compile CouchDB and run benchmarks against each Erlang/CouchDB pair. Since CouchDB also has at least one bit of native code, couchjs, I’d also used gcc and clang to compile that, too.

Then reality showed up and ruined my day. Compiling CouchDB on OS X is, well, a pain in the ass involving MacPorts and lots of dependencies if you do it wrong. Doing it right means grabbing Jan’s couchdbx-core project from Github and building new guts for CouchDBX. This is the way to go, IMHO. 10 – 15 minutes’ of compiling gave me a new Erlang, SpiderMonkey, and CouchDB I could drop into the Contents/Resources/couchdbx-core directory of my previously installed CouchDBX app.

Compiling with clang is similarly easy. All you need to do is set the environment variable CC to point to /Developer/usr/bin/clang. I found I also needed to set CFLAGS to -Qunused-arguments to get Erlang to compile.

The benchmark itself consists of three phases. The first phase creates a database and populates it with 40,000 documents. The second phase creates and executes a complex view named 'megaview' to really exercise the view collation code. The last phase creates and executes a simple view named 'simpleview'.

Let me quickly describe how the performance numbers were collected. The benchmark was executed 5 times against the clang and gcc versions of CouchDB. I've computed a raw average across all runs as well as a "normalized" average which throws out the highest and lowest times. Finally, I computed the standard deviation as a indicator of overall variability.

Enough talk. Here are the numbers:

Note: Document insertion times are given in seconds while view collation numbers are given in minutes

The megaview test really highlights the disparity between clang and gcc. It's worth noting the test also showcases a bottleneck in CouchDB and only used a single core on my laptop. If I had to guess I'd say the performance disparity is a function of the age of each compiler. gcc has been around forever so it makes sense it's had more time to implement more/better optimizations than clang.

September’s Speaking Gigs

I’m speaking at two events in September. First, I’m making the trek to DC to speak at the 9/10 combinedmeeting of the Erlang Users of Arlington/DC, Alt.NET, and NovaLang groups to talk about web development with webmachine and Erlang. I’d appreciate any tips on decent and inexpensive hotels in the area and reasonable places to co-work for the day. Of course, if anyone would like to have a resident Erlang developer at their offices for the day I’m up for that, too :-)

Then, on 9/12, I’ll be introducing Erlang to open source hackers at the ominously named Evil Robot Conference, organized by my friend and former Red Hat colleague Michael DeHaan. Evil Robot is shaping up to be a kick-ass eclectic mix of technologies and speakers. I’m personally looking forward to hearing about btrfs and UnCommon Web.