Popularizing Erlang

Posted: June 20th, 2007 | Author: kevin | Filed under: Erlang | View Comments

I’m definitely more of a backend systems guy than a UI designer. This might explain why I’m still using a Dell laptop running Linux rather than a shiny, sleek Apple MacBook like most of my friends. My interests tend to run towards networks, distributed applications, and databases. I’d rather spend my time reading about, say Plan9 or CouchDB than the latest Ajax tookit or CSS hack.

So, Erlang is just about the best language for me. It offers most of the features I need to write scalable, distributed applications. The _only_ thing I’d like to change about it is string handling. The fact that Erlang represents strings as byte arrays gives me the heebies. It’s not caused me any problems with the bit of hacking I’ve done so far, but it somehow just feels wrong. A little voice in the back of my head keeps saying “byte-based strings are gonna hurt someday”.

Back to my main point. I read this post and my first reaction to it is that it’s a nice idea motivated by a sincere desire to move Erlang forward. I have a tremendous amount of respect for programmers willing to throw wild ideas out to the world. As a learning exercise, I think Erlectricity is a fine idea. But I think this is exactly the wrong approach to popularize Erlang for a couple of reasons.

First off, the only real way to get more developers using Erlang is to get more developers writing Erlang. Using Ruby for all the heavy lifting and Erlang for just bits n’ bobs relegates Erlang to the “backend systems” ghetto and erects obstacles to finding truly innovative ways to use Erlang. Imagine if, before Ruby was a hit, a JNI bridge between Ruby and Java became popular. Java has a lot of libraries, arguably more than Ruby, even today. The argument could’ve been made that leveraging all those Java libraries made good sense. How would that have impacted Ruby library development? Would we have had more Ruby->Java wrappers than native Ruby libs? I think so. The point is programmers need to use and get comfortable with whatever language their trying to adopt, in this case Erlang. Otherwise, you get Ruby developers hacking, in the pejorative sense, Erlang instead of Erlang developers hacking, in the good sense,  in Erlang.

Second, there is a severe performance and scaling mismatch between Ruby and Erlang. It’s common knowledge that Ruby is slow. It’s slow for a number of reasons including a lack of native threading and a global interpreter lock allowing only one Ruby thread to execute at a time. Erlang lives and breathes for highly concurrent environments, like web applications. See Joe Armstrong’s post for an example on how well Erlang scales. I just can’t see how to interface the two languages that doesn’t result in a major performance bottleneck. At the very least you’ve got marshalling overhead between the two. And when the performance differences are factored in along with the single-threaded nature of Rails (the worlds most popular Ruby framework) it gets really gnarly.

My idea for popularizing Erlang is simple: write the code needed to make Erlang more commonly accepted. I’m not sure what that code needs to be, though I’ve got a few ideas. But I am sure that it’ll be written in Erlang .


  • Amr
    Sounds like a great idea! have you published any such code. I would be interested in any erlang work being done in the systems/admin space.. just curious as to what is possible..
  • Jan
    Heya Kevin,
    if you want to get your hands dirty with Erlang and
    help out with CouchDb, drop us a line! My contact
    info is on my website, Damien's at his'; damienkatz.net

    Cheers,
    Jan
    --
  • There's nothing about Erlang (other than the quoted-string syntactic sugar) that makes it impossible to implement Unicode-handling libraries that operate on binary data types.

    As a server-side person myself, one of the important tools I see as being necessary (Mnesia's existance notwithstanding) is something like what JDBC is to the Java world: connectivity to tradional relational databases. I don't think that'd be a super-hard thing to do, but it wouldn't be trivial.
  • Jason
    Regarding strings being stored as lists of integers (not bytes) in erlang: Yes this is not as nice as some other languages but the tools are available to convert these lists into various unicode representations (see xmerl_ucs.erl in OTP) and other character sets (see libiconv bindings in jungerl) should the need arise.
  • @jerith - You're completely correct. I was thinking about Ruby and Erlang when I wrote this and I confused the two. I guess the lack of baked-in Unicode support still makes me a bit uneasy regardless of how strings are stored. It hasn't hurt me yet and I can't really point to any reasons why it might, but I can't help but think that it might someday.

    @evgen - Total agreement. Erlang needs practical, useful code. My impression is that there's quite a bit of useful code available now but it's not really publicized. So I guess I see the job as twofold: a) Talk about the useful bits of Erlang available today and b) write more useful bits for the future.

    Hmmmm...I really like the idea of picking packages and just porting them straight to Erlang.
  • evgen
    One of the best things that Erlang evangelists can do is not just to write Erlang code, but to write Erlang code that will help other people get things done. Python, Perl, and Ruby all have a large collection of useful modules available that makes these languages easy choices for most problems. Erlang hackers need to pick a module/package of the week and port it.
  • Erlang's strings are lists of integers, not lists of bytes. This means you get multibyte characters for free. They're still lists, though, which means you can use the standard lists:foo() functions on them.

    I'm still not sure whether a new string type would be better, but I haven't run into any issues with strings that a simple flatten() didn't fix.
  • Ack, due to a copy-n-paste error, I didn't give the right URL link in my previous comment to fuzed. This one links correctly.

    Sheesh, now I've linked a twitter spammer I was reporting. :\
  • You should check out the Erlectricity library.

    I've been working on a library called Fuzed which uses YAWS and erlang concurrency and distribution to serve out Rails apps, using Erlectricity as the glue to Rails. Honestly, even with our pure-ruby marshalling the overhead is next-to-nothing. Maybe 2ms, maybe up to 5ms under load. This might change with very large datasets.

    So far, Erlang for the infrastructure and ruby for the string manipulation has been a great fit.
  • That's true but many more people use CRuby than JRuby, so the GIL is still an issue. FWIW, Python also has this same problem. See CPython and Stackless Python (http://www.stackless.com)
  • Behrang
    Regarding the global lock, it is not necessary always there. For example, there is not such a lock in JRuby.
blog comments powered by Disqus