Thursday, 8 September 2011

Comparing Go with Lua

The Go language developed in the last few years by Rob Pike and Ken Thompson from Google has gathered a lot of interest, particularly for its built-in concurrency support. Anything released by these gentlemen (who did the first Unix and later Plan9, which introduced UTF-8 to the world) is naturally going to cause a stir, since it's a little like a new album by Led Zeppelin. And unlike the reappearance of most rock dinosaurs, Go does not disappoint.

Most blogs naturally focus on the concurrency support, but here I'd like to point out some striking similarities with Lua. It may be convergent evolution, much as sharks and dolphins look similar to non-zoologists, or it may be a case of unconscious influence.

Both languages have anonymous functions, and these are proper closures, which is a fantastic thing to have. In Lua, closures reduce the need for classical classes enormously. This Go code works like the equivalent in Lua or JavaScript - the function will keep its own reference to name when it is later called.

 var name = "dolly"
 DoSomethingLater(func() {

But that is just common sense if your functions are first-class values, and applies to Scheme and Javascript as well (Scheme is an acknowledged influence on the design of both Lua and JavaScript). Go also has coroutines, although 'goroutines' are more than the single-OS-thread cooperative-scheduling model of Lua, being scheduled as OS threads whenever needed and having a language mechanism called channels for communicating between them. This is an area of weakness for Lua, where multi-threading can be an exercise in masochism.

But the big similarity is this: functions can return more than one value. This is quite distinct from returning tuples, as Python does and is very much more efficient. (Bruce Eckel errs in speaking of tuples in Go) So the following style is common: a function returning a value plus an optional error:

 var f, err = os.Open(filename)
 if err != nil {
     return "", err

Although statically-typed, Go uses static type inference to make life easier for developers so we don't have to remember the exact types (*File,os.Error) of these values.

The whole exception-handling movement was a reaction against the sorry state of error handling. A C function like fopen will return NULL on error, but then you have to look up the actual error in errno which often was implemented as a global variable. Not cool in any way! This was replaced by a mechanism in which a function could return anything it liked, but also return an error non-locally. The vision was that you could then write code in an optimistic way, the so-called 'happy path' and put it all in a try block and catch the exceptional cases.

Things fail all the time; not being able to open a file is not really exceptional. Handling multiple return values makes it possible to go back to an older way of dealing with errors directly, but this time doing it properly with the means to return both results and errors cleanly.

The equivalent Lua idiom is similar, practically identical if you regard curly-braces vs keywords as superficial syntax differences:

 local f, err =
 if err ~= nil then
     return "",err

Both Lua and Go generalize this to statements like this, which swaps two values:

 x,y = y,x

They both allow multiple return values to be consumed by another function which takes the same values as arguments:

 func sum (x, y int) int {
     return x + y
 func multiple () (first, second int) {
     return 10, 20
 s = sum(multiple())  // == 30

Lua goes a little further and will add the multiple returned values to the end of the argument list of a function; that is, this will also work:

 hello  10  20

Go and Lua provide mechanisms for protected calls which are similar in spirit. Something bad will happen in a function, and the idea is to prevent that badness from bringing down the process.

For go, the panic() function throws an error, which can be of any type; the recover() function allows you to stop that error from propgating. For instance, an example from the language specification: this function is passed a function of no arguments and guarantees that any panics happening in the function will be caught:

 func protect(g func()) {
     defer func() {
         if x := recover(); x != nil {
             log.Printf("run time panic: %v", x)
 protect(func() {
     panic("something bad happened")

The defer mechanism is very cool (and Lua could benefit from something similar); it is used to ensure that an expression is always executed, no matter how we leave the function (a common use is to close a file safely, e.g. defer f.Close()). g() executes and panics, which calls all the deferred functions until we get to the one defined in protect, which uses recover to get the original value passed to panic.

The equivalent in Lua is error() and pcall(), which stands for 'protected call':

 local ok, err = pcall(function()
     error("more bad things!")
 -- ok will be false, err will be the string

The strength of the Go approach is that there is an explicit mechanism for calling arbitrary code when a function stack frame goes out of scope. With Lua (as with most garbage-collected languages) the best you can do is define finalizers on objects and wait for the garbage collector to call them (eventually.)

To generalize protect above, you need to use reflection to call the function; see this implementation. With that, apart from arbitrary calls like pcall, it's possible to construct an equivalent to try/catch in Go:

 P := new(Protect)
 s := make([]int,3)
 if ! P.Try(func() { // try execute some code
   println("never get here")
 }) { // and catch...
   println("error:",P.Error) // runtime error: index out of range

Continuing with the list of resemblances: both languages use _ in similar ways. Compare:

 func sumv(values []float64) float64 {
     sum := 0.0
     for _,v := range(values) {
         sum += v
     return sum


 function sumv(values)
     local sum = 0.0
     for _,v in ipairs(values) do
         sum = sum + v
     return sum

The difference is that with Go you have to use an underscore if you want to ignore the value (it is very strict about unused variables) whereas in Lua it is merely a convention. Note also the similar for-iterator syntax which naturally flows out of multiple return values.

Compare the way that the languages attach methods to a type:

 type Map map[string]int
 func (self Map) set(key string, value int) {
     self[key] = value
 var mm = make(Map)

Compare this to:

 Map = {}
 Map.__index = Map
 function Map:set(key,value)
     self[key] = value
 local mm = setmetatable({},Map)

There's a little more ceremony with the Lua example, which does not directly support 'classes', but the style is similar. Types in Go and Lua are always 'open' in this way, although obviously this freedom has to be used in a disciplined fashion.

Update: Thanks for Gé Weijers for pointing out the influential CLU programming language, developed at MIT in the Seventies by Barbara Liskov and her students. Lua's multiple returns came from CLU according to The Evolution of Lua. It is interesting how long it can take for new programming language concepts to reach the mainstream.

I've taken out an over-generalized paragraph about languages and the restrictions they impose: Ruben Thomas' comment seems fair but hinges on the definition of 'restriction' which is an important topic, but not particularly relevant to this comparison.

Finally, Rob Pike has answered the question: it's convergent evolution. (He likes the classic rock revival reference.)

(The updated script for generating pretty HTML for Blogger now has support for multiple languages in the same document.)


  1. Interesting article, then this old canard: "It is better to have as few restrictions in a language as possible and shift the responsibility for good code back to the developer."

    Well, no. The point of restrictions in a programming language is to try to stop the developer thinking about things which waste their time. Automatic memory management is a good example of such a restriction (and something that doesn't seem to worry you). Static type-checking is another.

    On the other hand, such restrictions often make a language more verbose, and hence more painful to program in, typically because the programmer must repeat information. Innovations like type inference help avoid this.

    In languages like Scheme and Haskell there is a tradition of removing restrictions from the language, which has the effect of making more syntactically-correct programs into valid programs (in Scheme, typically this means running without a run-time error; in Haskell, compiling without error), and again increases the expressivity of the language (which is another way of saying "decreases its verbosity"). Various attempts at codifying parallel programming, such as goroutines, purposely reduce the generality of what has proven an extremely difficult area for programmers to grasp, so that they waste less time in errors, while still being able to write useful code.

    The point of all these measures is to shift the burden of thinking about certain things from the programmer to the machine. This frees up the programmer to think about other (hopefully appliction-oriented) things. This is exactly what making languages "higher-level" means. It's nothing to do with being grown-up or taking responsibility for one's code: programmers always have responsibility for what they write. Rather, it's about using the human brain's finite capacity better.

    At bottom, the idea of having "as few restrictions in a language as possible" is laughable these days: all modern languages have many strong restrictions, and those lacking in them (C, C++) are only bearable with heavy-duty static analysis tools (and even then, productivity is much lower than in higher-level languages).

    The author of "Real Programmers Don't Eat Quiche" would have a good laugh reading this article about two high-level, highly restricted languages and then getting to that coda!

  2. I suppose I don't think of something like automatic memory management as a restriction, but something that does enable the programmer to move on and solve the problem at hand. Likewise not being able to access physical memory directly.

    Actually, I regret that last paragraph because it's a classic bit of editorializing. I was mostly thinking of something like Java's "everything must be a class", i.e. an almost religious design decision which restricts the programmer from choosing more appropriate solutions.

  3. Javascript also allows multiple return values and value swapping for variables. It just makes this "array-like" return look like an array.

    var a, b;
    function multipleReturn() {
    return [1, 2];

    [a, b] = multipleReturn();
    [a, b] = [b, a];
    console.log([a, b]);

    You can copy-paste that into Firebug and you'll get [2, 1] like you expect. This doesn't work in IE or Chrome because it requires Javascript 1.7 and only Gecko implemented these extensions, unfortunately.

  4. Minor comment. The Go language is actually *being developed* by a team in Google which *includes* Rob Pike and Ken Thompson. There are many more who have contributed.

  5. Often ignored as of late, yet still my favorite -- but many of the above attributes apply to perl5 as well :) . Though really that's the beauty of modern programming languages in general, we all steal the good ideas from each other eventually!

  6. #DavidEllis: that's indeed a very cool feature; JS continues to shape up as a good language (I know some will be horrified by this statement). One day I will finally understand what 'this' means ;)

  7. #Rohit: yes, bad journalism. Should at least mention Andrew Gerrand here.

    They are very good at acknowledging the contributions of others as well:

  8. I'd like to point out that methods can only be put on types that are declared in the same package, so it isn't true that Go types are *always* open. This prevents type method sets from becoming unregulated global namespaces, i.e. hell. You can, however, always define your own type that adds methods on top of the original type. Then you can explicitly choose which method sets you want to use.

  9. #steven099: well, that's a relief! So 'open' in the usual sense of inheritance.

    With Lua (as with Ruby) everything is always open, although the Lua community is less enthusiastic about modifying classes and generally does not consider monkey-patching to be a good idea.