A Question of Notation: Revisiting Moonscript

Growing Up Nicely

Since the last time I reviewed Moonscript here it has matured nicely in ways that make it easier to use. Some nasty gotchas (like a[i-1] being misinterpreted) have gone away and there is better error reporting.

It has found its way into my toolbox for quick utilities and prototypes that I don't need to share with others. (That may change; my colleagues know Python, not Lua, and I suspect that they will find Moonscript easier to read.)

I hope to make the point that even people who use Lua and don't wish to bet on an 'experimental' language can benefit from learning a little Moonscript, since it makes an excellent notation for expressing Lua programs. In particular, its terse syntax is well suited to interactive exploration.

Get Interactive

Out of the box, there is no REPL, but writing a sufficiently-good one was not difficult. mooni was the result. It does not try to solve the tricky problem of when to start a block; you indicate this by ending a line with a backslash.

moon-article$ mooni
MoonScript version 0.2.3
Note: use backslash at line end to start a block
> ls = {10,20,30}
> ls
{10,20,30}
> m = one:1, two:2
> m
{one:1,two:2}
> for x in *ls \
>>  print x
>>
10
20
30

Moonscript works with tables in the same way as Lua, except that the more conventional colon is used for associative key-value pairs. You don't always have to use curly brackets for map-like tables (e.g. the assignment to m above). Since all statements in Moonscript can have a value, we don't need any special way to indicate that a expression is being evaluated. mooni also does pretty-printing of tables.

A language with no libraries is a no-starter, but a language which is essentially a new notation for an existing language starts off with a ecosystem. For instance, we have Penlight available.

> require 'pl'
true
> utils.split 'one two three'
{"one","two","three"}
> utils.printf "hello '%s'\n", 'world'
hello 'world'
> utils.import math
> cos(pi/8) + sin(pi/2)
1.9238795325113

Function calls don't need parentheses, except when you need to force a function call to bind with an argument - in that case, the opening paren must not be separated by space from the function. (A more formal way of stating Moonscript's semantics here is that the call operator has a much lower precedence)

This sensitivity to whitespace takes a little getting used to, since Lua has practically none, but the payoff is that most function calls require fewer keystrokes, which matters in interactive mode.

The first great thing about an interactive mode is that the beginner has a chance to try statements out one by one, and gets their rewards ('it works!') and punishments ('why did that not work?') in little incremental steps.

The second great thing comes from the fact that we are often beginners; testing out a new library, exploring an API, trying out one-liners before inserting them into thousand-line programs, etc. So even if you are a Lua programmer, using a Moonscript REPL will allow you to experiment with your Lua modules in a less tedious way.

The function notation is very compact, which makes a functional style more pleasant:

> -- 'return' is implicit here
> sqr = (x) -> x^2
> sqr 10
100
> -- functions of no arguments do not need argument lists
> f = -> 42
> f()
42
> add = (x,y) -> x + y
> -- partial application
> add1 = (x) -> add x, 1
> add1 10
11
> ls = List{'alpha','beta','gamma'}
> -- mapping a function over a Penlight list
> ls\map => @sub 1,1
{a,b,g}
> ls\filter => #@ > 4
{alpha,gamma}

The fat arrow is short for a function with an implicit self; @ is a shorthand for self.

Everything is a Value

The use of indentation to indicate blocks is now firmly associated with Python, so people with a Python background might feel superficially at home. But consider this:

> f = -> 10,20
> {f()}
{10,20}
> if 0 then 'ok'
"ok"
> if 1 > 2 then 'ok' else 'nope'
"nope"
> ls = for i = 1,3 do i
> ls
{1,2,3}
> [i for i = 1,3]
{1,2,3}

As in Lua, functions are first-class values, and they can return multiple values. (In Python there is the illusion of multiple return values, but really they are packed into a tuple and then unpacked in any assignment, which is a lot less efficient). If such a function is evaluated as the last value in a table constructor, all the values are captured. This is all standard Lua semantics. The if construct should come as a surprise to both Lua and Python users, since it returns a value. (The gotcha for a Python user is that 0 is not false; only false and nil evaluate as false in Lua)

for statements are expressions that collect their values into a table, only if they are in an assignment context. So they don't needlessly create tables! For this simple task, list comprehensions are better, but consider this example from the manual:

doubled_evens = for i=1,20
  if i % 2 == 0
    i * 2
  else
    i

(The general rule is that the block versions of if,for and while leave out the then or do keyword. There is no end keyword!)

The with statement works like the similar statement in VB or Pascal; a dot is needed to indicate the fields that will be set within the table. And it's no surprise that it returns that table as a value:

> with {} \
>>  .a = 1
>>  .b = 2
>>
{a:1,b:2}

This gives us a really cool way to write modules, because (again) the value of loading a file is the value of the last expression.

-- util.moon
with {}
    .greeting = (name) -> print "Hello ", .quote name

    .quote = (s) -> string.format('%q',s)
----------
> u = require 'util'
> u.greeting 'Dolly'
Hello     "Dolly"

(Note how we can use the dot for reading fields as well as writing them.)

Doing Programs Bit by Bit

require in Moonscript works just like in Lua, except it will load any *.moon files on the module path as well. But require is not so useful for incremental and interactive development, because it will only load once and cache the result. Which is why we will rather use dofile for this purpose - but the global dofile from Lua and only loads Lua scripts. It is easy to make a Moonscript-aware dofile using the built-in moonscript module.

> moon = require 'moonscript'
> dofile = (x) -> assert(moon.loadfile x)()
> u = dofile 'util.moon'
> u
{greeting:function: 0xfd9610,quote:function: 0xfbc140}

So now it's possible to reload a module and try out the changes. The finger-friendly syntax makes interactive use easier. If I had a module which controlled a robot, then Moonscript provides a nice command prompt:

> turn left
> speed 2
> obstacle -> speed -2

Now this isn't such an imaginary scenario. PbLua is a Lua port for the Lego Mindstorms NXT kit, and it has a Lua prompt. Getting Moonscript to work with pbLua does not require that the micro actually runs Moonscript! A custom mooni could translate everything into plain Lua and push that up, ditto for scripts.

This point needs emphasizing - moonc compiles Moonscript to Lua. The Lua environment that then runs that code could be stock Lua 5.1, 5.2, LuaJIT or whatever. The decision to use Lua as the intermediate language has given us a lot more flexibility in applications.

In mooni, if you want to see what Lua code was generated by the last executed statement, use this common expression of puzzlement:

> t = one:1, two:2
> ?que
t = {
  one = 1,
  two = 2
}

Making up New Constructs

For instance, it seems self-evident to most people that a modern language should have syntax for exception handling. Coating Lua's pcall in some convenient sugar is very straightforward:

try = (block) ->
    ok,err = pcall block
    if not ok then err\gsub '^[^:]+:%d+: ',''
    else nil, err

test = (foo) ->
  err,val = try ->
    if foo then return 2*foo
    print a.x
  if err
    print 'error!',err
  else
    val

print test nil
--> error!  attempt to index global 'a' (a nil value)
print test 2
--> 4

There is still an issue if the function did want to return nil, but I'll leave the resolution of this to any interested parties. (hint: use select)

This kind of thing has been possible in Lua for a long time now, but people get put off by the necessity for (function() ... end) here, and anywhere where we need a lazy way to get 'lazy evaluation'.

For instance, when working with many GUI toolkits it's useful to schedule an action to be run later on the main thread. This could be expressed as later 300,-> do_something(). GUI toolkits are all about firing events; for instance in AndroLua one can create a button and attach it to an action in two lines:

@button "Please Click Me!",->
    @toast "Thanks!"

The equivalent Java code is a lesson in how boilerplate obscures otherwise straightforward code, and explains why Java simply has to get lambdas to compete.

Moonscript's syntax can play nicely in the niche established by Ruby. For instance, this is a rakefile.

task :codeGen do
  # do the code generation
end

task :compile => :codeGen do
  #do the compilation
end

task :dataLoad => :codeGen do
  # load the test data
end

task :test => [:compile, :dataLoad] do
  # run the tests
end

And here is the equivalent lakefile for Lake

-- lakefile.moon
task = target

task.codeGen nil, ->
    print 'codeGen'

task.compile 'codeGen',->
    print 'compile'

task.dataLoad 'codeGen',->
    print 'dataLoad'

task.test 'compile dataLoad',->
    print 'test'

-- without any explicit targets, lake fires this ....
default 'test'

moonc lakefile.moon would creae lakefile.lua, which lake understands. If anything, the syntax is even cleaner - I've cheated slightly by passing dependencies to the targets as space-separated strings; they can also be written as tables like {'compile','dataLoad'} which is the internal representation anyway.

Imagine a hypothetical environmental monitoring system:

rule.too_hot -> temp > 37 and humid > 80
--- handle the rule asynchronously...
If.too_hot -> print 'get them out!'

Which suggests that if I were designing a DSL (Domain Scripting Language) for such a rule-based application then my users might find Moonscript easier than plain Lua. (Embedding Moonscript in an application is not difficult, since it's mostly Lua code with dependencies on LPeg and LuaFileSystem. The Windows binary has already compiled this all into a DLL.)

Tooling and Documentation

This is something that separates the goats from the sheep when evaluating a new language. Fortunately, leaf has provided editor support - the repackaged SciTE is a good choice for Windows users. You will probably have a better experience if you edit the configuration file `Options|Open Global Options' and put these lines at the end:

split.vertical=0
open.dialog.in.file.directory=1
statusbar.visible=1

Assuming that you do want other people to use your modules, it helps to have a documentation tool that understands the language. This simple List class has basic LDoc markup. Note the use of the @within tag to put the metamethods in their own section.

The output from ldoc -f markdown -S List.moon is here.

(This is all hot off the presses so you'll have to grab from the LDoc master. I'm considering whether metamethods should automatically be put into their own section by default.)

Differences and Distinctions

Moonscript is compiled to reasonably straightforward Lua, and its detailed semantics are a superset of Lua so it finds easily into the existing Lua ecosystem. The surface syntax is different, but comes from several design choices

• indentation is syntax - no more end
• : for key-value pairs in tables; purely map-like tables often don't need curly brackets
• line ends become significant, so commas are not needed in multiline tables
• function(self,y) becomes (self,y) -> or (y) => depending on taste
• function calls have very low precedence, hence less parens needed
• every statement can return a value; explicit return is usually not needed
• local-by-default means local is only needed to make scoping explicit
• there is sugar for list comprehensions, classes and extended assignment operators like += and *=. != is a synonym for ~=

In other words, it is a determined attempt to reduce the typing needed for common operations in Lua, at the cost of more rules. This makes it a good notation for interactive work, even if your work remains mostly in Lua.

Could a person learn Moonscript without knowing Lua? No reason why not, but it will require a good self-contained tutorial and there are more syntactical gotchas. It could make a good educational language, since there you do not necessarily want a language that some of the class already know; familiarity breeds conplacency, if not actual brain damage (as Dijkstra asserted about Basic.)

Moonscript is available using LuaRocks or Windows binary - On Unix, sudo luarocks install mooni will bring in Moonscript as well, since it's a dependency. mooni itself is a single Moonscript file and can be found here.