Emscripten 1.0!

It's been almost a year since I started Emscripten (which, if you haven't heard of it, is a tool to compile LLVM to JavaScript), during which it took up much of my spare time. So I am very pleased to announce that today Emscripten has reached the 1.0 milestone. This release comes with a demo of rendering PDFs on the web (warning: that page downloads >12MB, since it includes Poppler and FreeType. It's like downloading an entire desktop app, almost).

Other highlights in this release:

• Very significant optimization of memory use in the compiler. This was necessary for the PDF demo to build, since it is far larger than previous demos.
• Full support for the recently released LLVM 2.9.
• The Emscripten documentation paper is finished. It explains how Emscripten works, so you might be interested in it if you care what Emscripten does under the hood (but if you just want to use Emscripten you don't need to read it).

Overall Emscripten is now in very good shape. It can probably compile most any C/C++ project out there, subject to some limitations (like JS not allowing C-style multithreading). At times some manual intervention is needed, like changing the project's settings so it doesn't generate inline assembly, and of course bugs probably still exist, but recently the code I have compiled has tended to just work (hence the rate of commits has greatly decreased recently).

The speed of the generated code can be quite good. By default Emscripten compiles with very conservative settings, so the code will be slow, but optimizing the code is not that hard to do. Optimized code tends to run around 10x slower than gcc -O3, which is obviously not great, but on the other hand fairly decent and more than good enough for many purposes. And of course, that ratio will improve along with advancements in JavaScript engines, LLVM, and the Closure Compiler.

So, Emscripten 1.0 is in my opinion pretty solid. There are no major outstanding bugs, and no major missing features. (But I do have plans for some major improvements, which are difficult, but should end up with code that runs at least twice as fast.) Now that Emscripten is at 1.0, I am hoping to see it used in more places. I'm starting to propose at Mozilla that we use it in various ways, and also I'd love to see things like GTK or Qt ported to the web - if anyone wants to collaborate on that, let me know.

Emscripten moving to LLVM 2.9

LLVM 2.9 will be released very soon, and Emscripten has just been updated to support it.

Emscripten has a lot of automatic tests - they take over 2 hours to run on my laptop - so I won't be running tests for LLVM 2.8 anymore (that would double the time the tests take). Until LLVM 2.9 is formally released with binary builds, you can build LLVM from svn source (the instructions on the Emscripten wiki are useful), or use LLVM 2.8 with Emscripten 0.9 (the last release of Emscripten that supports 2.8).

If you do build LLVM 2.9 and put it in a different location than 2.8 was, don't forget to update your ~/.emscripten file so it uses the version you want. Also, if you update LLVM to 2.9 and want to use llvm-gcc, you need to update that to their current svn as well.

There were not a lot of changes for Emscripten to support 2.9, so it is possible 2.8 will still work. But as mentioned above, I am not testing it, so I can't say for sure.

Puzzles on the Web

Check out this very cool port of Simon Tatham's Portable Puzzle Collection to the web, by Jacques Le Roux, using Emscripten.

Nice quote from there:

This was basically just an experiment to see how hard it would be to port C code to a web application running entirely on the client (turns out not that hard).

:)

Emscripten 0.9!

The demo this time is OpenJPEG: JPEG 2000 decoding in JavaScript.

Aside from OpenJPEG, lots of stuff in this release, including

• Line number debugging: Emscripten can optionally add the original source file and line to the generated JavaScript (if you compiled the source using '-g'). Useful for debugging when things go wrong, especially with the new autodebugger tool, which rewrites an LLVM bitcode file to add printouts of every store to memory. Figuring out why generated code doesn't work is then as simple as running that same code in lli (the LLVM interpreter) and in JavaScript, and diff'ing the output, then seeing which original source code line is responsible.
  
  
• Line-specific CORRECT'ing: The main speed issue with Emscripten is that JavaScript and C have different semantics. For example, -5/2 in C is -2, while +5/2 is +2, whereas in JavaScript, naive division gives floating point numbers, but worse, there is no single operator that will create the same behavior as C (Math.floor on -5/2 gives -3, and Math.ceil on +5/2 gives +3). So in this example (unless we have a trick we can use, like |0 if the value is 32-bit and signed), we must check the sign of the value, and round accordingly - and that is slow. Similar things happen not just in rounding, but also with signedness and numerical overflows, and therefore Emscripten has the CORRECT_SIGNS, CORRECT_OVERFLOWS and CORRECT_ROUNDINGS options.
  
  With line-specific correcting in the 0.9 release, you can find out which lines actually run into such problems, and tell Emscripten to generate the 100% correct code only in them. Most of the time, the slow and correct code isn't needed, so this option is very useful. I will write a wiki page soon to give more examples of how to use it to optimize the generated code (meanwhile, check out the linespecific test).
  
  
• 20% faster compilation, mainly from optimizing the analyzer pass.
  
  
• Strict mode JavaScript. The compiler will now generate strict mode JavaScript, which is simpler, less bug-prone, and in the future will allow JS engines to run it more quickly.
  

Synchronizing a git mirror with hg-git

Posting this since it might save someone else the time it took me to figure out.

There are several tutorials for creating a git repo from an hg one using hg-git. But none of them go much into how to keep the git repo up to date when the hg one changes (assuming work is done in the hg repo). And, just doing a push with hg-git to github fails... silently :( (But it succeeded the first time, to create the repo.) Even adding -v --debug in hopes of getting some useful information is no help.

The solution is to push to a local git repo first, then push from there to github. But even pushing to a local git repo won't just work - you will get

error: refusing to update checked out branch: refs/heads/master
error: By default, updating the current branch in a non-bare repository
error: is denied, because it will make the index and work tree inconsistent
error: with what you pushed,

To get around that, create another branch in the local git repo (git branch side), switch to it (git checkout side), then push to it using hg-git. Then switch back to master (git checkout master) and finally push that to github.

I think it might also work to make the local git repo into a bare repo, but the above worked for me so I stopped there.

Git Mirror for Emscripten

We now have a git mirror on GitHub! Getting the Emscripten code is now as easy as

git clone git://github.com/kripken/emscripten.git

Code will be mirrored there, so you can either use git with that GitHub repo, or hg with the Google Code repo, and the result will be the same. (However for project stuff - issues, wiki, etc. - we will continue to use the Google Code project page.)

FreeType Demo

A simple demo of FreeType in JavaScript can be seen here.