Emscripten Overview Writeup

If you're interested in how Emscripten works, then this writeup I am working on may interest you. It is currently the best explanation of the underlying techniques Emscripten uses to compile LLVM to JavaScript, including the memory model, the Relooper algorithm, etc.

It isn't meant to be a manual or a practical guide. For that, as always see the wiki.

LIL browser demo

I just saw this demo of LIL (a Little Interpreted Language) running in the browser, compiled from C to JavaScript using Emscripten. Cool stuff!

A Completely Speculative History & Future of H.264 and WebM

1. Several years ago, Google decides something needs to be done about web video, because (1) H.264 requires royalties, which means that parts of the web are proprietary (even if it is a standard), and Google believes an open web is in its best interest, and (2) for similar reasons, H.264 is incompatible with the W3C, Mozilla's Firefox, and Opera, so it will never become universal anyhow. Google is willing to go to great lengths to solve this issue, including large sums of money and developer time.
2. Google approaches MPEG-LA (or major entities that are members), and quietly floats the idea of 'freeing' H.264, by way of a large one-time payment from Google, after which H.264 will be royalty-free, and can then be blessed by the W3C.
3. Negotiations fail. Google offers large amounts of cash, but it isn't enough for MPEG-LA, which believes it is close to having a complete lock on the market, which it can leverage for even more cash later on.
4. Google threatens to support a competing format with all its resources, thereby threatening the future profitability of H.264.
5. MPEG-LA decides to call Google's bluff.
6. Google makes good on its threat, buying On2 and freeing its VP8 video codec as part of WebM, a royalty-free format for web video. (Note: I'll use 'WebM' to refer to 'VP8', a lot of the time.)
   
7. Mozilla, Opera, etc. naturally support this move, as it is good for the open web. Apple and Microsoft, whose motivations are otherwise, do not support this move - they are both already heavily invested in H.264, and for them life would be simplest if WebM never existed.
8. As a reaction to WebM, MPEG-LA makes H.264's licensing less expensive, and for a longer period of time.
   
9. Google makes good on another part of its threat to MPEG-LA, removing H.264 support from Chrome. MPEG-LA is surprised Google is willing to hobble its own browser in order to get a leg up in this fight.
   
   (This brings us to the present time.)
   
   
   
10. Nothing much changes, at first. Most web video is seen through Flash anyhow. However, the block of WebM supporters, which is now Firefox, Chrome and Opera - whose share in the market is large, and growing - gets video providers on the web to pay close attention to WebM.
11. Flash introduces WebM support. Most video encoded for desktop viewing can now be encoded in WebM, and viewable through Flash or an HTML5 video element in Firefox, Chrome and Opera. Even video shown with DRM can be encoded in WebM, but must be shown in Flash. On the other hand, in the mobile space, a complete stack of hardware&software support is still really just present for H.264, and Apple doesn't support anything else, so video encoded for mobile viewing is primarily done in H.264.
12. WebM's video quality improves, in part benefiting from the fact that while open source and royalty-free, WebM is not a formal specification or standard, so rapid development and changes are possible. WebM becomes equivalent or superior to H.264.
    
13. Google switches YouTube to primarily use WebM for encoding video meant for desktop use. There is hardly any impact on users, due to most video being shown in Flash anyhow (which now supports WebM). However, video for mobile viewing remains encoded in H.264.
    
14. Hardware support for WebM begins to ship in a great deal of new mobile devices, and eventually in a majority of new mobile devices.
15. As a reaction to WebM's rise, MPEG-LA once more lessens the royalties for H.264, in an attempt to make it more competitive.
    
16. A new version of Google's Android ships, on a new flagship phone from Google, that has complete hardware and software support for WebM. The device primarily views YouTube video in WebM format.
    
17. Google, stating WebM's superior quality, begins to 'favor' WebM over H.264 on YouTube, for mobile content. More specifically, while both WebM and H.264 are supported, WebM content is encoded at higher quality levels (this is accomplished not by decreasing H.264 quality, but by adding a higher level of quality exclusively for WebM). The result is that mobile devices viewing YouTube give a better user experience if the device does so using WebM.
18. Apple makes the rational decision and supports WebM on new iOS devices - hardware support is already there, and Apple cannot compromise on user experience. Whatever monetary benefit Apple gains from MPEG-LA from H.264 is completely eclipsed by Apple's iOS business, so this is a no-brainer.
19. With the majority of new mobile devices shipping with WebM support (Android and iOS), smaller players (Blackberry, WebOS, Windows Phone) are forced to support it as well.
20. The online video market has its anti-DRM moment, just like online audio already had. Video is shown without DRM, which simplifies delivery and cuts costs, and piracy remains at the same levels as before (just as with audio).
    
21. Once it is clear H.264 has lost in the mobile space, and that DRM is no longer needed, there is no reason for Microsoft and Apple not to support WebM in the HTML5 video element, on Windows and OS X respectively, in order to ensure their users the best experience.
    
22. With DRM no longer an issue and widespread support for WebM in the HTML5 video element, WebM becomes the universal standard for video on the web, on both desktop and mobile. Content producers have little reason to even support a fallback to Flash - some do, but many do not, at little detriment to them or their users.
23. Google wins the fight, and the open web greatly benefits.
    

Some things that can change this future history:

• MPEG-LA deciding to make H.264 100% royalty-free. This will kill MPEG-LA's profits, but may still be worthwhile for MPEG-LA members, since if done properly - and promptly - it can ensure H.264 becomes the standard for web video. Whether there remains enough profit from H.264 (from hardware, services, etc.) for this move to make sense, is not clear. But if this does happen, WebM loses, but really Google wins, since it got what it set out to get.
  
• A new video format can appear, or a newer version of an existing format, which requires new hardware support but has benefits to justify the switch. Given the battle between H.264 and WebM, I would expect the new format's backers to learn the lessons of the past and make it free on the web (or, if they are not willing to do that - then to not even bother to create a new format). If such a new format appears, and becomes the universal standard for web video, the result is that Google wins in this case as well.
• The fight gets taken to the courts. I doubt a simple injunction will be granted to either side, as both are powerful, influential, and have many patents to back up their claims - so there is no quick victory. Instead there is a lengthy court battle. To justify the cost, there must be a significant chance of large future profits, and if H.264 is already in decline, that might not be the case. However, it might still make sense for MPEG-LA to take Google to court, just to get it to settle for some amount of money, in which case Google wins overall, but MPEG-LA gets a little more money than otherwise. However, if the goal isn't a settlement, but an actual attempt to kill WebM, then things can get interesting. I don't think anyone can say for sure how that fight would turn out - does WebM infringe on H.264 patents? Does H.264 infringe on WebM patents (VP8 patents, granted to On2, and now owned by Google, which would countersue)? Perhaps both? Such a 'fight to the death' in the courts seems unlikely, in part due to that unpredictability, so all we can say for sure in this case is that several law firms will greatly benefit.
  

DISCLAIMER: I have no inside knowledge about any of this.

Emscripten Usage Change

I refactored the python scripts in Emscripten to make them more sane. There is one difference in how Emscripten is used: If ~/.emscripten does not exist, it will copy tests/settings.py to ~/.emscripten, at which point you would edit the paths etc. in ~/.emscripten - not in tests/settings.py, which is how things were before (and that was bad).

If you already have a file at ~/.emscripten, and you probably do if you already ran Emscripten in the past, then that file will not contain all the necessary information. The simplest thing is for you to copy your edited tests/settings.py into ~/.emscripten. Or, you can delete ~/.emscripten and run the tests (python tests/runner.py), which will copy tests/settings.py for you into ~/.emscripten (but remember to change the paths, if you need to).

Sorry for the inconvenience, but the previous way things were was just a hack, which had to be fixed.

Wiki pages on the project site have been updated.

JavaScript, Native Client, and Emscripten

This is a response to this blog post, which is titled "Mozilla’s Rejection of NativeClient Hurts the Open Web", which appeared prominently on Hacker News.

I disagree with the thesis in that blog post. My reasons are entirely technical.

First off, NaCl is not portable yet. PNaCl is working towards that, but it will take time. Until it is portable, comparing it to JavaScript is comparing apples and oranges. They simply do very different things - one is fast, the other is portable.

Second, while PNaCl is being worked on, at the same time a lot of effort is being put into making JavaScript engines faster. Now, for purposes of comparison to PNaCl, we don't need all JavaScript code to run as fast as native code. For our purposes here, we can care only about 'implicitly statically typed' JavaScript code - the code that can, in theory, be compiled so it runs as fast as native code. Implicitly statically typed code is code that uses a single type for each variable, and even though it's written in a dynamic language, could correspond almost 1-to-1 to code in a fast statically typed language like C.

Such code can be created automatically from C or C++ using something like Emscripten. Or, you can write such code on purpose, for example, PyPy is written in RPython, which is basically implicitly statically typed Python. More generally, you might write the performance-sensitive parts of a JavaScript application in an implicitly statically typed manner, while the rest can have fun with dynamic typing and the benefits that gives.

The important issue is that JavaScript engines can optimize implicitly statically typed code very well, both theoretically and practically. Techniques like tracing, type analysis, etc. are already being used in JavaScript engines like SpiderMonkey and V8, and progress is very fast. Also, PyPy can give us an idea of the long-term potential in such an approach.

So, while NaCl is working towards portability, JavaScript is working on speed. To clarify, again, I'm not talking about running all JavaScript code at native speed. But there is a big, relevant subset which can be run very quickly. Once JavaScript engines achieve the goal of running that code at native speed, then the performance advantage of NaCl will have vanished. At that point both NaCl and JavaScript will be fast (and, if PNaCl is completed, they will both also be portable).

Once we get there, I believe JavaScript will be preferable for the simple fact that it has a natural fallback built in - implicitly statically typed JavaScript is perfectly valid JavaScript, so even if your JS engine doesn't achieve the full speed of native code, at least it can run it. Whereas NaCl will simply not run at all unless the NaCl plugin is installed (and that may never happen on iOS devices, and may never happen by default on any desktop browser but Chrome).

Note that one might devise a fallback for NaCl by writing an emulator, or even a compiler, for PNaCl in JavaScript - perhaps using Emscripten (which does exactly what is needed - convert LLVM into JavaScript). If the speed-intensive parts of the code in that approach are implicitly statically typed, then we have come full circle, and the two approaches of JavaScript+Emscripten or PNaCl+Emscripten essentially converge, with a minor disadvantage to NaCl for being more complex (a special NaCl compiler, and a special NaCl runtime).