Saturday, June 27, 2009

Autoturrets Video

This is done using the Projectiles.js library, which is now mature enough to be put in the standard library. Note that like all other libraries, it's entirely optional, and work on it cannot affect the stability of the engine or the core API (which is why it was added during feature freeze).

Some details:
  • The automatic turrets are controlled by the server, with projectiles managed on both server and clients for responsiveness
  • The turret model was kindly contributed by Gregor Koch under a creative commons license, and consists of a base that turns left and right and a barrel that turns up and down (the commands to render the separate parts are done by a script). Edit: For a closeup, see here.
  • Notice the reflections of the shots in the water
  • Music: The intro to "You Can Take The World" by Predator, from their album Sick Society

Thursday, June 18, 2009

Intensity Engine 0.9.5 Release

More screenshots can be seen here.

Since 0.9, a month and a half ago, a lot of work has gone into the 0.9.5 release which is out today (rev. 324). This release is still not considered stable, but is a significant step towards that (in fact it may be the last release before 1.0). Two main areas of focus in this release are:
  • Allowing people to run their own infrastructure - master server, asset servers, and server instances - without any connection to Syntensity. This allows both federation (separate independent servers, with loose connections - like the WWW) and makes it easier to customize the Intensity Engine for non-standard uses.
  • Improving content creation and collaboration. This is now in a good enough state to allow focusing on content creation, which is one of the main tasks remaining before the Intensity Engine 1.0 release and public launch of Syntensity.
In more detail, here are some of the changes:
  • 'clientSet' state variables, which are applied first on the client, allowing better responsiveness in an easy way
  • Initial login to server instances made faster
  • AreaTriggers made significantly faster, and now work correctly on server
  • Allow maps to extend the position protocol messages, for faster updates of map-relevant information like custom animation settings
  • Allow rendering models (including players) from map scripts, for more flexibility in visuals
  • Added almost-finished Stromar character model
  • Update of server-side NPC/bot system to current API
  • Various API extensions (e.g., allowing gravity to be changed)
  • Various minor GUI and usability improvements
  • Full support for building on Ubuntu 8.10, Ubuntu 9.04, Windows XP and Windows Vista
  • Plenty of bug fixes
Binary builds aren't planned for this release, as things are still moving quite fast.

Towards 1.0

The Intensity Engine is now in feature freeze. Only bugs in existing features will be fixed for the 1.0 release, which is dependent upon
  • a decent level of stability and polish, and
  • a reasonable amount of working content that can be distributed with the engine or at least used to demo it

Thursday, June 11, 2009

Roadmap Changes

I've made some decisions regarding the roadmap for the next few months:
  • The Intensity Engine has been made completely independent of Syntensity, and will continue that way. My original intention was that I would focus on Syntensity, and through that promote the Intensity Engine (the Intensity Engine being an open source project utilized by Syntensity), and later on get around to making the Intensity Engine more compelling as a separate project. But several people (for various reasons) have already expressed an interest in the Intensity Engine by itself, and I have also been hearing some very good arguments for completely open sourcing the Syntensity master server. Because of that, I've started to add to the Intensity Engine the functionality it needs to be usable without the Syntensity master server, namely, an example master server was committed a few days ago, and better documentation on running the various server components as well. Furthermore, improved master and asset servers should be appearing soon.
  • The Intensity Engine 1.0 release is still planned for later this summer. Bugfixing is progressing well, and no major features are planned at this point, just minor stuff. Recent decisions like including a master server as mentioned above should not be a cause of delay.
  • Syntensity will remain in 'closed' alpha, and later 'closed' beta. I say 'closed' with scare quotes, because the source code is open, and I'll give an invite to Syntensity to anyone that asks, so 'closed' here is kind of a silly term. But what I mean is that I won't make an effort to promote Syntensity, and probably not provide pre-built binaries either. The reason is that I think opening up to a wider audience should wait for better content, and it looks like getting that content will take less time than I originally thought, so why not wait.
  • The Syntensity public launch might be delayed somewhat. Basically I will delay it until I feel the content is good enough, probably not by very much, but there is no reason to have a rushed release.

Friday, June 5, 2009

Code Quality Improvements

The Intensity Engine's stability has significantly improved over the past two weeks. In fact, it's been a few days since I ran into a bug. But, the bugs you don't know about are just as bad, so I spent a few days doing various code quality procedures, including running valgrind and doing more serious cross-platform testing.

Valgrind is an excellent Linux-only tool for detecting memory errors at runtime (and a lot of other stuff). For memory errors, valgrind basically checks every memory allocation, read and write to see that it's valid. This is quite an intensive process, making the program run 10 times slower or worse, but one of the nice things with Cube 2 is that it's so fast that, actually, you can run it in valgrind at a decent speed and perform actual tests. And I'm talking about the client, not the server (which can also be done, but is a much less impressive feat).

Even so, running valgrind is a time-consuming task, as it takes a lot of repeat runs to get useful results (as you fine tune the 'suppressions file', the list of warnings that it can safely ignore). This took a few days, during which the following came to light:
  • Several minor issues with Cube 2 itself, including accessing uninitialized values and a mismatched new/delete. I reported them and eihrul committed appropriate fixes (which I then ported to the Intensity Engine). I am somewhat surprised none of these led to noticeable issues in practice, but I guess most compilers initialize enough stuff by default to prevent it. (Perhaps compilers should have an option to not initialize values, for testing purposes?)
  • One serious issue with Cube 2, concerning bone processing in the skeletal animation system: When bones were 'unused' (not appearing in actual blend weights, etc.), that led to a read at index -1 in a C array. As with the previous issues, I reported this and eihrul committed a one-line fix.
  • One minor issue with how the Intensity Engine uses Cube 2: I assumed a value was initialized by default, but it wasn't (not sure why it isn't, it somewhat goes against the conventions elsewhere, but regardless the fix was trivial).
  • One serious issue with, of all things, the Intensity Engine's logging system: An incorrect reuse of a function receiving a variable number of arguments by functions passing it only one (it got confused in some cases and looked for arguments that didn't exist). The fix was trivial.
This actually went better than I expected: Given the 65,000 lines of Cube 2 code, some of which I modified, +30,000 lines of Intensity Engine code that I wrote, I would have expected more issues, or at least more serious ones. Only one serious issue in each of Cube 2 and the Intensity Engine is not that bad.

As a side issue, the trickiest part with running valgrind turned out to be Google V8. Valgrind reports a lot of issues in V8, presumably because V8 dynamically generates machine code from JavaScript, and furthermore modifies that machine code on the fly. In other words, the issues valgrind finds aren't problems with V8 itself, but with the dynamically generated code. And generating an appropriate suppressions file for such stuff isn't easy (not sure it's even 100% possible).

The second thing I did was finally get around to some serious testing on Windows, as during the last few months I only found time for some partial testing myself now and then, and some very useful community contributions. So, the time was right to make sure this worked, and after several hours the client was running fine (there remains an issue with the server, something minor about how linking is done, which I'll fix later on). Interestingly, as always cross-platform testing uncovered some stuff:
  • The serious issue mentioned above with the Intensity Engine logging system led to actual crashes on Windows (while on Linux no errors occurred in practice). So, interestingly, this problem could have been discovered by either valgrind or by cross-platform testing.
  • An issue with reading files using the C API: What I wrote worked fine on Linux, but not on Windows, apparently due to underlying platform differences. The fix was trivial, and even shortened and improved the code: To use Python's file reading system, which is already tested for cross-platform compatibility.

Overall, writing in C++ definitely has its downsides, as the issues mentioned above will attest, and that's why big parts of the Intensity Engine are written in Python or JavaScript: No memory leaks (for the most part), no invalid memory accesses, fewer cross-platform compatibility issues, etc. But with game engines and virtual worlds platforms, the core speed/memory-intensive part really has no choice but to be written in C++. It's not easy nor always fun, but it is manageable.