Blog All Title

Masks is what we wear, the truth is behind it, not out there. Made from cardboard, plastic or papier-mache, we think it hides all our cares away. Sometimes we switch masks, but we never take it off, afraid forever, to love.

As mentioned in the previous post, the main problem was that Cg allows to specify vertex/fragment shaders independently whilst GLSL does not. In fact as I explained last time, GLSL shaders require a linking process that binds the vertex and the fragment shader together. The linking process makes a GL object called a "program" which represent an entire pipeline itself. Shader parameters and other stuff are acting on this object. The shader plugin system of Crystal Space wasn't much designed this way, however layers on top of the plugin system are considering a shader as a pair of vertex and fragment shader, which simplified the work.

Basically, Crystal Space wraps vertex and fragment shaders (csShaderProgram, which implements iShaderProgram) into a single csXMLShaderTech class that is used for rendering. The shaders are specified in the corresponding XML shader file. The basic idea was "iShaderProgram should represent an entire pipeline (so a set of shaders) instead of a single shader". To ensure backward compatibility, this was achieved by adding a simple wrapper class, csXMLShaderWrapper, that holds a vertex and a fragment shader from the old Cg or ASM plugins.

The new GLSL plugin doesn't need to use this wrapper since the shader object provided already contains an entire pipeline, ready to use. A new XML syntax has been setup for shaders that don't need wrapping. You were currently specifing shaders like this:


<pass>
<vp plugin="glcg">
<cgvp>
<program>
<![CDATA[ /* code... */ ]]>
</program>
</cgvp>
</vp>
<fp plugin="glcg">
<cgfp>
<program>
<![CDATA[ /* code... */ ]]>
</program>
</cgfp>
</fp>
</pass>


As you can see, plugins are specified independently; a GLSL plugin could hardly fit this design. The following syntax has been defined for the new plugin:


<pass>
<shader plugin="glsl">
<vp>
<program>
<![CDATA[ /* code ... */ ]]>
</program>
</vp>
<fp>
<program>
<![CDATA[ /* code ... */ ]]>
</program>
</fp>
</shader>
</pass>


The "unification" is made more explicit and the code is lighter.

Welcome to my GSoC 2011 blog! Here you'll be able to follow the progression of my project for Crystal Space.

This is my first time participating to the Google Summer of Code, and I'm glad I've been accepted into the project I was most interested in. My project will consist of, basically, add support for OpenGL GLSL shaders into Crystal Space, which currently provide support for OpenGL ARB and nVidia Cg shaders. Ideally I'd also like to add support for the famous geometry and tessellation shaders, which would allow Crystal Space to implement new generation and cool rendering techniques based on them. The project will be implemented as a Crystal Space plugin.

The main design difference between Cg (and ARB) and GLSL shaders is that Cg allows you to bind the vertex and fragment shaders independently while GLSL requires them to be linked (and thus non-separable) into a "program" before being used. Fortunately, Crystal Space's architecture doesn't use much of the "separable" functionnality provided by Cg, facilitating therefore the setup of a "unified" design, much like GLSL's. This will involve a new syntax in the XML shader files used by Crystal Space in order to make explicit the use of the "unified" model.

I hope that by the end of the summer I could provide a nice tessellation demo, but for now I have to focus on the core of the new GLSL plugin and its integration into the engine.

Have a nice summer! :)

This post will talk about the different ways to import assets into Crystal Space, and will present the recent improvements that have been made about that.

Guided tour of the import of assets into CS

CS had until now several different solutions to import the models and scene objects from the many different 3D design tools and file formats.

Blender has always been until now the editor with the best support for CS. The blender2crystal project is a huge piece of software, with many advanced functionalities such as import and export of CS files, CS previsualization window directly embedded into Blender, animation tree edition, etc. Unfortunately, the blender2crystal project is no more active and has been overtaken by the Python 2.6 and Blender 2.5x series, it is blocked to the Blender 2.49b with Python 2.5 version and may hibernate there forever.

Currently, the solution to export data from Blender 2.5x into CS is to use the B2.5CS exporter from Peragro Tempus. It does not yet have support for animeshes, but it is planned to become the official exporter for CS and should therefore be improved in the future.

3D Studio Max had also some rather good support with either the exporter script from the PlaneShift team or the exporter plugin added recently by Mike. And for other 3D design tools and model file formats, CS had several solutions with either dedicated importers and/or exporters for COLLADA, Maya, Cal3D, md2, BSP.

And there there was... the Assimp plugin

So this was the situation until recently, but a big part of this has now been obsoleted by a new addition made recently to CS, namely the new plugin for the Open Asset Import Library (A.K.A. Assimp).

As suggested by its name, Assimp is an open source library providing a generic interface to load 3D models and scenes from a huge list of different file formats. It can import the meshes, the material properties, the skeletal animations, the lights, cameras and complete scenes. The support for vertex and morph animations is planned but not yet implemented. The support for meshes and material properties is rather excellent for all the listed file formats, while the quality of the import of the animations may vary depending of the format.

The Assimp plugin that has been implemented in CS uses the new guidelines for the preprocess and tool plugins. This basically means that the plugin is integrated completely transparently into the CS loader system: you simply drop any asset file of one of the format supported, and it is loaded magically in CS, without any further need to manipulate or export/import previously the file.

An example has been added in CS with the Seymour COLLADA animated test model. It is composed of one data file and one texture file that have been packed in a ZIP file in 'CS/data/seymour.zip', and the following image is the result of the command:

• 'viewmesh -R=data/seymour.zip Seymour.dae'

Technically speaking, this plugin simply implements the iBinaryLoaderPlugin and iModelLoader interfaces of CS, as any other CS-specific loader plugin. Therefore, when the iLoader is used to load a file, the Assimp plugin will be selected if it supports the file format. It will then analyze the file, and import the objects as genmeshes, animeshes or whole scenes depending on the loader interface used and the options that have been set.

Another interesting feature of the loading system of CS is that the definition of an object can be split in several files. It is therefore possible to have your model in its base format and being kept updated by your artist team, while still being able to already use this model in your application and extending its definition with CS concepts such as physics or more advanced management of the animations. These concepts will simply be defined in a separate CS XML file and will add their data ontop of the one loaded by the Assimp plugin.

Morph targets

A problem that has been mentioned about the Assimp library is that it does not yet support the import of vertex and morph animations. To overcome this, some tools have been added to CS in order to load and merge an animesh with its morph targets defined in several different files. Again, this functionality has been made using the new guidelines for the preprocess and tool plugins, meaning that you can activate this process transparently through the iLoader interface. An example of that has been added, using the default model from the FaceGen software. Here is the result of the command:

• 'viewmesh data/jane/jane.cslib'

Lastly to be mentioned, there is the new exporter plugin for the PnP TerrainCreator editor, that has also been added recently by Mike. This last plugin should be presented more closely in an incoming post on the new 'csisland' demo.

Sorry, but that will be a rather long post. Hopefully I will post more regularly in the future...

Most of the work that was made until now was about procedural animation, at first with the physical simulation thanks to the Bullet physics library, then with skeletal animation with improvements to the new animesh. All this work will be available in the incoming 2.0 release of Crystal Space.

The Bullet's physical plugin

The Bullet plugin of CS was only partially implemented and suffered of several bugs and problems. The plugin has now be completely overhauled and almost all the features of the initial iDynamicSystem interface have been implemented. As a result, the Bullet plugin is now fully useable, and is clearly preferable compared to the ODE plugin, simply because it works much better and has much more features. The 'phystut' application demonstrates how to use most of these functionalities.

In addition to that, several new features have been added. Most noticeable are:

• Kinematic objects for objects acting on the physical simulation but controlled by the user. You can attach a mesh to it in order to have it updated automatically when the mesh is moved.
• Soft bodies to simulate deformable objects such as flags, clothes, ropes, etc. A new animation controller (CS::Animation::iSoftBodyAnimationControl) and some utilities (CS::Physics::Bullet::SoftBodyHelper) have been written in order to create easily a soft body from a genmesh and to animate the genmesh depending on the results of the simulation. The controller is also able to maintain correct anchoring of the soft bodies when they are attached to an animesh which has a deformable topology (see below).
• Terrain colliders integrated with the terrain2 mesh. They are able to load and unload automatically the terrain cell's colliders on demand.
• Debug plugin to display information on the colliders and the objects of a physical scene. This plugin works with both the ODE and the Bullet plugins.
• Physical hit beams and grab joints.

The main functionalities missing are the setup of the information of the collisions and the filtering of these collisions.

The animated mesh system

The animesh (namely the class CS::Mesh::iAnimatedMesh) which was introduced during the YoFrankie project has been improved a lot. Many already existing features have been improved and fixed, such as the sockets, the blending of the animations, the import with blender2crystal, the Finite State Machine animation node, the computation of the bitangents, the display of the animeshes in viewmesh, etc.
Both the skeletal and the morphing animations have also been optimized and run now decently (although all computations are still made only in software).

Many new features have also been implemented, most of them are visible in the 'avatartest' application:

• A new high detail test model has been made specifically for Crystal Space: Krystal. This model has been made thanks to the great help of a friend nicknamed Korbak. It is made in Blender and is based on the Gothic Woman model by Tiziana. At first, a Blender script was written in order to import motion captured animations into this model, but a much better solution has been implemented after that (see below).
• Definition of the "bodymesh" (class CS::Animation::iBodySkeleton), holding the geometric and physical description of the skeleton of an animesh. Bone chains can also be defined to represent sub-parts of the skeleton (to be used e.g. by Ragdoll or Inverse Kinematic nodes).
• New "Ragdoll" animation node. This node can manage automatically the physical model of the animesh, and put parts or whole of the skeleton in a dynamic or kinematic state. The dynamic state allows to have some parts of the skeleton animated by the physical simulation, e.g. to animate props or dying avatars. The kinematic state is useful to have the animesh interact with the physical scene, and to attach some soft bodies to it.
• New "Lookat" animation node. This node can control a bone of the skeleton in order to make it look at a given target. It is useful to bring life into an avatar and making it more responsive to what happens in the scene. The animation node handles automatically the transitions when the target gets in and out of visibility, so there's no need to manage the transitions at a higher level (but the exact behaviour is configurable).
• New "Speed" animation node. It takes as input a set of animations of the mesh moving at different speed (e.g. idle, walking, running), and blends them together to achieve any custom speed in between.
• New "Inverse Kinematics" animation nodes. Such nodes allow to control the position of an end effector placed on a sub-chain of the skeleton. It is useful to control the precise position of some bones of the skeleton such as the hands or the feet. There are two different implementations of this node, one is a quaternion version of the Cyclic Coordinate Descent algorithm, the other uses the physical simulation in order to achieve the effect.
• New "Retarget" animation node. This node can retarget an animation from one skeleton to another. It is useful for example to re-use animations or to input motion captured data into your animesh. The two skeletons must have a similar topology and limb proportions, otherwise the results won't look good. To overcome this, the behavior of the retargeting can be tweaked, but a more advanced solution would be needed in the future.
• New "Debug" animation node. It currently simply allows to display the state of the skeleton.
• All animations and nodes can be loaded from the CS's XML scene description file format. A facility has also been setup to manage easily the splitting of the definition of the objects in several different resource files.

This is not the end of the list of new functionalities:

• Decals can now be defined on animeshes, allowing an easy personalization of the meshes. The decals are updated automatically to match the deformations of the animesh.
• Clothes (i.e. physical soft bodies) can be easily and precisely attached to an animesh. They will be animated by the physical simulation and their attachment point will also be updated depending on the deformations of the animesh.
• I mentored Alexandru Voicu during its Google Summer of Code 2010 project on hair simulation and rendering. Thanks to his work, there is now a dedicated system to simulate hairs and fur. The motion of the hair strands are animated by the physical simulation, and the system has many properties to personalize the appearance of the hairs.
• A facial animation demo has been written, showing how to define facial expressions and control the animation of the morph targets to achieve the desired effect. This system would however need to be generalized into a more advanced solution, hopefully really soon.
• A motion capture facility has been written. The system is able to load animations from the BVH (Biovision Hierarchical data) file formats. A viewer (csmocapviewer) allows to visualize easily the content of the BVH file and the retargeting to a given animesh. The viewer is also shipped with a Point Light Display system, which has been designed to be used in psychology studies on the human perception of biological motion. It uses a Perlin noise to add points with random motion.
• Animeshes have been integrated in the "Project Bias" made by SueastSide. It is a demo of a medieval First Person Shooter, accessible as a demonstrator in CS (the 'csbias' application).
• The libnoise library has been integrated into CS (namespace CS::Noise). It is useful to generate coherent noise functions such as the Perlin noise, and can be used to generate and animate e.g. clouds or fire, but are also helpful for procedural animation of avatars (see the work on this by the master himself).

Future work

We will keep working on CS, CEL, and particularly the animeshes as part of the developments needed for the Immersive Virtual Environment. The main topics that are planned are:

• Hardware skinning pipeline and Level Of Details. These are the main missing functionalities in the current animeshes, but should hopefully be managed by our incoming developer.
• Custom animation channels, e.g. for morph or shader variable animation
• Artificial Intelligence: inclusion in CEL and interaction with CEL's behavior trees and path finding systems
• Assets management and edition tools
• Animation events system
• More advanced Finite State Machine animation node
• Playing with libnoise's inclusion in CS and Ken Perlin's ideas on using noise to add details in the animations
• Locomotion system and walking on uneven terrains
• More advanced animation retargeting
• Assets pipeline from the MakeHuman editor into the CS animeshes
• More clothes, props, and personalization of the animeshes

I'm a research assistant at the Electrical engineering department of the Institute of Information and Communication Technologies, Electronics and Applied Mathematics at Université catholique de Louvain, Belgium.

We are setting up an Immersive Virtual Environment to be used by psychologists for experimentation and therapy, and we have chosen Crystal Space as the game engine that will simulate and render our environments.

We will therefore keep working and using Crystal Space, and particularly its animated mesh. One of the main task that we plan to work on in the next months is the implementation of the hardware skinning pipeline that is still missing.

I also started this blog in order to give more transparency to the public on what is going on in Crystal Space. I'll try to present here regularly the new features that are added by the CS team.

You ever been running so hard, you forgot why? You look down behind you and all you see is an empty alley. You slow down taking big steps as you come to halt feeling like a moron, leaning over and supporting your body with your hands on your knees. Desperately trying to breathe...
You hear screeching tires, two cones of light drift around the corner, blinding you. Your vampire movie obsession makes you duck as you hiss at the high beams.

Sometimes the need arises to monitor something.

How to install:

• Get the files from here. and place them in
  /home/youruser/vb/
• Make file executable: chmod +x /home/youruser/vb/virtualbox-applet.py
• Open virtualbox-applet.server and edit
  
<oaf_server iid="OAFIID:GNOME_VB_factory"
type="exe" location="/home/youruser/vb/virtualbox-applet.py">
...
to point to the right path.
• create a sym link in /usr/lib/bonobo/servers/
  
sudo ln -s /home/youruser/vb/virtualbox-applet.server /usr/lib/bonobo/servers/virtualbox-applet.server

• Right click a panel, click 'Add to panel', do a search on 'virtualbox', select 'Python VirtualBox Applet', click 'Add' and voila it should show up.

It seems to be a task that programmers by and large despise, but which is necessary nonetheless: Creating GUIs in the form of menus, buttons, the whole shebang. CS' GUI of choice is CEGUI. However, getting it to run is, right now (Oct. 18, 2010) is... troublesome.
First of all, you need a sufficiently current version, which means >= 0.7.0, which in turn means that the packages in the Ubuntu repositories are too old by far, so figuring out dependencies, installing them, checking out the SVN and building it yourself is the way to go:

apt-get install libtool libpcre3-dev
svn co https://crayzedsgui.svn.sourceforge.net/svnroot/crayzedsgui/cegui_mk2/branches/v0-7 cegui_mk2-0-7
cd cegui_mk2-0-7/
./configure --disable-xerces-c
make
sudo make install

So far, so good. But now we're getting to the Python bindings. If I were a little less curious than I am now, I'd say "It's not worth the hassle, just wait until they're officially supported" (which, for now, is scheduled for Nov. 19, 2010). Well, but I *am* that curious and hassled Kulik, so... You know what? Nevermind. Wait until they're officially released. The other way includes downloading bindings that are probably gonna change, writing build scripts yourself, all the good stuff. Just build CEGUI itself so you can enjoy CS' ceguitest and hairtest.

So, what is Crystal Space and what can it do for you?

Crystal Space is, centrally, a realtime 3D engine. In addition to that it includes everything that one needs to make a game or application; sound, keyboard, mouse and tons of other stuff. CS is event-driven, which means that once you've set up the map for playing, your code gets called each time something relevant happens.

To make juggling all that data easier, the Crystal Entity Layer (CEL) has been introduced. It adds the concepts of entities, property classes and messages. An entity is a "thing" in your game, the properties of which are determined by the property classes given to it. Property classes are, for example, pcmesh (which keeps the entities reference to a mesh), pctimer (which you can use to get a message either after a set delay or every frame), pccommandinput (keyboard and mouse input), pcmeshselect (which sends you a message when the user clicks on the pcmesh that the entity having this class also has) and so on.
Messages, lastly, are CELs mode of inter-entity communication. Property classes send messages to their entities when something relevant happens, and you can make your entities send messages to each other, too. Messages have a name and parameters, which are keys and values.
A somewhat deprecated concept is that entities also have behaviours. Those are the part of code that "receives" the messages, that is, gets called when a message with a name which is also the methods name in the code (I'm specifiaclly talking about writing behaviours in Python, but you can also do it in HTML) comes in, and is given the messages parameters. This concept is outdated only insofar as these message handlers now are encapsulated into "regular" property classes. So instead of having one and only one behaviour script per entity, you can now load and unload behaviour script property classes as you like.
Last, but not least, I also should mention entity templates. If you have multiple entities with are rather similar to each other, you can create a template from which those entities then are created.

To make things even easier (yes, that paragraph just now really was about how things get easier, not more complicated ;) ), CELstart was written. While all the defining templates, entities, meshes etc. can be done in map files, there still has to be a program that actually loads and runs those files. CELstart is that program. All it needs is a .zip file that includes your map files, artwork, whatever code you may have written and configuration files. It'll take those and start up the engine, load or create whatever you mentioned in celstart.cfg in your .zip, and off you go: Your game is running.

Well, that's the theory. All you have to do now is to think of a game or application that you want to write, then you find out what your entities are, then you assign property classes, model, texture and animate the models you want to use, write behaviour property classes and... Voila. Presto game.