I've been busy for a few days with a rather boring user study and the results of it were less than great ... so I needed a pick-me-up. For me, this means some creative work. Hence, I decided to spend a little time modeling a new world to test my lighter2 changes. Something considerably more complex than the cornell box and construction in wood and that would logically include every type of light (point, directional and spot). It's fun because it's art (in a way ... right) and for me I learn how to use blender better and can refine my workflow for getting things into CrystalSpace's XML world format.
Anyways, Here's what I've created:
It's a combination of a bunch of different cannonical models from the graphics community (from Stanford specifically ... yes the bunny is in there) all placed in the Sponza Atrium model that's been appearing in global illumination papers for the past several years now. The atrium is texture mapped (and has bump maps somewhere that I have yet to track down). The Stanford meshes are incredibly dense (100K faces each AFTER I decimated them, they had millions in their original form ... some 10s of millions). So, this is an intense little scene to render and particularly tough to run through lighter2. It's a more realistic example and hopefully will test just how robust the system is. Consequently it takes a VERY long time to light. Most of the time is spent laying out the lightmaps (not raytracing or photon mapping) but be prepared to wait a while if you want to re-calc the maps and put it on a machine that you can let run for potentially a few days. I'll post some more images as lighter2 finishes running.
Blender model: I setup the world in blender and here is the file for that. It took a lot to get all of this into blender. Here's a breakdown of what I did -
The model is still missing several textures and when you export it there will be several errors that need to be fixed by hand. These are all present in the original 3ds file and were just easier to fix by editing the world file by hand.
SponzaBlender.zip (15.2 MB, hosted externally)
Material fixes for world file: Here's the extra textures and a snippit of XML that can be pasted into the world file over it's 'textures' and 'materials' section to fix all the errors created by Blender2Crystal. Note that bump maps are still missing. I can't seem to find these on the net anywhere.
SponzaMaterialFix.zip (499 KB, hosted externally)
CrystalSpace world: Here's the exported world with the material and texture fixes already applied.
SponzaCSWorld.zip (16.4 MB, hosted externally)
Static lit CS world: Here's the same world with lightmaps included so you don't have to wait for it to finish raytracing. This is just direct raytracing for now. A photon mapped version is forthcoming.
[Still rendering, will post soon]
I had a thought on how to do some basic unit testing to eliminate one possibility for error in my code. Each of the distribution functions I've implemented has never been tested, just copied out of the book (or worked up by myself) and assumed correct. This includes the code to distribute rays from a spotlight. So, I decided to write a quick program that would visualize each of these distribution functions as a way of verifying them (and also a first step towards testing the spotlight code).
Here's some images for each distribution:
I found, by doing this, that there was an error in my equal distribution function. Namely, it was distributing rays in the entire sphere not just the hemisphere. Easy fix. this would results in lots of lost photons and did help brighten the photon simulation which is always good considering how much darker it is than the raytracing version.
Here's the spotlight with different 'outter' values:
This all looks good so I think the spotlight code is correct assuming I've interpreted the parameters correctly.
Lots of posts today, sorry for that. This one will be short.
So, Final Gather is slow as molasses. Furthermore it seems to be not working right in my code. But, it is very important for smoothing out noise in the photon map. Enter the irradiance cache.
The irradiance cache is a concept introduced by Greg Ward and co. back in 1988 and I believe is part of Ward's Radiance renderer. The paper that describes it in detail (and which is surprisingly easy to read) can be found here (hint to Scott, check this out):
The basic idea is that diffuse inter-reflections are very hard to compute using Monte Carlo methods (and Final Gather is a Monte Carlo method) but by-and-large they are very uniform and slowly changing across a surface (i.e. a good candidate for interpolation). So we would like to reduce the number of times it needs to be computed and interpolate everywhere else.
Ward describes a two tiered method: the primary method is the standard Monte Carlo method and the secondary method will interpolate cached values from previous computations. The secondary method needs to know when it can interpolate and how to weight the cached values. This is done by approximating the gradient of irradiance between the cached point and the point we need irradiance for. If the gradient estimate is too high a new value is needed. Otherwise we can weight the cached value by the inverse of the error inherent in the estimate and get a very good approximation that is considerably cheaper than the Monte Carlo method.
The devil's in the details and I won't bore you with it (other than to say it involves an octree based around the valid distance of each cached sample). But with Jensen's summary of Ward's paper (and the paper itself) I think I've got the Irradiance Cache implemented. It needs some testing and such (still to come) but hopefully this will help with noise and speed ... provided I can then figure out what is wrong with the Final Gather!
I'd kill for smaller unit testing right now but we'll see if we need it first.
To round out my previous post concerning GSoC I thought I should list out the changes I have made and how the milestones are lining up.
Despite the noise and artifacts still present I think it's safe to say Milestone 1 is complete (or nearly so). Here's a list of things still to finish/implement.
Immediate priorities (considered unfinished Google Summer of Code obligations):
Future changes (for fun and profit!):
It seems appropriate to send thanks out to Res and Scott at this point as they have both been quite helpful this summer and I'm sure will continue to be.
And now, we carry on ...
So today is the final day of GSoC 2009 and I have just committed my final major revision of the program. Although this will not be my final contribution to CrystalSpace, I thought it fitting to take the time to summarise the project undergone this summer; both the deliverables achieved and those that were missed.
Dismissed at an early stage due to unforeseen delays, this feature was originally intended but has not been produced. Unfortunately, I can not provide any insight at this time as to how future work could implement this feature.
CEL Expressions in Triggers
Again a personal target that was not reached. The mailing list discussion on this topic faded rapidly and with my faulty system delaying the generation of documentation in the final week, this deliverable was pushed back in favour of the documentation deliverables. Unlike nesting, however, I do have some insights as to how or if this feature should be implemented that I would like to share.
Firstly, my concern comes in that CEL expressions optionally require a parameter block to be parsed. This is readily available in rewards and sequences as when rewards are executed the calling trigger passes a parameter block consisting of the dynamic (@) parameters that specific trigger generates. It was then noted that this could be passed on to sequences when started or finished by the associated reward and therefore CEL expressions have been implemented into sequences (a small compensation for the lack of this feature in triggers.) However, with no data stored in a parameter block inside triggers and certain expressions calling parameter block methods on the optional argument could easily cause runtime errors and, even with these caught and reported, the full functionality of expressions would not be possible.
This opinion though is based on my current understanding of CEL expressions and may be inaccurate. Anyone considering taking up this development issue in future would be wise to consider this but also to read around themselves. Predominately, the following expression tokens need to be considered for their use of the parameter block:
Similarly, the method GetParameter from ParamaterManager that calls for the evaluation of expressions also attempts to get parameters of type dynamic/@ which are only generated inside triggers. Therefore, I think a new method probably should be called to prevent the accidental, attempted use of dynamic parameters inside a trigger.
The remaining list of features not developed were not originally proposed but have been raised in mailing list discussions as being of interest to the community. Anyone keen to take on a quest-related project may want to consider:
- Adding constraints to sequences
- Creating filters for Message Trigger by sender and parameters
- Making quest variables accessible as pc properties
- Standardising the start-finish protocol of sequences
After the sour note of what was not achieved, the following is a brief overview of what I have added to CrystalSpace during GSoC 2009.
Rewards, Triggers, Sequences & Parameters now Quest Independent
One of the largest goals of this project was to factor out Rewards and Triggers from the Quest system where they may be of use to developers in other contexts. This development led to the refactor also of sequences and quest parameters with the creation of cel sequences and the parameter manager. All of these tools can now be created and used without the need to construct a quest.
Added CEL Expression Functionality to Sequences
As discussed in the previous section, CEL expressions can now be passed as parameters to sequences and seqops.
A Brand-New Behaviour Tree Tool for Character Development
Successfully completed in time, CEL now offers the tools needed to create and execute a Behaviour Tree. Serving as a proof of concept for the usefulness of factoring out rewards and triggers from the Quest subsystem, the Behaviour Tree tool is a new AI method quickly gaining popularity in the commercial game development scene to create reactive and complex characters in game environments.
Tutorials on Quests and Behaviour Trees
An original deliverable that, having personally struggled to understand at first the Quest subsystem, is proudly presented to help all new-comers to CEL and this powerful FSM implementation.
It is hoped that the inclusion of such documentation for the Behaviour Tree implementation at the time of its creation will help to introduce existing CEL developers to the new tool, growing the BT user base and resulting in the further development of BT features. Examples of such future BT work could include:
- Crystal Architect integration
Already discussed briefly on the mailing list, this will require the addition of Error handling and (De)Serializing Methods.
The current implementation evaluates the whole tree each frame.
This could be detrimental to a game's performance. Instead a scheduler could dictate how many nodes to expand each frame. A BT scheduler also would allow for the development of more advanced features such as a parallel selector.
- Expansion of the Decorator Plugin
Currently only three decorators have been implemented. These three show the use of a decorator but only begin to implement the functionality. A wide range of possible decorators are possible and will hopefully in time become available in CEL.
A Final Personal Note
To conclude, I would like to thank everyone that has supported or shown interest in this project and a special thanks to Guillaume for being my mentor and helping throughout the programme.
Special mention also to Pablo who has been very vocal in all discussions, without your input a lot of this may not have been achieved. I hope you find my work useful and that we can continue to work on BT integration with Crystal Architect in the near future.
My personal plan, is now to enjoy the next two weeks of my summer and to take a break from my computer (although this will involve at least sometime trying to recover my fallen machine... grumble :( ). I am busy then throughout September with a conference in Milan and moving back to university. However, I will be available at all times by email so will keep track of the mailing list for any quest/bt discussion or anything required of me to successfully finish this programme.
My intention then still remains to continue developing for CS/CEL. My interest for the BT implementation is obvious and I hope to work further on this alongside my PhD that begins in October.
I believe that the project has been a great success, and hope that you all agree likewise. If anyone has any queries or difficulties with what I have produced I will as always remain open to questions and love to help. I hope that this work is found to be useful to the community and that it makes it into a CEL release version soon. :)
Kind Regards, Sam
With the end of GSoC approaching I want to take inventory of what has been achieved so far. To reiterate I do fully intend to stick with this and keep improving lighter2, assuming I have not overstayed my welcome.
Here's some results. I've spent today rendering images so that I can show what I can achieve with my photon mapper when I hold it's hand and try my best to get good results.
First up, an image of the Cornell Box with just basic raytracing. This scene was rendered with global ambient turned off, all lights forced to realistic attenuation and all light power scaled by 8.0, as such:
lighter2 --directlight=raytracer --noglobalambient --forcerealistic --lmdensity=20.0 --lightpowerscale=8.0 data/NewCornell
Next, an image of the Cornell Box with just Photon Mapping (for both direct and indirect light). Here we shot 5M photons and sampled 5K times for each density estimation. The command line was like such:
lighter2 --directlight=photonmapper --indirectlight=photonmapper --numphotons=5000000 --maxdensitysamples=5000 --sampledistance=20.0 --nofinalgather --lmdensity=20.0 --pmlightscale=100.0 data/NewCornell
Lastly, an image of the Cornell Box with direct light done with raytracing and indirect light with photon mapping. It was VERY difficult to get the two values to have a comprable exposure (i.e. photon mapping was consistently too dark). Recent changes to the way light is scattered have made this matter worse but are conceptually necessary to get the simulation to be correct. Needless to say, I had to fudge the light power manually until the image 'looked' okay. Very imprecise but good enough for today. Here's what the final command line looked like. Note that I bumped the number of photons up to 25M to help fight noise which can be particularly noticeable for indirect lighting:
lighter2 --directlight=raytracer --noglobalambient --forcerealistic --lightpowerscale=8.0 --indirectlight=photonmapper --numphotons=25000000 --maxdensitysamples=5000 --sampledistance=20.0 --nofinalgather --lmdensity=20.0 --pmlightscale=16.0 data/NewCornell
The code I check in today will be able to do all of this. Note that I used an old version of walktest.exe to render these images (from the 08 SOC branch for lighter2). The one in my branch is still not working with light maps for unknown reasons.
My handling of light attenuation turned out to be incorrect. Res and Martin set me straight. I had the conceptual model that distance attenuation accounts for the phenomenon of light losing power as it travels through a medium (even just the atmosphere). This is called attenuation in physics and optics but in graphics this is not what distance attenuation is accounting for. Distance attenuation accounts for the fact that as light moves away from its source the energy spreads out (I assume like a wavefront spreading out). It does so like the surface of a sphere so the 'realistic' distance^2 factor accounts for this spreading out perfectly.
The consequence of this type of attenuation (the correct type) is that photon mapping attenuates automatically. We are distributing photons equally around the sphere of the light source and when they land they will be distributed according to how far away they are from the light. The density of this distribution already has this spreading effect built in automatically.
So, to really do calibration between raytracing and photon mapping I need to remove the attenuation from the photons (already done) and then switch all the lights to use 'realistic' attenuation (which is not the default). My apologies to res for second guessing his advice as this was his original suggestion. As soon as I did this it became apparent that things were dramatically more comparable between raytracing and photon mapping:
As you can see, despite the similarity resulting from the change to realistic attenuation there is still a marked difference in the exposure of the two. After revisiting this from many different angles, over and over again, and after changing the code in different ways and attempting both a mathetamical and visual calibration I've decided that this issue is going to have to wait. Here's an example of the problem:
Note that the raytracing and photon mapping graphs have similar but miss-aligned shapes. This miss-alignment is the problem. There is no easy way to simply fudge things and fix it as it will be entirely dependent on the scene being rendered. Furthermore, I'm starting to think (after talking with colleagues) that there is a mistake somewhere in either the RT code or the PM code that is causing this miss-alignment and simply fudging things to fix it is not a permanent solution (or one that I should be spending so much time on).
So, three days gone on this but at least I have something to show for it. New configuration options! (well, and a lot of frustration!) Here's the new options:
'forcerealistic' - This option can be enabled or disabled and will force all the static and pseudo-dynamic lights in a world to use 'realistic' attenuation mode. This saves the trouble of having to re-do your world in order to use it with photon mapping.
'lightpowerscale' - Scale all the lights in a scene by the given scaling factor. This scale is applied to the light color which is essentially the same as it's power. When you use 'forcerealistic' things tend to get much darker so the lights need to be scaled up to compensate. Again, this option avoids having to edit the world file to achieve this.
'pmlightscale' - Like 'lightpowerscale' this will scale all the static and pseudo dynamic lights in the scene but only for the photon mapping phase. This is in addition to any scaling applied by 'lightpowerscale'. This allows you to fudge things from the command line and bring the exposure of the photon mapping simulation and the raytracer in line with one another.
After posting about my intention to calibrate the energy between photonmapping and raytracing 'res' sent me a message concerning attenuation. This reminded me that I had previously considered that light was being attenuated in the direct lighting raytracer but didn't seem to be in the photonmapper. I had left this in light of larger problems with the photonmapper but as res pointed out, it is critical to address this prior to calibration.
So, I explored the Light class and noticed that it has an internal mechanism to call an attenuation function that will attenuate for distance based on the attenuation coefficients and mode. I decided to move this function (ComputeAttenuation()) from the protected section of the class to the public section so I could access it from the photonmapping code. So now, each photon gets attenuated after each bounce by the distance it traveled according to the attenuation parameters of the light it was emitted from. This small change already made a big difference in the quality of the simulation! It also caused the calibration problem to become even worse as everything got noticeably dimmer in the photonmap.
So, now we're ready to calibrate. To do this, I adjusted the lights from 1.0 down to 0.1 in the world file in 0.1 increments (changing each color channel equally) and generated lightmaps that contained direct light only (one with raytracing, one with photon mapping). For now, I'm just trying to do a visual comparison between the results and scale the photonmapping version until it approximately matches the raytraced version. You can see the progress below. I will add more as I am able.
Hi Again Everyone -
Haven't posted in a while, so I thought I would give an update on how things have progressed on the COLLADA convertor. Between Mike Gist and myself, we've been able to get just about everything working. The only holdup - the surface triangulator - has been implemented, and appears to be working as expected. Currently, the surface triangulator needs some testing, though, as I've only tested it with basic models. So, if you have a chance, I'd really appreciate it if all of you folks out there with COLLADA models could give it a try. Here's what we're looking at testing:
The most recent addition, in terms of features, to the system is the last one - the materials. Since we're not 100% sure how we're going to handle shaders from COLLADA to CS (if at all), the materials area of the converter may be a little sparse. And, it's also somewhat volatile.
Also, we need testers on the following systems (as I don't have access to them):
My post this week is coming noticeably early. Unfortunately this is due to me being able to do little else today. I have been having a few problems over the past fortnight with my machine, but today it decided to finally blue screen and fade away. I am sad to see its demise, but am optomistic about being able to repair it soon.
However, given that we are now in the final week of GSoC I have put that on hold for now and am in the middle of installing/recompiling all the required software/code on an old laptop for me to remain functional. This is not ideal but I almost have a working environment set up again for the final programming tasks and the documentation will be easily producable on this machine.
Although I have lost this day, I am confident that my deadlines will be met for producing the documentation.
With regards to the final programming deliverable of implementing cel expressions in triggers I have had some difficulties but hope the mailing list discussion will overcome these. If anyone reading feels they can help in anyway please contribute to the discussion.
I will not make a further post this week, but I will keep the post on progress below updated.
Kind Regards, Sam
As of 14/08/09
- Parameters, Sequences and all Rewards/Triggers/SeqOps factored out
- Clean redundant code
- Tutorial Program - Partially Complete (appquesttest)
- Update manual pages on quests
- Bug fix factor out of sequences
- Implement cel expressions into sequences and seqops
- Complete questtest
- Write tutorial
- Implement cel expressions into triggers
- Implemented all selectors, decorators and lear nodes
- Basic functioning behaviour tree demonstrated in example program
- Behaviour tree executes inside csDefaultRunLoop
- Complete bttest
- Write tutorial
- Write manual entry
Regrettably Unlikely During GSoC 2009
- Deserialize/Load methods for parsing an XML BT
- Allow for nesting FSMs
- Add constraints to sequences
- Filters for Quest Message Trigger by sender and parameters
- Make quest variables be accessible as pc properties
- Standardise start-finish protocol
I have had a very successful week implementing the Behaviour Tree(BT) I introduced in the design documentation below. I am happy to say that I now have all selectors, decorators and leaf nodes implemented and have a working example of a BT that will form the basis of the tutorial.
The final major obstacle to overcome to reach my originally planned deliverable is to have the BT execute within the default run loop. At this time, the example BT is executed once effectively before the "game" begins. However, I am confident that this issue can be overcome in the few remaining days of coding left.
In addition I am very keen to support the future integration of behaviour trees into the crystal architect editor. I am taking careful consideration of the issues raised both on the mailing list and in irc with regard to this and hope to implement as many of the required methods as I can before the GSoC period ends.
Finally, google has set the suggested pencils down date as next Monday with the remaining week dedicated to documentation. I do not think that the documentation I have left to produce will take me a whole week, however, I do want to ensure that I provide good quality documentation covering all implemented features. Therefore, my plan is to start writing on Monday at which time I hope at the very least the final BT obstacle will be overcome and committed. I aim to complete the documentation by Wednesday/Thursday next week leaving a few days remaining to tackle any left over programming tasks and further develop BT features for CA integration.
As always I will remain available on irc as often as possible and can be contacted via the mailing list. Both discussions of CA integration and BT design are still active on the mailing lists and I appreciate any and all input.
Kind Regards, Sam.
Final Gathering. No, it's not the name of yet another horror movie sequel (although Google isn't much help for figuring out what it in fact is).
It's a technique for effectively smoothing noise in global illumination from Lambertian surfaces. Basically, given a solution to global illumination (like radiosity or a photon map) instead of looking up the diffuse light value in the GI solution, you do one final bounce of light by shooting rays out across the hemisphere above the point you are rendering. These rays sample the secondary light that would be hitting this point much like a distribution raytracer would send out rays to sample the BRDF. In this case, a random sampling of the Lambertian distribution is not the best bet (according to Jensen anways). It is better to use a grid of points placed across the hemisphere according to Lambert's cosine law and then jitter these points slightly to ensure the full hemisphere gets sampled.
When these rays hit a surface a distribution raytracer would send out more rays to sample the light hitting that surface. In FG, you use the precomputed GI solution instead. So, like shadow rays, FG rays do not bounce. However, FG is intentionally used for Lambertian surfaces (perfectly diffuse surfaces). This means that the hemisphere above the point must be FULLY sampled and that takes a lot of rays. Doing this at every point in the scene is very inefficient.
Enter the irradiance cache. Diffuse lighting changes very slowly across a surface; think of a big white wall in an office (the one exception would be a caustic which is actually a diffuse effect but we'll ignore that for now). Slowly changing functions don't need to be sampled as frequently as quickly changing functions so re-computing the FG value at every point across a large surface is wasteful. Instead, we could sample it sparsely and use interpolation of nearby values to fill-in the gaps. This technique is known as irradiance caching and the math behind it is pretty intense.
We still have noise in our simulation and the best way to combat this will be with a final gathering step (something that the previous GSoC project had attempted to include but which I believe was not implemented properly). Unfortunately adding FG is going to severely tank our performance during the lighting calculation phase so (time permitting) we are going to also need an irradiance cache to make it work in a reasonable amount of time. The cache itself is quite simple (very similar in fact to a photon map) but the metrics used to determine where a new sample is needed and where a pre-existing one can be used instead are not so simple. Jensen discusses the irradiance cache in full detail in his book (although he never uses the term 'Final Gathering' that I can see) so implementation should be a matter of translating all the summations and integrals into effective code.
Photon mapping simulates both direct illumination and indirect illumination. However, the simulation of direct illumination is not as precise as a raytracing solution. Standard raytracing is very efficient and exact at simulating direct illumination and lighter2 already has a good implementation of this. The best solution would be to combine the results of raytracing and just the indirect lighting from the photon map.
To do this I've played around with ignoring the first bounce of the photons (this would be the direct illumination) and only storing photons that have scattered at least once. We then add the irradiance estimate to the direct lighting solution from raytracing. The results are quite promising but need to be calibrated. That is to say, the 'energy' in the photon mapped solution does not match the energy in the raytraced solution.
To calibrate, I think the best plan is to do some simple direct lighting simulations with just the photon map (include only first emitted photons and exclude the scattered ones). We can compare the overall brightness at different light power levels to the raytraced solution and hopefully figure out how to scale the two so that they match.
In the meanwhile, I've restructured lighter2's options a bit. Instead of just enabling direct and indirect you now specify which engine you want to use for each (raytracing or photon mapping for direct and photon mapping or none for indirect). This will make this calibration easy to perform and will give the option to those that would prefer it to use photon mapping for the entire lighting solution.
I'll add some images to support this post a little later.
I think we've got it. I did some playing around with the different sampling parameters and after some scaling and few bug fixes to make sure the energy stayed consistent no matter the number of photons and now we're getting some good results. Here's the latest light map for the Cornell Box:
Here's a table of many different photon counts (y axis) and sampling amounts (x axis) (click on any image for a full size view):
|<< <||Current||> >>|
This is the long description for the blog named 'Blog All'.
This blog (blog #1) is actually a very special blog! It automatically aggregates all posts from all other blogs. This allows you to easily track everything that is posted on this system. You can hide this blog from the public by unchecking 'Include in public blog list' in the blogs admin.