OllieBrown

With the end of GSoC approaching I want to take inventory of what has been achieved so far. To reiterate I do fully intend to stick with this and keep improving lighter2, assuming I have not overstayed my welcome.

Here's some results. I've spent today rendering images so that I can show what I can achieve with my photon mapper when I hold it's hand and try my best to get good results.

First up, an image of the Cornell Box with just basic raytracing. This scene was rendered with global ambient turned off, all lights forced to realistic attenuation and all light power scaled by 8.0, as such:

lighter2 --directlight=raytracer --noglobalambient --forcerealistic --lmdensity=20.0 --lightpowerscale=8.0 data/NewCornell

This image uses light maps with direct light only computed with raytracing. Note the dark shadows and the relatively small amount of light on the ceiling.

Next, an image of the Cornell Box with just Photon Mapping (for both direct and indirect light). Here we shot 5M photons and sampled 5K times for each density estimation. The command line was like such:

lighter2 --directlight=photonmapper --indirectlight=photonmapper --numphotons=5000000 --maxdensitysamples=5000 --sampledistance=20.0 --nofinalgather --lmdensity=20.0 --pmlightscale=100.0 data/NewCornell

This image shows the results of lighter2 photon mapping only. The image captures all of the major features of the raytraced image (shadows, shapes of light attenuation) but it also captures some indirect light (though not yet with color). The shadows and ceiling are brighter in this image than in the raytraced image. Unfortunately, the noise is still too high but better than it has been all summer.

Lastly, an image of the Cornell Box with direct light done with raytracing and indirect light with photon mapping. It was VERY difficult to get the two values to have a comprable exposure (i.e. photon mapping was consistently too dark). Recent changes to the way light is scattered have made this matter worse but are conceptually necessary to get the simulation to be correct. Needless to say, I had to fudge the light power manually until the image 'looked' okay. Very imprecise but good enough for today. Here's what the final command line looked like. Note that I bumped the number of photons up to 25M to help fight noise which can be particularly noticeable for indirect lighting:

lighter2 --directlight=raytracer --noglobalambient --forcerealistic --lightpowerscale=8.0 --indirectlight=photonmapper --numphotons=25000000 --maxdensitysamples=5000 --sampledistance=20.0 --nofinalgather --lmdensity=20.0 --pmlightscale=16.0 data/NewCornell

This image shows the results of lighter2 combining raytracing and photon mapping (for direct and indirect light respectively). Note the brighter shadows and ceiling as well as the indirect brightening happening on the rear box. All of this increases the realism of the final image at the expense of noise but also requires significant hand-holding and tweaking of variables to achieve.

The code I check in today will be able to do all of this. Note that I used an old version of walktest.exe to render these images (from the 08 SOC branch for lighter2). The one in my branch is still not working with light maps for unknown reasons.

After posting about my intention to calibrate the energy between photonmapping and raytracing 'res' sent me a message concerning attenuation. This reminded me that I had previously considered that light was being attenuated in the direct lighting raytracer but didn't seem to be in the photonmapper. I had left this in light of larger problems with the photonmapper but as res pointed out, it is critical to address this prior to calibration.

So, I explored the Light class and noticed that it has an internal mechanism to call an attenuation function that will attenuate for distance based on the attenuation coefficients and mode. I decided to move this function (ComputeAttenuation()) from the protected section of the class to the public section so I could access it from the photonmapping code. So now, each photon gets attenuated after each bounce by the distance it traveled according to the attenuation parameters of the light it was emitted from. This small change already made a big difference in the quality of the simulation! It also caused the calibration problem to become even worse as everything got noticeably dimmer in the photonmap.

So, now we're ready to calibrate. To do this, I adjusted the lights from 1.0 down to 0.1 in the world file in 0.1 increments (changing each color channel equally) and generated lightmaps that contained direct light only (one with raytracing, one with photon mapping). For now, I'm just trying to do a visual comparison between the results and scale the photonmapping version until it approximately matches the raytraced version. You can see the progress below. I will add more as I am able.

	Raytracer	PM Before	PM After
0.1
0.3
0.5
0.7
0.9

Photon mapping simulates both direct illumination and indirect illumination. However, the simulation of direct illumination is not as precise as a raytracing solution. Standard raytracing is very efficient and exact at simulating direct illumination and lighter2 already has a good implementation of this. The best solution would be to combine the results of raytracing and just the indirect lighting from the photon map.

To do this I've played around with ignoring the first bounce of the photons (this would be the direct illumination) and only storing photons that have scattered at least once. We then add the irradiance estimate to the direct lighting solution from raytracing. The results are quite promising but need to be calibrated. That is to say, the 'energy' in the photon mapped solution does not match the energy in the raytraced solution.

To calibrate, I think the best plan is to do some simple direct lighting simulations with just the photon map (include only first emitted photons and exclude the scattered ones). We can compare the overall brightness at different light power levels to the raytraced solution and hopefully figure out how to scale the two so that they match.

In the meanwhile, I've restructured lighter2's options a bit. Instead of just enabling direct and indirect you now specify which engine you want to use for each (raytracing or photon mapping for direct and photon mapping or none for indirect). This will make this calibration easy to perform and will give the option to those that would prefer it to use photon mapping for the entire lighting solution.

I'll add some images to support this post a little later.

I think we've got it. I did some playing around with the different sampling parameters and after some scaling and few bug fixes to make sure the energy stayed consistent no matter the number of photons and now we're getting some good results. Here's the latest light map for the Cornell Box:

This light map contains only photon mapped light. It was generated with 1M photons and 1000 samples per element. While the image is still too noisy we are seeing all the desirable lighting effects and the overall impression is much closer to the desired result.

Here's a table of many different photon counts (y axis) and sampling amounts (x axis) (click on any image for a full size view):

	10	100	1,000
1K			N/A
10K
100K
1M

That should be the mantra of every graphics programmer ... at least, that's what some professor told me one time.

I worked up a direct visualization of the photon emission stage by simply drawing points in space for each photon. I set the color of each point to the power of the photon and now I'm seeing something very important. The power is not attenuating ... AT ALL. That's why we aren't getting any shadows and that's probably why everything is a constant power and too dim.

I thought that I could safely ignore the photon power until milestone 2 but I think I need to deal with it now so that's going to be the current task.

Here's the visualizations:

Directly visualizing the Cornell Box Photon Map with 1000 photons emitted total (250 from each light).

Directly visualizing the Cornell Box Photon Map with 10,000 photons emitted total (2,500 from each light).

Directly visualizing the Cornell Box Photon Map with 100,000 photons emitted total (25,000 from each light).

Directly visualizing the Cornell Box Photon Map with 1,000,000 photons emitted total (250,000 from each light).

Note: the number of photons listed is the number of emitted photons. Since photons are recorded each bounce there are actually MANY more being added to the map and drawn. With russian roulette in play, the photons are bouncing about 5 times on average so multiply the number of emitted photons by 6 to get the number being drawn and the number of rays being traced. This is all still happing quite efficiently. The last case has about 6M rays to trace and it does so in only a few minutes. Not bad! Unfortunately, the splatting/final gather phase is painfully slow still. I think it's because the kd-Tree for the photon map is not being properly balanced.

Some Observations about these images:

• Almost no photons are landing under the boxes (which is as it should be). The ones that appear to be there are actually on the front face of the boxes. I thought this was a problem as there is light appearing in the light map in these areas but it must be getting there in the splatting/gathering phase
• The photons are VERY uniformly distributed. Again, this is as it should be since we are doing purely diffuse bounces.
• The shadows are missing. My new theory is that this is because we are not attenuating the power after each bounce
• Each scene just gets brighter and brighter. This shouldn't happen as the power should be evenly divided between the photons emitted. This is also a problem with the power attenuation.

So far, it's been a lot of house cleaning. There's still several key problems with the photon map algorithm that did not resolve themselves as I expected.

The key change was to the photon emitting phase. I added a progress structure to this phase so that we could see when it was happening and how many rays it was creating. More importantly, I changed the photon scattering code to scatter photons diffusely instead of specularly. In the end, we are going to need both but for now, the diffuse scattering is more important and I don't think the specular scattering was being done right anyways. My hope is that by changing to diffuse and by ramping up the number of photons being emitted we would get better results right away. This has not been the case.

There are two key problems in the final light maps that have yet to be resolved:

1. They are WAY too dim. About a tenth of the brightness we get with the direct lighting version.


2. There are no shadows. PM should get the shadows if you shoot enough photons and we're shooting millions so they should be there.

While these are not the only problems, these problems are the most troubling ones and ones that I theorized were caused by improper photon scattering.

To proceed, I'm going to finish up the scattering with both diffuse and specular components chosen with statistical russian roulette (exactly as suggested in Jensen's book) and then start working on the splatting / gathering phase of the simulation. The code for this phase comes straight from Jensen's book so mostly I'm just going to confirm that its correct before I start to play with it and debug the implementation.

Here's some visuals for what's going on. These images are the actual lightmaps generated by lighter2. In both cases lmdensity was set to 10.0 so that the images generated would be high enough resolution to examine directly:

This is the current production state of lighter2. Only direct illumination is simulated which captures shadows well but no global effects.

This is the state of the photon mapping implementation after my changes (and with an artificial brightening by 4 throughout). Shadows are missing, flat surfaces are noisy and light is getting into places it shouldn't.

A New Plan
Thanks to all who offered feedback for my previous post. With the discovery of the GSoC '08 branch for lighter2 with photon mapping plans need to change. I've been examining Greg Hoffman's changes to lighter2 to determine what work could be done and I think there's a good chunk here to constitute a project. Here's my assessment of what the branch contains:

• There's a basic Photon Map data structure and code to emit and gather photons in a single sector.
• This code does seem to do something but I don't think it's correct in all cases (or at least robust) yet
• There are things missing (proper handling of materials, diffuse to diffuse light paths)
• It's missing some options to fine tune the convergence (no max error, max recursion depth, no control over number of photons emitted)
• Right now it's slow and has room for optimization

So, it seems given the original content of my proposal and this discovery from last summer that the new course of action should be to work on the photon mapping implementation. So, here's a basic outline of what I could do again welcoming comments:

Milestone 1: Repair

1. Ensure the PM calculation is correct and fix where needed (such as handling LD+SE paths)


2. Add the missing settings for controlling convergence


3. Ensure it scales well from small test cases (like the Cornell Box) to large game levels

Milestone 2: Improve Quality

1. Handle light traveling across portals


2. Handle all materials properly (materials are ignored right now)

Milestone 3: Improve Speed/Features

1. Add importance sampling to avoid redundant photon emission


2. Move calculations to GPU (BIG speedup)


3. Optional: Add support for 'dielectrics' (refracting materials) and caustics (via reflection and refraction)

Concerning the optional task under Milestone 2, Photon Mapping just handles caustics well (it's famous for it) and as such it would be easy to render this if the information about refraction is available in the material structure (namely index of refraction). It could make for some interesting but very specialized effects.

Time-line
I'm planning about two weeks for each milestone with an extra week for the first one just for getting out of the starting gate. Here's a rough time-line to completion of these milestones:

• Milestone 1 (already in progress): Now - July 21st
• Milestone 2: July 22nd - Aug 4th
• Milestone 3: Aug 5th - Aug 18th

I want to make sure that the amount of work I'm doing is worthy of a full SoC project regardless of the time frame. I'm definitely slow getting started here and I want to ensure all involved that I will make that up as we go either by putting in extra time now or beyond the scheduled GSoC end. Therefore, I think it is best to make sure I get a project defined that is of a scope appropriate for SoC so that no one feels short changed.