Although I have completed this dissertation, the actual program was not a success.  Therefore, I will be keeping this blog alive with the intention of picking up the program once my computer has made its way over here to sunny Sweden!

Dave

Introduction

Post-processing is a technique fundamental to graphics programming today. It allows for effects to be realised in great amounts of detail. This honours project will make use of this technique to demonstrate the potential it has to create visually stunning and accurate effects. This demonstration will be achieved by applying atmospheric scattering to a scene as a post-process to create very realistic skies.

Atmospheric scattering has been chosen as it is very realistic. It describes how incoming light from the sun is affected by the atmosphere to produce the stunning aura of colours observed daily. Another visual effect created with atmospheric scattering is that of haziness. This haze adds to the realism of the game environment and also gives the player much better depth cues.

Research Question

Atmospheric scattering has a reputation for being an expensive effect to implement. Post-processing could speed this up. To that effect, the question being presented is:

What are the benefits of using post-processing techniques to simulate atmospheric scattering in a computer game and what are the issues associated with its implementation?

Literature Review

“Modelling surfaces with polygons is often the most straightforward way to approach the problem of portraying objects in a scene. Polygons are good only up to a point, however. Image-based rendering (IBR) has become a paradigm of its own.” (Möller, T., et al, 2008)

As Möller et al discuss in their book, Real-time Rendering, there are many advantages to using post-processing in a scene. Complex objects like fur and clouds can more easily be added in this 2D image space after the entire scene has been rendered first. Complex lighting techniques, such as atmospheric scattering, are also ideally suited for post-processing. The reason this is true is because in post-processing you are only operating on the final pixel colour. There is no pixel overdraw associated with this technique.

To include post-processing in a game, one must first create an off-screen buffer. The entire scene is then rendered to this buffer instead of the back buffer. Once the drawing is complete, this off-screen buffer must be displayed on the screen. To do this, it is simply added as a texture to a quad that fills the screen. These are the principles of post-processing. The entire process is analogous to editing a photograph, where the photographer has a 2D image and adds Gaussian blurs, increases contrast, and decreases saturation for example, resulting in an altered image. By passing filters over the off-screen buffer, the programmer is able to achieve the same effect as in the photography example.

One important consideration with applying 3D effects, such as atmospheric scattering, as a post-process is that the depth buffer must also be resolved to another texture. The information in this texture must then be reconstructed in world-space to obtain the depth of each pixel. This last step is important as the depths are stored in a non-linear format.

The principles of atmospheric scattering are simple. Two coefficients must be solved individually and then used in the complete equation. These two coefficients are Rayleigh and Mie. Rayleigh describes what happens to light as it passes through the gases of the atmosphere itself, which determines sky colour. During the day the short wavelengths are scattered the most resulting in the blue sky. At sunrise and sunset, when the light travels through more of the atmosphere, the longer wavelengths are scattered more and this gives the red and orange skies.(Tomoyuki et al. 1993)

Mie scattering on the other hand describes how light from the sun interacts with more dense molecules in the air, such as dust and water vapour. This results in haze which limits how far a person can see. According to psychologists “aerial perspective is a fundamental depth cue that humans use to estimate distances, and the only absolute depth cue available for distant unfamiliar objects.” (Preetham, A. J., et al 1999). It is therefore vital that aerial perspective be implemented properly.

These coefficients are then used in the scattering equation to determine the final colour of the pixel. The equation is given by L(s, ?) = L_oF_ex(s)+ L_in(s, ?); where L_o is the initial colour of the pixel, F_ex is the extinction factor which has the effect of darkening objects that are further away and finally L_in is the inscattering component which has the effect of adding more light to distant objects. The function takes two arguments. The first argument, ‘s’, is the distance of from the viewer to the object and the second, ?, is the angle viewer-object vector and the sun direction.

There are two main approaches to implementing atmospheric scattering equations. One is to use the method proposed by A. J. Preetham in 1999 which is an analytic model. His model uses many simplifications and precomputations that give a reasonable effect, but limits the scalability. Another model for consideration is the model proposed in 1993 by Tomoyuki, N., et al. in 1993. This model is far more general and the ability to scale coefficients dynamically is possible. The downside to this model however is its inherent computational expense. Unlike Preetham’s model, Tomoyuki et al designed their model to allow the atmosphere to be viewed from space.

For adding realism to outdoor scenes in CryEngine2, Carsten Wenzel used the model proposed Tomoyuki et al 1993. However, only the skylight model was used and the aerial perspective was neglected. Instead they went with another implementation, which does exactly what the aerial perspective model proposed by Tomoyuki et al would do. This work around is then rather redundant as there are now two separate systems that handle the same thing.

After e-mail discussions with Damianno Iannetta of Rare Ltd on 7^th October 2009 it was recommended that this dissertation also use the model proposed by Tomoyuki et al and use the correct model for aerial perspective to give a more accurate effect.

Methodology

For the implementation of this project OpenGL will be the graphics API of choice. It has been chosen for its ease of use and functionality as well as its performance. The target platform will be Windows, however the framework will make use of SDL as it handles all the events very nicely and requires very little to set up and manage.

A simple scene will be required so that the user has a point of reference for up and down, left and right. This will take the shape of a simple heightmapped terrain. This terrain will be lit and textured in the conventional way with GLSL shaders. The user will be able to walk on, or fly around the terrain to view the effect from different angles.

As for the effect itself, floating point framebuffers will be used to store the fullscreen effect. The reason for this is to add high dynamic range rendering to the scene to make it look sharper. This buffer will then have the entire scene rendered to it before being applied as a texture to a full screen quad. Another texture will need to be used to hold all the depth information for the scene. This will be created in another pass and sampled from at the same time as the fullscreen effect. Together these two textures will contain the information needed for the effect.

Finally, a small scale GUI will have to be created to allow for dynamic changing of variables. This sort of feature is desirable as it allows for different settings to be created in no time at all and the effect seen instantly. This will be done using OpenGL to create sliders and display text information.

Evaluation

When evaluating this particular project there are a few important questions to answer:

1. Does the effect look good, does it do what it should?

2. How has using post-processing techniques benefitted this effect?

The first question is rather subjective, but it is an important and relevant question as the aim of this dissertation is to create something that is visually stunning and accurate. If it does not look good then the variables might need scaling. The second question is one that is slightly harder to answer, given that this dissertation will be focused on post-processing only, however data available from past research in the field of atmospheric scattering should provide some good comparisons.

Issues

As with any project, there are some issues that need addressing. For example, with post-processed atmospheric scattering, the depth buffer must be stored in a texture for sampling in the shader. Traditionally this requires another pass, however it might be possible to store this depth information in the same pass as the initial rendering by storing it in the alpha channel of the scene texture.

While post-processing reduces the number of times that the expensive scattering equations are performed, the equations can still have a detrimental effect on performance. Use of lookup tables and pre-computing coefficients may help alleviate this.

Resources Required

The resources required for this project are fairly minimal. Access to a computer with a suitable graphics card, such as an NVIDIA 8800GT, will suffice. Some sort of debugger for GLSL would be ideal, but not essential as any problems arising will most likely result in visual artefact, making them easier to track down.

Conclusion

At the end of the project it should be clear what the benefits are of post-processing over physical modelling. Some issues may arise that can provide a basis for future work however, most issues that arise should be manageable. Most importantly, by the end of this project there will be a strong application demonstrating atmospheric scattering as a stunning effect that adds accurate skies and usable depth cues for gamers.

References and Bibliography

Möller, T., Haines, E. and Hoffman, N. 2008. Real-time rendering. 3rd ed. Wellesley, Mass.: A.K. Peters

N Hoffman, A. J. P. 2002. Photorealistic real-time outdoor light scattering. Game Developer. pp.32-38.

O’Neil, S. 2005. GPU gems 2 : programming techniques for high-performance graphics and general-purpose computation. London: Addison-Wesley

Preetham, A. J., Peter, S. and Brian, S. 1999 In Proceedings of the 26th annual conference on Computer graphics and interactive techniquesACM Press/Addison-Wesley Publishing Co.

Tomoyuki, N., et al. 1993 In Proceedings of the 20th annual conference on Computer graphics and interactive techniquesACM, Anaheim, CA.

Wenzel, C. 2006 In ACM SIGGRAPH 2006 CoursesACM, Boston, Massachusetts.

Yoshinori, D., Tsuyoshi, Y. and Tomoyuki, N. 2002 In Proceedings of the ACM SIGGRAPH/EUROGRAPHICS conference on Graphics hardwareEurographics Association, Saarbrucken, Germany.