Introduction:

The focus of this dissertation is on atmospheric scattering as a post-processed effect.  Atmospheric scattering is the reason that our sky has colour.  It describes the way that the light from the sun disperses when it comes through our atmosphere.  In terms of game development, it can add REAL depth cues to vast landscapes and increase the immersion that the player experiences.  Plus, it looks good!

CryEngine2 uses this effect very well and creates astoundingly beautiful environments :

The image on the left is of Kualoa Ranch in Hawaii and the image on the right is the CryEngine2 replica.

In this implementation, though, the aerial perspective calculations used were not those defined by the atmospheric model they used.  Instead, they forced their own implementation to work.  Though it looks good, it is still not as good as it should be.  The aim of this honours project, then, is to use the same atmospheric model as CryEngine2 and to add the correct calculations for aerial perspective.

Issues:

The main issue surrounding this effect is that it is very expensive to calculate.  In the original implementations – see Nishita93 and Preetham99 – highly tessellated sky hemispheres were used.  This meant that for better representations of the sky, more vertices had to be used.  In a game, this would have a serious effect on performance.

Introduction

Atmospheric effects in games are becoming increasingly more popular; however some implementations are less efficient than others, giving arguably better results – (Tomoyuki et al. 1993) vs. (Preetham et al. 1999) for example. These models can be performed on a per-pixel basis to provide very realistic looking skies in games. It was suggested in an e-mail discussion with Damianno Iannetta of Rare on 7^th October 2009 that this project should steer itself toward the model proposed in (Tomoyuki et al. 1993) and as such, this dissertation will be focusing on an efficient post-processed implementation of this technique for PC and/or PS3, making the most use of hardware – i.e. GPGPU and Cell for PC and PS3 respectively.

Motivation

“Modelling surfaces with polygons is often the most straightforward way to approach the problem of portraying objects in a scene. Polygons are good only up to a point, however. Image-based rendering (IBR) has become a paradigm of its own.” As written by (Möller et al. 2008) in their book Real-Time Rendering begins to explain the importance of post-processing a 3D scene in a computer game. Things like clouds and fur are too complex to be represented as models. By creating images of the scene, these objects can be added as a post-process at a potentially lower cost – dependent on screen resolution and technique used – to look more realistic.

In a similar manner, this technique can be used to create realistic renderings of the sky between sunrise and sunset as demonstrated by (Wenzel 2006) using CryEngine2 and (Tomoyuki et al. 1993). The atmospheric effect put forward for use in CryEngine2, however, does not use the aerial perspective model put forward by (Tomoyuki et al. 1993), only the scattering model is used. It is therefore desirable to complete this technique by adding the correct method for aerial perspective.

There are numerous benefits to using accurate models for atmospheric conditions. Firstly, when a user begins to play a game, the visuals presented are one of the first things that will be noticed. Certainly, introducing magnificent effects will only aid in increasing the immersion experienced by the player – if supported by sufficient gameplay. According to a recent survey conducted by Steam last month 65.38% of gamers using Steam own an NVIDIA graphics cards and a further 27.26% own an ATI GPU (Steam 2009) . This implies that the majority of PC gamers at least are expecting to buy games with high graphics capabilities.

Secondly, great looking images can help with marketing. Again, when screenshots are released to the public it is important that the game looks appealing and carefully selected graphical effects can enhance a game’s reputation. Another example of how accurate models for atmospheric conditions can enhance immersion is in the simulation genre which includes games such as the Gran Turismo series by Polyphony Digital where the aim is to create the most visually stunning products possible as well as with the most realistic physics. This particular series includes some races that can last several hours and having a sky change over time can help solidify the feeling of time passing in game.

The problem with creating the most accurate model, however, is the effect it can have on gameplay. If it is not coded carefully enough then it could ruin the illusion of being in another world. Other things that can affect performance are the hardware of the player, especially for a PC based product where the variations in player hardware is almost infinite. Hardware such as the NVIDIA G80, G92 and GT2xx series graphics cards are ideal for these graphically intensive games with plenty of statistics supporting especially the latter series in iBXT Labs “NVIDIA GeForce GT2XX Reference” which goes on to explain technologies such as CUDA and PhysX and the acceleration they provide (Berillo 2009).

This brings us to the age old compromise between accuracy and efficiency. This dissertation will strive to get the very most accuracy with the least impact on efficiency. It is hoped that a full implementation of (Tomoyuki et al. 1993) can be achieved without compromise.

Research Question

The research question proposed, then, is:

Can atmospheric scattering be implemented efficiently in computer games, with minimal compromise in accuracy?

Addressing the question

The main issue being addressed is achieving maximum accuracy without affecting the performance of the video game. It has been seen in various white papers that performance is an issue with regard to atmospheric scattering. According to O’Neil in GPU Gems 2, using the methods described in (Tomoyuki et al. 1993) would require upto 300 computations per vertex using 5 samples for each scattering equation. (Pharr and Fernando 2005). It is obvious that using more samples for greater accuracy can greatly increase the number of computations. This project will be calculating the atmospheric model on a per-pixel basis as a post-process to eliminate the need for highly tessellated sky domes, which can help reduce computations further if care is taken. So in the case of the previous example, there would be 300 computations per-pixel.

The basic principal of post-processing is to render the entire scene to an off screen buffer and then apply this as a texture to a quad the size of the screen. An issue with performing atmospheric scattering as a post-process is that all the per-pixel calculations are performed in 2D space. This means there is no depth associated with each pixel and depth is required for attenuation of distant objects for correct aerial perspective. The work around for this is to create another texture that stores just the depth – the same technique used for shadow mapping. This texture can then be sampled at the same time as the scene texture to give distance. (i.e. pixel(ij) = depth(ij))

The final problem associated with this technique is of course efficiency. While it is highly accurate, this comes at the cost of performance, which in games is important. Some options exist for increasing performance without sacrificing accuracy. Firstly, a number of constants can be pre-computed as they are never likely to change during run-time. Another option is multi-threading – a very important concept in computing – meaning that several parts can be computed at any one time in parallel.

One such example of this is GPGPU or General Purpose GPU. GPGPU is an interesting development in graphics hardware as it allows for general purpose computation on the GPU directly, meaning that physics solvers, as an example, could be computed off the CPU either in part or whole, leaving the CPU free to handle other tasks, such as AI, gameplay, audio etc. It should be noted that the most appropriate use of GPGPU is with tasks that lend themselves well to parallelisation. Such tasks could be spread over roughly 12,288 threads on an NVIDIA 8800GTX GPU, implying that there could be a potentially huge performance increase. (David Luebke 2008) This means that potentially, the screen can be split into regions to be worked on by different threads on the GPU, potentially leading to an increase in performance.

On the PS3 there exist 7 programmable cores as well as a 256MB NVIDIA GPU, all of which can be optimised to run this post-processed effect. For example, an SPU could be used to calculate Rayleigh and Mie coefficients for scattering and then the GPU would only have to apply these numbers to each pixel in post-processing the scene. This could also lead to an increase in speed and it this technique that will be researched further for the purposes of the dissertation.

What equipment/resources will be needed/used

Access to the SIGGRAPH database for reference material is required as is a PS3 SDK. Some form of debugger for the PS3 would be desirable for speeding up the bug fixing process. There is little need for 3D models or high quality textures, as the effect is only applied to a texture and generates all required imagery for the effect to look realistic.

Reference

Berillo, A. 2009. NVIDIA GeForce GT2XX Reference. [online]. iXBTLabs.com. Available from: http://ixbtlabs.com/articles3/video/spravka-gt2xx-p2.html [Accessed 20 October 2009].

David Luebke, M. G., John Owens, Kevin Skadron. 2008. GPU Computing: The Democratization of Parallel Computing. [online]. Architectural Support for Programming Languages and Operating Systems. Available from: http://gpgpu.org/static/asplos2008/ASPLOS08-1-intro-overview.pdf [Accessed 19 October 2009].

Möller, T., Haines, E. and Hoffman, N. 2008. Real-time rendering. 3rd ed. Wellesley, Mass.: A.K. Peters

Pharr, M. and Fernando, R. 2005. GPU gems 2 : programming techniques for high-performance graphics and general-purpose computation. London: Addison-Wesley

Preetham, A. J., Peter, S. and Brian, S. 1999 In Proceedings of the 26th annual conference on Computer graphics and interactive techniquesACM Press/Addison-Wesley Publishing Co.

Steam. 2009. Steam Hardware Survey. [online]. http://steampowered.com. Available from: http://store.steampowered.com/hwsurvey/ [Accessed 22 October 2009].

Tomoyuki, N., et al. 1993 In Proceedings of the 20th annual conference on Computer graphics and interactive techniquesACM, Anaheim, CA.

Wenzel, C. 2006 In ACM SIGGRAPH 2006 CoursesACM, Boston, Massachusetts.

I have formulated several possible questions thus far and all are related to the topic of atmospheric scattering, as this is my main topic of interest.  I shall list them below:

• Can atmospheric scattering be used accurately in computer games?
• Is it feasible to include atmospheric scattering in computer games?
• Can post-processed effects – such as atmospheric scattering – be used in computer games without hindering performance?
• Can post-processed effects – such as atmospheric scattering – be used in computer games without negatively affecting overall performance?
• Can atmospheric scattering be implemented efficiently in computer games?
• Can atmospheric scattering be used as an effective depth cue in computer games?
• Can atmospheric scattering be used as an effective enhancement to traditional global illumination models?

While the last one holds the most interest for me, I do not think there is enough time to play with this question as it would involve the use of radiosity, ray tracing or photon mapping.  With that in mind, I feel that the question that will allow me to explore atmospheric scattering and post-processing would be the third-last question:

“Can atmospheric scattering be implemented efficiently in computer games?”

With this particular question, the suggested scope should be small enough to investigate GPGPU techniques and post-processing as well as provide me the choice of using either Preetham or Nishitas’ method for the atmospheric scattering model.

It was also brought to my attention earlier today that using Nishitas’ model would allow an appropriate use of GPGPU techniques as there are lots of variables to compute over several frames.

On a final note then, the research question I am putting forward is:

“Can atmospheric scattering be implemented efficiently in computer games?”

First I must state the topic of interest.

I have a keen interest in landscape photography and lighting composure is a key aspect in getting the perfect photograph.  In particular, sunset and sunrise are very pleasant effects.  I am interested in mimicking these atmospheric conditions in an interactive environment using post-processing techniques.

Currently the issues surrounding these post-processed effects is efficiency in real-time.  Various post-processing effects are used in Adobe Photoshop and The GIMP, where time isn’t really of the essence.  With atmospheric effects – such as scattering – game programmers and artists have traditionally used rough approximations due to computational expense at the hands of mathematical complexity, leading to less believable – and thus less immersive – environments.  However, these effects are becoming more popular now due to better hardware.

As stated I intend to mimic atmospheric scattering effects by applying it as a post-processed effect.  This principally means taking a screenshot of the framebuffer and attaching it to the depth buffer and applying this texture to a quad the size of the screen.  From here it is passed straight to a pixel/fragment shader where the effects are applied.  I will primarily stick to the methods outlined by Preetham in his 1999 SIGGRAPH paper “A Practical Analytic Model for Daylight” as the techniques he outlines have remained virtually unchanged.

There are two ways to implement these techniques:

1. The traditional CPU and GPU combination
2. The new GPGPU – General Purpose GPU – methods

In this project I will be looking into method 2 using NVIDIAs’ CUDA technology.  Due to lack of access, I will not be able to create an implementation using AMD Stream as I do not have access to a required ATI card.  I would be interested as well in creating a port of this project to a major console – time permitting – such as the PS3 or the XBox360.  However, I doubt there will be much time left over to try and make this port.

The aim of this honours project is to create a visual effect that can be used in games to create stunning and practical depth-cues for vast landscapes.