The Future of Triangles Part 4: Pie in the Sky

Per-pixel lighting is something I hope to have in X-Plane soon.  A number of other features will take longer, and quite possibly might never happen.  This is the "pie in the sky" list - with this list, we're looking at higher hardware requirements, a lot of development time, and potential fundamental problems in the rendering algorithm!

High Dynamic Range (HDR) Lighting

HDR is a process whereby a program renders its scene with super bright and super dark regions, using a more detailed frame-buffer to draw.  When it comes time to show the image, some kind of "mapping" algorithm then represents that image using the limited contrast available on a computer monitor.  Typical approaches include:

• Scaling the brightness of the scene to mimic what our eyes do in dark or bright scenes.
• Creating "bloom", or blown out white regions, around very bright areas.

Besides creating more plausible lighting, the mathematics behind an HDR render would also potentially improve the look of lit textures when they are far away.  (Right now, a lit and dark pixel are blended to make semi-lit pixels when far away as the texture scales down.  If a lit pixel can be "super-bright" it will still look bright even after such blending.)

Besides development time, HDR requires serious hardware; the process of drawing to a framebuffer with the range to draw chews up a lot of GPU power, so HDR would be appropriate for a card like the GeForce 8800.

While there aren't any technical hurdles to stop us from implementing HDR, I must point out that, given a number of the "art" features of X-Plane like the sun glare, HDR might not be as noticeable as you'd think.  For example, our sun "glares" when you look at it (similar to an HDR trick), but this is done simply by us detecting the view angle and drawing the glare in.

Reflection Mapped Airplanes

Reflection maps are textures of the environment that are mapped onto the airplane to create the appearance of a shiny reflective surface.  We already have one reflection map: the sky and possibly scenery are mapped onto the water to create water reflections.

Reflection maps are very much possible, but they are also very expensive; we have to go through a drawing pass to prepare each one.  And reflection maps for 3-d objects like airplanes usually have to be done via cube maps, which means six environment maps!

There's a lot of room for cheating when it comes to environment maps.  For example: rendering environment maps with pre-made images or with simplified worlds.

Shadows

Shadows are the biggest missing feature in the sim's rendering path, and they are also by far the hardest to code.  I always hesitate to announce any in-progress code because there is a risk it won't work.  But in this case I can do so safely:

I have already coded global shadow maps, and we are not going to enable it in X-Plane.  The technique just doesn't work.  The code has been ripped out and I am going to have to try again with a different approach.

The problem with shadows is the combination of two unfortunate facts:

• The X-Plane world is very, very big and
• The human eye is very, very picky when it comes to shadows.

For reflections, we can cheat a lot -- if we don't get something quite right, the water waves hide a lot of sins.  (To work on the water, I have to turn the waves completely off to see what I' m doing!)  By comparison, anything less than perfect shadows really sticks out.

Shadow maps fail for X-Plane because it's a technology with limited resolution in a very large world.  At best I could apply shadows to the nearest 500 - 1000 meters, which is nice for an airport, but still pretty useless for most situations.

(Lest someone send the paper to me, I already tried "TSM" - X-Plane is off by about a factor of 10 in shadow map res; TSM gives us about 50% better texture use, which isn't even close.)

A user mentioned stencil shadow volumes, which would be an alternative to shadow maps.  I don't think they're viable for X-Plane; stencil shadow volumes require regenerating the shadow volumes any time the relative orientation of the shadow caster and the light source change; for a plane in flight this is every single plane.  Given the complexity of planes that are being created, I believe that they would perform even worse than shadow maps; where shadow maps run out of resolution, stencil shadow volumes would bury the CPU and PCIe bus with per-frame geometry.  Stencil shadow volumes also have the problem of not shadowing correctly for alpha-based transparent geometry.

(Theoretically geometry shaders could be used to generate stencil shadow volumes; in practice, geometry shaders have their own performance/throughput limitations - see below for more.)

Shadows matter a lot, and I am sure I will burn a lot more of my developer time working on them.  But I can also say that they're about the hardest rendering problem I'm looking at.

Dynamic Tessellation

Finally, I've spent some time looking at graphics-card based tessellation.  This is a process whereby the graphics card splits triangles into more triangles to make curved surfaces look more round.  The advantage of this would be lower triangle counts - the graphics card can split only the triangles that are close to the foreground for super-round surfaces.

The problem with dynamic tessellation is that the performance of the hardware is not yet that good.  I tried implementing tessellation using geometry shaders, and the performance is poor enough that you'd be better off simply using more triangles (which is what everyone does now).

I still have hopes for this; ATI's Radeon HD cards have a hardware tessellator and from what I've heard its performance is very good.  If this kind of functionality ends up in the DirectX 11 specification, we'll see comparable hardware on nVidia's side and an OpenGL extension.

(I will comment more on this later, but: X-Plane does not use DirectX - we use OpenGL.  We have no plans to switch from OpenGL to DirectX, or to drop support for Linux or the Mac.  Do not panic!  I mention DirectX 11 only because ATI and nVidia pay attention to the DirectX specification and thus functionality in DirectX tends to be functionality that is available on all modern cards.  We will use new features when they are available via OpenGL drivers, which usually happens within a few months of the cards being released, if not sooner.)