Sunday, April 04, 2010

Ray Tracing: Shoot Before You Fly

I'll take a break from iPad drivel for a few posts; at least one or two of you don't already own one. (Seriously, it's simply easier to blog about X-Plane for iPad because it is already released; a lot of cool things for the desktop are still in develpment.)

In response to my comments on water reflections in X-Plane 950, some users brought up ray tracing.

My immediate thought is: I will start to think more seriously about ray tracing once it becomes the main technology behind first person shooters (FPS).

Improvements in rendering technology come to FPS before flight simulators (and this is true for the combat sims and MSFS series too, not just us). Global shadows, deferred rendering, screens-space ambient occlusion...the cool new tricks get tried out on FPS; by the time they make it into a flight simulator the technology has moved from "clever idea" to "standard issue."

Consider that X-Plane now finally has per-pixel lighting. Why didn't we have it when the FPS first did? Well, one reason is that the FPS were cheating. If you look at the papers suggesting how to program per-pixel lighting, at the time there were all sorts of clever techniques involving baking specular reflections into cube maps and other such work-arounds to improve performance. These were necessary because titles at the time were doing per-pixel lighting on hardware that could barely handle it. X-Plane's approach (as well as other modern games) is to simply program per pixel lighting and trust that your GeForce 8800 or Radeon 4870 has plenty of shader power.

I believe that the reason for the gap between FPS and flight simulators come from two sources:
  1. Viewpoint. You can put the camera quite literally anywhere in a flight simulator, and thus the world needs to look good from virtually any position. By comparison, if your game involves a six foot player walking on the ground (and sometimes jumping 10 feet in the air) you know a lot about what the user will never see, and you can pull a lot of tricks to reduce the performance cost of your world based on this knowledge. (This kind of optimization applies to racing games too.

    To give one simple example of the kind of optimization a shooter can make that a flight simulator cannot, consider "portal culling". A portal-culled world is one where the visibility of distinct regions have been precomputed. A trivial example is a house. Each room is only visible through the doors of the other rooms.* Thus when you are walking through a room, virtually no other room is being drawn at all. The entire world is only 20 by 20 meters. Thus the developers know that they have the entire hardware "budget" of computing power to dedicate to that one room and can load it up with effects, even if they are still expensive.

    (A further advantage of portal culling is a balance of effects. Because rooms are not drawn together in arbitrary combinations, the developers may find ways to cheat on the lighting or shadowing effects, and they know nothing will "clutter" the world and ruin the cheats.)**

  2. Often the FPS will have pre-built content, rather than user-configurable content. Schemes like portal culling (above) only work when you know everything about the world ahead of time and can calculate what is visible where. The same goes for many careful cheats on visual effects.

    But a flight simulator is more like a platform: users add content from lots of different sources, and the flight simulator rendering engine has to be able to render an effect correctly no matter what the input. This means the scope of cheating is a lot smaller.

    Consider for example water reflections. In a title with pre-made content, the artists can go into the world in advance and mark items as "reflects", "doesn't reflect", reducing the amount of drawing necessary for water reflections. The artist simply has to look around the world and say "ah - this mountain is no where near a lake - no one will notice it."

    X-Plane can't make this optimization. We have no idea where there will be water, or airports, or you might be flying, or where there might be another multiplayer plane. We know nothing. Everything is subject to change with custom scenery. So we can't cheat - we have to do a lot of work for reflections, some of which might be wasted. (But it would be too expensive in CPU power to figure out what is wasted while flying.)

Putting it all together, my commentary on ray-tracing is this: the FPS will be able to integrate small amounts of ray tracing first, because they will be in a position to deploy it tactically, using it only where it is really necessary, in hybrid ray-trace + rasterized engines. They'll be able to exclude big parts of the scene from the ray tracing pass, improving performance. They'll be able to "dumb down" the quality of the ray trace in ways that you can't see, again improving performance. The result of all of this will be some ray tracing in FPS when the hardware is just barely ready.

For a flight simulator, it will take longer, because we'll need hardware that can do a lot more ray tracing work. We won't know as much about our world, which comes from third party content, so we won't be able to eliminate visually unimportant ray traces. Like deferred rendering, shadow mapping, SSAO and a number of other effects, flight simulators will need more computing power to apply the effects to a world that can be modified by users.

(Is ray tracing even useful, compared to rasterization? I have no prediction. Personally I am not excited by it, but fortunately I don't have to make a good guess as to whether it is the future of flight simulation. The FPS will be able to, by effective cheating, apply ray tracing way before us, and give us a sneak peak into what might be possible.)

* There never are very many windows in those first person shooters, are there?
** To be clear: there is nothing negative about the term "cheat" in computer graphics. A way to cheat on the cost of an algorithm means the developers are very good at their jobs! "Cheating" on the cost of algorithm means more efficient rendering. If the term cheating seems negative, substitute "lossy optimization".

10 comments:

Christer Hellholm said...

I do hope that Laminar Research will at spare time create a new fork that is an X-Plane with OpenRT. When time is right (i.e. general hardware requirements and business model), Laminar can easily switch to X-Plane OpenRT fork. Like Apple did with Mac OS X PowerPC and Mac OS X Intel.

I know that Laminar Research do have limited resources, but test developing an OpenRT version at spare time shouldn't cost so much.The positive side effect is that you're building OpenRT competence.

Dan said...

What about atmospheric scattering? Is it just as taxing on the hardware as ray tracing?

Anonymous said...

do you have any idea what fsx uses?
on siny surfaces you can see reflections of trees sky and land. is this a form of ray tracing or just a visual trick?

Benjamin Supnik said...

FS X is almost certainly using environment mapping.

http://en.wikipedia.org/wiki/Reflection_mapping

It's not ray tracing - it's a "trick". Tricks are good. Tricks mean 30 fps, not 0.3 fps. :-)

Steve said...

What sort of trick might work to simulate the sort of effect ray tracing provides in the way of self shadows? Like the shadow that a high mounted wing will cast on the fuselage of an aircraft? Since most X-Plane flying happens at altitude, ray traced ground shadows are less useful than shadows on the aircraft. This might limit the amount of ray tracing needed, allowing for frame rate to remain high.

Benjamin Supnik said...

"What sort of trick might work to simulate the sort of effect ray tracing provides in the way of self shadows?"

Shadow mapping.

http://en.wikipedia.org/wiki/Shadow_mapping

Stencil shadow volumes would also technically work

http://en.wikipedia.org/wiki/Shadow_volume

but shadow mapping has a number of advantages over stencil shadow volumes for a flight simulator.

Steve said...

Interesting. Does shadow mapping account for differences in light angle? Is it also something that the aircraft author would need to provide, or would X-Plane calculate the "volume?" Is this a possibility for XP 10?

Benjamin Supnik said...

Shadow mapping does correctly change the shadows with response to a change in light direction.

Shadow mapping _can_ cope with non-parallel light rays (think of a lamp casting light in all directions) or parallel (think of the sun - so far away, rays are nearly parallel).

Shadow mapping does not require authors to annotate or modify their content in any way.

Shadow mapping does NOT provide translucent shadows, so translucent windows tend to be a problem. Usually the best approach is to simply not shadow them.

I cannot comment on any future features in any future version of X-Plane.

Anonymous said...

If these "tricks" were implemented in x-plane then fsx would have nothing left.
X-plane has made huge improvements lately and if it used shadow mapping and environment mapping i can only imagine how good it would look

Jack said...

Well said. Companies should achieve a good flight simulation engine first and then put an FPS into it later. I've been trying to get night looking good in Crysis Wars, but because of all the graphics and vis cheating, it looks HORRIBLE! Once you start trying to mess with Wars, you find out its really a load of crap and cheats. I really don't understand why it takes so much power to run, it must be really bad programming. And after seeing what is coming out in Crysis 2 which is aimed at consoles, that game looks even worse.