Thursday, February 07, 2008

GeForce 7 and Water Performance

A number of Windows and Linux GeForce 7 users have discovered that the command-line option --no_fbos improves their pixel-shader framerate a lot. Windows and Linux Radeon HD users have also discovred that --no_fbos cleans up artifacts in the water. Here's what's going on, at least as far as I can tell. (Drivers are black boxes to us app developers, so all we can do is theorize based on available data and often be proved wrong.)

Warning: this is going to get a bit technical.

FBO stands for framebuffer object, and simply put, it's an OpenGL extension that lets X-Plane build dynamic textures (textures that change per frame) by drawing directly into the texture using the GPU. Without FBOs we have to draw to the main screen and copy the results into the dynamic texture. (You don't see the drawing because we never tell the card "show the user".)

We like FBOs for a few reasons:
  • Most importantly, FBOs allow us to draw big images in one pass even if the screen is small. For example, if we have a 1024x1024 dynamic texture but the screen is 1024x768, then withou FBOs we have to draw the image in two parts and stitch it together. That sucks. With FBOs we can just draw straight to the texture and not worry about our "workspace" being smaller than our texture. This is going to become a lot more important for future rendering features where we need really-frickin' big textures.
  • It's faster to draw to the texture than to copy to it.
  • If you're running the sim with FSAA, then we end up using FSAA to prepare all of those dynamic textures. In virtually all cases, we don't need the quality improvements of FSAA, so there's no point in taking the performance penalty. When we render right into the texture, FSAA is bypassed and we prep our dynamic textures a lot faster.
Since copying to a texture from the screen predates these new-fangled FBOs by several years, most drivers can copy from the screen to the texture very quickly; however we have hit at least one case where FBOs are much faster than copy-from-screen. That's really a rare bug, and as you'll see below, we see more weird behavior with FBOs.

When do we use FBOs vs. copying? Well, it's pretty random:
  • Pixel shader reflective water and fog use FBOs.
  • Cloud shadows and the sun reflection when pixel shaders are off do not use FBOs.
  • The airplane panel uses FBOs if the panel is 1024x1024 or smaller; if the panel is larger than 1024x1024 we draw from the screen and patch things together. So the P180 and the C172 are using different driver techniques!!
When you run X-Plane with --no_fbos, you instruct X-Plane to ignore the FBO capability of the driver, and we use copy-from-screen everywhere.


There is one more element: mipmapping. A mip map is a series of smaller versions of a texture. Mipmapping allows the video card to rapidly find a texture that is about the size it needs. Here's an example: imagine you have a building with a 128x128 texture. If you park your plane by the building, the building might take up about 100x100 pixels on the screen; your 128x128 texture is a good fit.

Now taxi away from the building and watch it get smaller out your rear window. After a while the building is only taking up 8x8 pixels. What good is that 128x128 texture? Its' much too big for the job. With mipmapping, the card has a bunch of reduced-size versions of your texture laying around...64x64, 32x32,16x16, 8x8, 4x4, 2x2, 1x1. The video card realizes the building is tiny and grabs the 8x8 version.

Why not just use the 128x128 texture? Well, we'd only have two options with this texture:
  1. Examine all 16384 pixels of the texture to compute the 64 pixels on screen. That sucks...we're accessing VRAM sixty four times for each pixel. Accessing VRAM is slow, so this would kill performance.
  2. Simply pick 64 pixels out of the 16384 based on whatever is nearby. This is what the card will do if mipmapping is not used (because option 1 is too slow) and it looks bad. Thsoe 64 pixels may not be a good representation of the 16384 that make up your building side.
So mipmapping lets the video card grab a small number of pixels that still capture everything we need to know about the building at low res.

We don't mipmap our dynamic textures very often; the only ones that we do mipmap are the non-pixel-shader sun reflections and the pixel-shader sun reflections.


As far as we can tell, the current ATI Catalyst 8.1 drivers do not generate mipmaps correctly for an FBO-rendered texture. This is why without --no_fbos ATI users on Windows or Linux see very strange water artifacts. --no_fbos switches to the copy path, which works correctly.

At risk of further killing my track record of driver bugs in v9, we do think this is a bug. We have good contact with the ATI Linux driver guys so I have hopes of getting this fixed.


It appears that the process of creating mipmaps for FBO textures is not accelerated by the hardware on the GeForce 7 GPU series. This is why GeForce 7 users are seeing such poor pixel shader performance, while GeForce 8 users aren't having problems.

Now poor performance is not a bug; there's nothing in the OpenGL spec that says "your graphics card has to do this function wickedly fast". Nonetheless, what we're seeing now is unusably slow. So more investigation is needed -- given that the no-FBO case runs so quickly, I suspect the hardware itself can do what we want and it's just a question of the driver enabling the functionality. But I don't know for sure.


Karoliina said...

The water performance on GeForce 8800GTX by the way is excellent on Linux. Having all settings full does not slow it down at all (XP9 is slow for other reasons like not utilizing all CPU cores that are on the machine).

Benjamin Supnik said...

I think you are mixing up the PROBLEM (being CPU bound) with the SOLUTION (using more cores). I think you can have enough evidence to know that x-plane is CPU bound using simple performance analysis tools.

But I do not think you can know that multi-core (in particular using 4 cores or 8 cores or any specific number of cores) is the solution without knowing a lot more about x-plane's code than you could know, because the source code is not available.

I will comment about this more with your other post.

Karoliina said...

Actually the situation seems to be the following for CPU cores (running X-plane beta 21):
11500 pts/1 RLl+ 9:59 ./X-Plane-i686
11500 pts/1 SLl+ 0:00 ./X-Plane-i686
11500 pts/1 SLl+ 0:00 ./X-Plane-i686
11500 pts/1 SLl+ 0:00 ./X-Plane-i686
11500 pts/1 SLl+ 0:00 ./X-Plane-i686
11500 pts/1 SLl+ 0:00 ./X-Plane-i686
11500 pts/1 SLl+ 0:04 ./X-Plane-i686
11500 pts/1 SLl+ 0:00 ./X-Plane-i686

X-Plane uses multiple threads but rarely other than the main thread is utilized.

Gnome system monitor reports CPU utilization as follows:
CPU 1: 2%
CPU 2: 100%
CPU 3: 19%
CPU 4: 6.1%

(This changes over time, but it never reaches situation where two cores would have 100% utilization at one time).

Memory utilization is 1.3 GB for the X-plane process tree on our machine which runs X-plane on 30 inch monitor at resolution 2560x1600.