Sunday, June 01, 2008

Limits On Texture Paging

I seem to be in a philosophical mood these days with my blog posts...thought for the day: the human mind easily goes from the specific to the general. Our brains are generalizing machines, pattern matchers finding the rule in the noise.

My preference in creating new scenery-system features is to make them very limited, and my reasoning is: our brains don't go backward very well.  We do not go from the general to the specific.

Now you might think: when making a scenery-system addition, the best thing would be to have a general feature, more useful because it can be used everywhere.  But I say: the most important thing is to fully understand the feature - otherwise the feature comes out buggy. 

(Consider the piles and piles of bugs and weird behaviors that you get when combining OBJ animation with OBJ hard surfaces.)

Since the human brain doesn't go from general to specific well, it is hard to start with a rule ("let's allow feature X in all parts of the scenery system") and comprehensively derive all of the implications; it is human nature to be surprised later by some unintended side-effects.

It is always easier to extend a feature later to its natural full implications than to declare certain uses illegal later, after authors of planned or started trying to use the feature in that way.  If the generalization of the feature makes sense, extending it is often quite painless.

Texture Paging - Scope For Now

Texture paging is the ability for X-Plane to raise and lower the resolution of scenery textures dynamically as you fly.  This means more VRAM used for nearby things and less for far away things.  In practical terms, this reduces VRAM used by orthophotos by down-sampling the far-away textures, making larger orthophoto scenery packages possible.  As you fly, the sim reloads some textures at higher resolutions and some at lower.  The cost of the features is the load time while you fly, which burns up some extra CPU cores.

It is my hope that we will productize some very simple texture paging in the next major patch of X-Plane 9 (that would be 920, not 902).  But the usage will be pretty specific:
  • Texture paging will only be available for .ter and .pol textures (we can extend to other scenery types later if it makes sense).
  • Texture paging will require changing the .ter and .pol files (X-Plane will not automatically analyze your scenery to see what can be paged.)
  • Texture paging will not be available for ENV scenery.
  • If you share textures and texture page, the results will probably be really bad and cause chaos.  Be sure to use only one .ter or .pol file (and reference that text file only once in the your DSF definitions section) if you want sane paging.  We can extend paging to shared textures in the future, but for now orthophotos are the intended target.
I am also deferring work on dataref-driven textures; we'll get there eventually, and the infrastructure from the pager will make it easier.  But dataref-driven textures really need to be available in a lot more places - it's a bigger, more complex feature* and I can't keep adding scope to 920.

Make New Meshes!

While paging will be available for both overlays (using .pol files) and base meshes (using .ter files) I strongly, strongly recommend going the base-mesh .ter route.  RealScenery sent me their new "State of Washington" package to use as test material; I was pleasantly surprised at the high framerate.  Part of that comes from them using base meshes and not overlays. 

Overlays cause the sim to draw the scenery twice (first the old scenery, then your overlay), burning a lot of pixel shader and fill power.  Base meshes simply replace the old mesh which is at least twice as efficient.

(I'm just going to keep beating the dead horse of base meshes because I believe that the sooner everyone moves toward base meshes, the more bang for our hardware buck everyone gets.)

* In particular, remember that texture paging happens on threads.  But datarefs can come from plugins that are not threaded!  Insert anarchy here...


Peter Meininger said...

(Consider the piles and piles of bugs and weird behaviors that you get when combining OBJ animation with OBJ hard surfaces.)

You mean I could build a obj format animated lightning that'll give you a jolt you when you happen to be in the wrong place?


Squash said...

"But datarefs can come from plugins that are not threaded!"

- There shouldn't be any problem spawning a thread from a plugin though, right?

Benjamin Supnik said...

Squash - a plugin can spawn a thread, but all of the XPLM routines must be called on the main thread, from your callbacks...the XPLM is essentially an arbitration system between plugins and the sim - and it is single-threaded. So the sim needs to solve its own threading problems before it even talks to the XPLM (which in turn talks to plugins).

Dan31 said...

"While paging will be available for both overlays (using .pol files) and base meshes (using .ter files) I strongly, strongly recommend going the base-mesh .ter route."
OK but how could we had 3D objet without overlays ?

Benjamin Supnik said...

Dan31, anything that can be in an overlay (3d objects, roads, facades, forests) can be in a base mesh too. So you'd put your 3-d objects into your DSF with your base mesh.