Texture Compression Is Your Friend

Try this experiment: take three screen-shots (use ctlr-; to get a stable, clear view), with these settings:

1. Extreme res, with tex compression
2. Extreme res, without tex compression
3. Very high res, without tex compression

Now consider the effect on VRAM. Each time you go down a texture resolution level, tex sizes are cut in half, and you cut VRAM use by a factor of 4.

If you use texture compression, you cut VRAM use by a factor of 4 for textures with alpha and a factor of 6 for textures without alpha. (Some textures are unaffected because they are internally marked as "never compress".)

Now look at your visual results. I would argue two things:

• At extreme res, turning off texture compression doesn't make the image a whole lot nicer. (It does make the machine a whole lot slower!)
• Texture compression is a lot nicer than going down a res level.

I suspect that if your machine can't handle extreme res, you can get similar results running the whole experiment down one notch.

I think that almost everyone who has a limited machine is running with texture compression these days - we default with it on and it helps fps. I would only argue that any users who are running with a huge machine and compression off might consider turning it back on. It doesn't hurt much and it does make the machine run faster.

(Even if you have a ton of VRAM, X-Plane can be limited by how fast texture data transfers from VRAM to the GPU, within the card. Texture compression helps this.)

Spankings

Austin and I have this debate over and over when we find a case where third party content doesn't work right with X-Plane (but did in the past due to X-Plane's permissiveness):

• If we code X-Plane to work around the problem, users get to enjoy the scenery, and authors will never fix the problem.
• If we code X-Plane to abort, users get grumpy, and authors fix the problem real fast.

What to do? A few versions ago I implemented the scenery warning system (the spanking system) to try to get the best of both worlds. I just discovered another problem that illustrates how this works well.

Before WED some authors made apt.dat 850 layouts by hand using a text editor, and sometimes they forgot to include a closing code on their rings and chains. I can hardly blame them, writing an apt.dat file by hand is very difficult, and I am impressed (and grateful) that they did this.

The apt.dat 850 spec is clear:

• Pavement chunks (line code 110) must comply with the following:
  • Chunks MUST terminate in a segment point of type 113 or 114.
• Linear features (line code 120) must comply with the following:
  • Linear features must terminate in a point of type 113, 114, 115 or 116.
  • Linear features do not need to be "closed".
• Airport boundaries (line code 130) must comply with the following:
  • Boundaries MUST terminate in a segment point of type 113 or 114.
  • Boundaries must be a closed loop and cannot contain holes. As with pavement chunks, the airport boundary cannot overlap itself.

So, you must have one of the closing codes at the end of a boundary, taxiway, or line. But X-Plane 860 doesn't check this and simply assumes a "closed ring" if no code is provided. With X-Plane not telling authors, the authors never caught this mistake.

So now that I know about this (and now that X-Plane needs the end codes to be right, due to other bug fixes) it's time for the spanking system. In future versions of X-Plane:

• Custom scenery packages with no ending code on a polygon block will automatically be adjusted to be "ring"-type on load-in, for compatibility with X-Plane 860.
• A detailed message will be printed per polygon in the log.txt file.
• A single alert box will be shown to the user saying "there is a problem with the custom scenery pack XXX".

The idea is for the dialog box to walk a fine line between being unobtrusive enough for users and annoying enough for authors.

No more scenery in the Resources/Earth nav data folder.

Back in X-Plane 7 the default scenery location was the "Earth nav data" folder of the Resources folder. Custom scenery went into packages in the "Custom Scenery" folder, just like now.

In X-Plane 8 we introduced the concept of a "default scenery" folder in the resources folder. The default scenery folder acts exactly like the custom scenery folder, in that it holds scenery "packages", but it has two special properties:

1. Packages in this folder have lower priority than any custom scenery package.
2. It is in the resources folder and is therefore the turf of Laminar Research and our updater -- the Custom Scenery folder is for third parties.

Okay that's not really a special property, more of a convention. But the main idea here was to remove special cases from the scenery system...in X-Plane 8, all scenery comes in packages, no matter what it is and who ships it.

Scenery comes in packages, period. Repeat it like a mantra.

The original X-Plane 8.0 still installed legacy ENV files into the Resources/Earth nav data folder (but installed DSFs into a package). X-Plane 8.20 (when we shipped full DSF scenery) installs everything into packages, as will all future versions of X-Plane.

X-Plane 8 still looks in "Resources/Earth nav data" as a last resort. But this is going away! Future versions of X-Plane will only look for real scenery packages in the "Custom Scenery" and "Default Scenery" folders.

What does this mean to you? Almost certainly nothing...but...

• If you are an author and your scenery needs to be installed into the resources/Earth nav data folder, fer cryin' out loud, please put your scenery into a custom scenery package. It takes about 5 seconds and will make life easier for you and your users.
• If you are a user and you have been moving files into the the resources/Earth nav data folder, please don't do this! Learn how scenery packages work and put whatever you've been moving into a custom scenery pack. You'll find things much easier to deal with.

WED beta 2

I've posted WED beta 2.

The executive summary regarding split beziers: use beta 2 and re-export your layout and you should be more or less fine. There is no need to avoid them anymore.

Also a note on the source code: I will update the source code base of WED about every two weeks during beta, or whenever anyone asks me to post latest, whichever comes first. If no one asks, this will mean the source code is slightly behind the latest binary beta, but I think we can live with that...it takes a while for me to ball up and post the source code. (Again, if you want the current source code now, you are absolutely entitled to it - send me an email and I'll post ASAP.)

I will make sure the code is updated when WED goes final, of course I expect a new binary beta about every 3 or 4 days until we get it mostly shaken out.

A thread on X-Plane.org referenced whether program should be labeled beta. I am definitely guilty of leaving some of my old programs labeled as beta, when I should have done one more build and marked them as final. I will try to avoid doing this in the future! But WED is definitely beta material right now.

Thanks to everyone who reported bugs in beta 1 - the feedback was very high quality. The new download's README contains the list of bug fixes.

Split Beziers Part 3: A Workaround

In my previous post I described a bug where, when we have a split vertex in an apt.dat layout, X-Plane would draw taxiway lights all over the place. Fortunately, there is a workaround.

Remember that a split vertex is represented as two or three colocated vertices with one or two zero-length segments connecting them. The bug is that zero-length segments cause lights and lines to go haywire.

The fix is therefore simple:

1. Programs that write apt.dat files can simply set the attributes on the zero-length segment to none.
2. Future versions of X-Plane (with a real bug fix) can join taxiway lines even when there is a zero-length "break" in the attributes.

This picture shows the results - a split vertex with no attributes on the zero-length segment.


The next WED beta will automatically export split vertices this way - there will be no need to patch X-Plane or change your WED layouts.

Split Beziers Part 2: The X-Plane 860 Bug

In my previous blog post I defined a split vertex in an airport layout and described how they can be simulated in an apt.dat pile using multiple points.

Now that WED is in public beta and people can easily make split beziers, many have noticed the "split bezier" bug in X-Plane 860:


This is a vertex that is split...on the left you can see what it should look like - on the right you can see what it does look like. There are two problems going on:

1. The taxiway lights have gone crazy at the split vertex. (This is what everyone sees.)
2. The taxiway line is a bit jumbled at the split vertex too.

It should be noted that this bug will can also happen for any vertex (even unsplit) in rare occaisions due to interactions with the mesh. Same symptom, same buggy code, different cause.

The good news is:

• There is a workaround that will make the lights look correct and the lines acceptable in X-Plane 8.60.
• The workaround will be implemented inside WED - no need to change anything.
• When X-Plane is fixed, it will make the lines correct too.

In my next post I'll describe the fix and what the results look like.

Split Beziers Part 1: What's a Split Bezier

(This blog entry explains the background of split beziers - the next parts will explain the bugs that they cause and the workarounds.)

In apt.dat terminology, a split vertex is any vertex of a polygon where the control handles on either side of the vertex are not exact mirror images. (When there is only one control handle it is therefore by definition split!)

You need a split bezier any time you want to:

1. Have a sharp corner between two curves and control the tangents of the curves or
2. Have a sharp corner between a truly straight segment and control the curve of the next segment.

Split Beziers and apt.adt

Now here's the rub: the apt.dat format does not allow for split beziers - each curved point has only one control handle - the other is calculated by x-plane by mirroring...thus no vertex can ever be split.

(This is due to a total lack of brains on my part when working on the apt.dat format, which is quite embarrassing considering how long I spent thinking about it.)

The Hack

There is a way to simulate a split bezier: if you use zero-length segments (that is, multiple points on top of each other), you can create a shape that works as if it is split.

In its simplest, a split bezier can be created by using 3 vertices.

1. The first vertex uses the control handle of one side.
2. The second vertex is not curved.
3. The third vertex uses the control handle of the other side.

Why does this work? Well, a bezier curve between a curved point and a straight point has zero length if the two points are on top of each other. So what we've done is inserted two zero-length segments. The result of this mess is that the control handles on either "side" of this cluster of points can be different!

Is the second point really necessary! Yes! The reason is this: if we simply had the first and third point (two bezier points with different control handles), X-Plane would draw a loop from the first to the second. Remember: two colocated points with ONE control handle form a zero-length curve, but two colocated points with TWO control handles form a loop.

(To see this for yourself, just draw some examples in WED or photoshop. :-)

Line Continuity

There is one more wrinkle we have to add to the puzzle in order to understand how this works, and what the pitfalls are: line continuity.

A bezier path (taxiway edge, linear segments, etc.) is made up of one or more bezier curves. Each curve has zero or more attributes.

When X-Plane draws the actual taxi lines and lights, it looks for continuous adjacent bezier curves with the same attribute and makes sure the linkage between those attributes is correct.

(This linkage is computed separately for each type of property. So if you have taxiway lines on two segments and lights on one, the taxiway lines will still link!)

The picture above shows a correct vs. an incorrect link. When dealing with unsplit beziers and non-curved points, linkage is pretty much automatic, it just works.

But there is a pitfall to our above hack for split vertices: we have three points on top of each other. They must all have the same attributes in order for linkage to work. A "break" in the continuity of the line for one of the zero-length segments still counts as a break in linkage. The picture on the right was produced by creating a split bezier and removing the double-yellow-line attribute from the second of three vertices.

In my next post I'll explain the bugs that this causes in X-Plane 8.60.

Another Confession - Exclusion Zones

An overlay DSF can define exclusion zones - rectangles where scenery from lower priority DSFs is not shown. Exclusion zones are organized by entity type - that is, you must make a separate exclusion zone for objects vs. forests.

The problem with exclusion zones in X-Plane 860 is that the implementation of exclusion zones isn't quite right for lines and areas.

Essentially any element of a DSF is zero, one or two dimensional:
0d - Points (objects, very small facade objects).
1d - Lines (beaches, roads, large facades with no roofs, bezier lines and bezier object chains).
2d - Areas (large facades with roofs, bezier pavement, forests)

The problem is that X-Plane eliminates any entity if and only if one or more of its vertices intersects an exclusion zone.

This can be wrong in two ways:
- If an entity intersects the exclusion zone, it is deleted entirely, rather than having the exclusion zone subtracted.
- If an entity surrounds or goes through an exclusion zone without a vertex being in the zone, it is left alone.

So Why Doesn't X-Plane Look Like This?

Every now and then someone sends us this:

http://www.xtremesystems.org/forums/showthread.php?t=117500

The question is of course, why doesn't X-Plane look like that yet?

Now there's a lot of reasons why flight simulators don't look like first person shooters...you can definitely optimize any game content for a specific viewpoint -- X-Plane's lack of constraints on the camera position (you can put the camera quite literally ANYWHERE on the Earth at any time of day, atmospheric condition, and orientation) means that there are going to be views that don't look so hot. And the global scope of X-Plane means that we have to focus on quantity to a certain extent over quality. (If we made KSBD look totally awesome and didn't ship any state but California in global scenery, where would we be.)

I've been working lately on pixel shaders and lighting...when you look at those shots, the total integration of a number of great lighting effects is responsible for a lot of the look. But...I think there's a more fundamental issue that pixel shaders and carefully made content wouldn't address.

Simply put, X-Plane's LOD system isn't scalable enough.

In order to get images that look that good you need to put a huge amount of detail up close near the camera, where the user can see it, but not put that detail in the background. (Imagine if every tree in those scenes was done in the detail of the foreground - the polygon count would be unmanageable.)

But X-Plane's scenery SDK (the interface by which scenery content is specified to X-Plane, in other words, "the file formats") lacks really strong LOD capabilities.

• The terrain mesh is fixed - you can use LOD to eliminate overlay details, but you can't actually simplify terrain.
• The cost of LOD in objects is high enough to prohibit really gradual LOD. No morphing is provided.
• Textures are mipmapped but loading is not variable, so our VRAM budget doesn't benefit from LOD and locality of textures in X-Plane space.
• Generated geometry (roads, trees, etc.) don't have any LOD except "eliminate whole feature".
• There is no far view of 3-d clouds.
• Airport layouts are tessolated at only one complexity.

I could go on and on...the bottom line is, X-Plane's rendering model is very static.

Why did we do that? Well, it seemed like a good idea at the time. In particular, recalculating LOD is very expensive on the CPU and at the time we had only one core. Recalculating LOD would cost more in lost fps than it would benefit in offloading the GPU. So we went for static meshes that we could blast out to the card "real fast".

What you might not know about those screenshots is what kind of hardware it's running on:

http://www.gameklip.com/v/1606/

Yep...four cores, and overclocked by over a ghz. (I can only speculate that shortly after the clip ends the machine caught on fire. :-) In a multicore environment we can apply additional CPU resources to dynamically rebuild the environment to increase the "LOD range" (difference between the near and far view).

In fact, we already started to do that! In X-Plane 830 we modified the sim to build 3-d content (roads, forests, etc.) on a second core while flying, instantiating only the close ones. This saved RAM and improved the overall performance of the sim, and it increases our LOD range.

(Even if something isn't drawn, it has a cost just to exist - by saying that things that are really far away don't even exist we improve performance.)

As you can see from the above list, there's still a lot to do on the LOD front. But the scenery system is continually growing - new features for the various primitives and improvements to the engine will let us continue to improve sim efficiency.

That's one biiiiig polygon

Something I'm seeing now that WED is in beta: airport layouts with the entire taxiway structure made from one really complex polygon.

I'm not really sure if this is a good idea. First the potential problems:

• I suspect that the creation of the taxiway layouts can get slow when the number of sides in the airport layout is really huge and there are holes. I don't know this for a fact because we let the OpenGL libraries do the heavy lifting. Because this loading is done on a second CPU, it might not be noticeable to all users.
• The pavement can have only one texture direction per polygon, so multiple polygons may be necessary.
  
• Certainly in WED having a few large complex polygons slows down editing -- if all else is equal, the tools work better with smaller polygons.

Now...overlapping pavement is generally bad (that is, there is a performance cost), but more sides are also expensive. More thoughts:

• The more square feet of overlap, the worse. So a small overlapping intersection is not so bad, but avoid layering a huge polygon on top of another huge polygon, which just strains the video card.
• Fewer segments are better. Consider two crossing taxiways...8 segments with overlap, but 12 by making a plus.
• But wait - the above example is misleading...if you need to change the light types so the blue taxiway lights don't cross the intersection then you'll need to add 4 more segments, so now it's 12 and 12 - a wash. (In this case, having one big polygon is probably easier to manage.)

And for performance...

• Try going to your layout from far away and watch the last step of loading...if it starts to take a long time to "preload" things it means your layout might be a bit complex.
• Do not expect X-Plane to become faster at loading airport layouts...the limiting factors are proportional to complexity, so if you have a killer polygon now it'll be pretty expensive later too.

One other note, from a conversation with Tom...WED splits vertices into a fixed number of segments (per zoom level) so splitting a bezier makes it smoother. X-Plane does not! X-Plane splits beziers based on the overall curvature, so adding more nodes without changing the shape has no effect.

So please do not try to use the split command to make X-Plane "smoother". We'll provide a rendering setting for this some day. The current value was chosen because anything smaller looks awful and you have to make it a lot bigger (read: a lot slower fps) to get an even marginal visual improvement.

Scenery Tools and Compatibility

In my previous post I mentioned that the scenery tools are separate code with a separate release schedule from X-Plane. This implies that the updates to the tool won't be in sync with X-Plane. (The same thing is already true for the AC3D plugin and all of the wonderful third party tools that people have written.)

X-Plane's scenery compatibility strategy is "backward but not forward". In other words, if you open old scenery with a new X-Plane and it won't open or looks different, that's a bug (please tell us). You should never have to modify your OBJs or scenery if they are correctly made, and the definition of "correctly made" shouldn't get more strict over time.*

Why no forward compatibility? Well, it's possible to have syntactic forward compatibility (have X-Plane read a file it doesn't understand because at the time of release the spec didn't exist) but it's not possible to have X-Plane display that file with any kind of sanity.

You could say "just leave out stuff you don't understand". Well, imagine if we had done that with apt.dat -- imagine if X-Plane 840 tried to display an 850 apt.dat by simply skipping what it didn't understand. Since taxiways are fully replaced with a new code, 850 layouts would have no taxiways at all if shown in 840. This is certainly unacceptable.

The only way around this would be to require all new features to have "fallback" content, e.g. have the file contain two copies of all taxiways. Since the structure of 850 and 810 taxiways are so different, this would basically require the author to do twice as much work...similar problems happen for animation commands, and missing datarefs.

So the content has to be older than the sim. What about tools?

Well, first, I don't write the tools with forward compatibility. Since they are all free a user can easily get the newest tools to work (in a sane manner) with new content.

The tools do have to support old file formats though - we don't usually post old copies of the tools (the AC3D plugin is an exception, since old versions work with old host copies of AC3D). Instead the tools are designed to support restricted file formats. For example, the AC3D plugin will export to OBJ7 or OBJ8 - authors targetting X-Plane 7 can export to OBJ7.

WED only exports apt.dat 850 files - this is by design from day 1, and it will not ever be an apt.dat 810 editor. (Use TaxiDraw if you need to do this.) However, as new X-Plane features emerge (and I am sure they will), WED will have options to target future X-Plane versions, or restrict the export to only features supported by older versions, perhaps with a warning if the WED document contains features that will be lost in an old export.

* One problem we have with this is: X-Plane isn't a strict validator of scenery file formats, so it is possible to have mistakes in scenery that were always illegal by definition of the file format, but X-Plane doesn't notice at first. When I detect a case like this, I try to make the problem a "warning" rather than have X-Plane quit, so that authors can get information on buggy files without users suffering too much. X-Plane will print only one warning per package per scenery load to keep things streamlined.

WED Lives!!

WED went public beta yesterday. You can find it under the tools section of the X-Plane scenery web page.

I want to say thanks to the users who helped me with the alpha testing of WED. These users tested new builds quickly, wrote up great bug reports, and put a lot of effort into using WED at a time when it was really no fun to use.* Thanks guys!!

To reiterate a few points about WED:

• WED 1.0 edits only apt.dat files, not DSFs or DSF overlays. WED will be the platform for additional editing features, but for now it only edits apt.dat files, and it will be like this for a while.
• WED is still in beta, so use caution! Save backups, save often! Save earth.wed files and export to apt.dat for extra safety.
  
• WED is free and open-source. It is not distributed with X-Plane, and you don't need to buy X-Plane to use WED.

Enjoy!


* Remember, "alpha" software means "we can test all functions, but they could all be broken", while beta means "we think there are no bugs that would cause data loss or crashes". So...while WED was alpha, it tended to crash or trash files, so the alpha testers couldn't get any real work done.

Going AWOL

It's that time of year - I will be away from all forms of technology next week, enjoying cool mountain air and not having email. When I return I will post an udpate on the status of WED.

Don't Change lights.txt

Some very advanced users have asked: can we change lights.txt. The answer is: please don't.

Lights.txt is not a part of the "scenery SDK", that is, it's not a file whose format we will keep the same and allow you to modify. (The fact that it isn't accessible via the library system is an indication of our intention NOT to make it part of the scenery system.)

The problem is basically this: lights.txt translates named lights into the inputs to our pixel shaders for the hardware-accelerated lights. That pixel shader is really new and likely to change a few times. If the shader changes, we might need new parameters not in lights.txt, requiring a fundamental format change.

For example: those who have poked in the named lights file have noticed that the hardware lights can either have directional or flashing properties. This is because they run on two different shaders, each taking only four input values. This was done a while ago, when we were using low level assembly language shaders. In the future we might merge the two shaders and have 8 params per light. This would give us more flexibility (directional flashing lights), more bus usage (pushing 8 params per light instead of 4) and fewer state changes (we have to change shaders right now).

My point is: we can't predict what will happen, so we can't safely expose these parameters. The best thing to do is: email me and request named light types. We can easily have hundreds of named-light types (see how many there already are just for airports!).

Named lights make our lives easier because it tells us WHAT to draw but not HOW to draw it. So when we put in the next evolution of the lights code, we can remap the named lights to look the best they can for the new technology, instead of worrying about how to map the old params to the new ones.

(I appreciate the input from the users who emailed me about this -- it gives me more insight into what extensions to the scenery system would be useful.)

Liveries Part III: Image Contracts

The livery system I described in my previous blog entry (proposed independently by several users) would imply a contract between the aircarft creator and livery creator. The way I look at it:

• The livery creator agrees to utilize the texture conventions established by the aircraft modeler.
• The aircraft modeler recognizes that changes in the fundamental texturing scheme of the airplane become much more "expensive" because they potentially invalidate a whole set of liveries.

This is an exceptional situation in X-Plane...we have avoided contracts regarding the layout of images in all previous cases. (Examples: you can't use the library to replace an OBJ's texture. You specify terrain via a text file that references the image file, not the image file itself, so information on how to use the texture isn't part of the contract.)

But in the case of liveries, I think the exception is the inevitable outcome - a livery quite literally is a repainting of an aircraft, and the X-Plane community has been living with the limits of this kind of add-on for a while.

What this means is that it would be appropriate to override object textures in a livery system, but this feature is unlikely to appear anywhere else in the sim.

Liveries Part II: A Simple Approach

These ideas are similar to things that both Cormac and Peter have discussed with me...to propose a very simple naive livery implementation:

• An aircraft folder has a "liveries" folder inside it.
• Each folder within "liveries" is a "livery package", that is a folder that can be dropped in to extend how the plane looks.
• When X-Plane is using the livery , the liveries/ folder is searched before the aircraft's folder to find image files and objects.

X-Plane would have to provide some kind of UI to switch between liveries. This would allow authors to publish the base airplane and then publish liveries that simply drop inside, just like custom scenery drops inside the "custom scenery" folder.

Liveries Part I: Contracts

Any time two independently varying components need to interoperate, a contract is required.

Because X-Plane varies independently of third party add-ons (we release patches, you make new add-ons), any extensible part of X-Plane implies a contract. That contract basically says what legal things the add-on can do and how X-Plane will react.

We have to consider this "contract" with third party add-ons any time we modify the sim, and in some cases it means we can't change things. How this applies to liveries will become clear later.

Libraries and PNGs

There was some discussion on X-Plane.org about whether it should be possible to share and/or override PNGs via the library system independent of their objects. I say "no" for this reason:

The library system connects independent third party add-ons, that is, components that vary separately. Therefore there needs to be a "contract" any time the library is used, between the package requesting something from the library, and the package fulfilling it.

My concern with PNGs is that a library PNG would have to have a fixed layout. But realistically the kinds of PNGs that people want to share get reorganized on a regular basis. In particular, people want to reference the default sceney PNGs. But where we have a contract, we are limited in what we can change, and reorganizing the default scenery is critical to our ability to grow our content. I agree with Aussie's comments on x-plane.org that fixed-layout PNGs don't add a lot of value to the library system.

File System Vs. Text Files

There are a number of design trends in all of X-Plane's third party add-on systems. One is the use of the file system to specify modifications to the sim. For example:

• Put a "cockpit" folder in your aircraft, we'll try it first.
• Some files in the aircraft folder must have certain names.
• Sometimes putting _LIT after a texture causes it to be used as a lighting map.

On the other hand, I've gone a very different route with the scenery system:

• Libraries are looked up via a text file, not via filename.
• In all scenery cases, the lit texture is specified in a text file, not by filename.

What's the difference? Well, the file system way seems to be simpler for most users to understand. The text file mechanism is a lot more flexible. (Consider: we have versioning info in a text file, we can have each line in the text file clearly identify a feature, and so there is no risk of a file name being mistaken for a feature.)

Whether we use file names or text files to control extensions, doing so creates a contract, so we must ask: can we easily extend or safely modify the contract later? Can we express what we want using filenames (or text files)?

In the case of the scenery system, I think we need the full expressiveness of text files. Imagine if we wanted to provide a new kind of lighting that uses a different format of texture. Or a way to control at what time of day the lights get turned on? Or what if we want seasonal varying textures, and the rules for how seasons affect the light map aren't simple? In all of these cases, a file name convention rapidly becomes unworkable.

On the other hand, the airplane system has done reasonably well with filenames. Based on what I see, the problems with airplane distribution mostly come from a lack of features (no livery support, no plugins built into airplanes), not file names as a convention running out of flexibility.

Why Objects Kill [Framerate]

Austin and I were running some numbers on the KSBD demo area DSF as part of a discussion of how instancing will someday allow X-Plane to render more objects. (Instancing is the ability to render multiple simple objects with a single instruction to the GPU...the requirement of one command to the GPU per object means that total object count is bottlenecked by the CPU->GPU connection.)

Here's some numbers:

KSBD contains 868,220 mesh vertices - at 32 bytes per vertex, we have about 27 MB of geometry per DSF. In one view we picked, about 20% of those vertices were on screen. (But since there are six DSFs loaded, really only about 3% of a DSF mesh is seen at one time at lower altitudes.)

KSBD contains 153,816 objects. Near the airport, we average about 238 vertices per object. (This is a GOOD number - less than 100 vertices would imply that we aren't sending out enough vertices for each command to draw.) But this means that if we were to simply store all of the objects as one huge object, we would have 1.1 GB of object memory! This is why you can't just make a single huge object for the world. :-)

(X-Plane of course stores each OBJ file once, saving a lot of memory. But then we burn CPU power telling each OBJ to be drawn over and over in many places.)

It also explains why forests keep causing people to run out of memory. Consider that only about 3% of the mesh may be visible (when we're at lower altitudes, where you can see the trees). This means that we need about 30x as much memory to storage geometry as the card can draw. With cards so fast, we easily run out of memory before we max the card out.

Color Matching in X-Plane

Sergio sent me a scenery package with the question: why don't these two textures appear as the same color in X-Plane (they did in Photoshop). The answer is: gamma correction.

Some background (with much hand-waving): gamma refers to how bright the mid-tones of an image appear. (It's a lot more complex than that, but that's what Wikipedia is for!) Basically Macs and some other computers adjust the colors in an image to compensate for the deficiencies of CRTs, while PCs leave them alone. The result is that the same numeric color levels, when sent to Mac and PC hardware, result in brighter images on the Mac than the PC.

Since X-Plane is authored almost entirely on the Mac, the old complaint (around the time of X-Plane 6, with BMPs) was that X-Plane looked too dark on Windows. PNG addresses this issue: the gamma curve of the system an image was created on can be written into the PNG, allowing X-Plane to adjust the colors (making them brighter or darker) depending on what destination system we are running on.

Unfortunately, X-Plane isn't too brilliant about this, in two ways, one of which isn't our fault:

• X-Plane assumes the platform default gamma for both Mac and PC (that would be 1.8 for Mac and 2.2 for PC). If you read that background article, you know that this isn't real clever of us. But it actually is better than doing nothing at all.
• Gamma in PNG files is optional; if we get a PNG file with no gamma information, we make the rather arbitrary assumption that it came from a Mac. Since X-Plane used to be authored on Macs, this seemed like a reasonable thing and in the case of no gamma information, we're going to be wrong half of the time no matter what.

If you open up the default scenery PNG files in Preview, you can see the gamma inforation - 0.4545. This is 1/2.2, meaning the files are encoded to PC gamma standards. It turns out that one of the textures Sergio sent me had no gamma information, so X-Plane assumed Mac gamma (0.5555, or 1/1.8). Thus the brightness of these textures were being adjusted by different amounts.

My recommendation to authors is simple: make sure that all of your PNG files always have gamma values written into them. Otherwise there is a risk that the default gamma guess that X-Plane makes will not be the one you authored under, causing color shifts.

Don't use the panel texture in a non-cockpit object.

Panels and airplanes aren't really my thing. But we're using the OBJ engine more and more in aircraft, blurring the lines between scenery code and aircraft code.

In X-Plane you can actually use the panel texture in any object. Pleaes don't do this! There are two cases:

Airplane Objects

With X-Plane 860 you can attach many objects to an aircraft. But...you should only use the panel texture in the cockpit object.

The cockpit object is special! It is the only object that gets mouse-click tested. In the future we will decide how much of the 2-d panel to render to a texure (for the 3-d cockpit) based on the cockpit object. So...don't use the panel texture in your other aircraft objects. Put your panel-textured triangles in the panel object.

(You should do this anyway; switching to the panel texture creates a new batch*, so using it in a lot of separate objects is bad for framerate.)

Scenery Objects

Technically you can also use the panel texture in scenery objects. This is really just a big hack -- all of the issues with optimzation apply to this case, plus: how can you even know the layout of the panel texture in scenery? Don't do this!

* "Batch" is the technical term in the game development and authoring world for a set of triangles that can be drawn by the graphics card via single CPU command. While graphics cards can process tons and tons of triangles, the number of batches they can process is limited by CPU and bus speed, which advance much more slowly. Generally the number of batches is the limiting factor in X-Plane's framerate.

It's easy to see how many batches your OBJ8 contains: open the file and count the number of "TRIS" and "LINES" commands at the end. Ignoring lights, the sum of the TRIS and LINES commands is the number of batches! (HINT: fewer is better, and only one batch is great!)

Crash Bang Goes the Airplane

Ari asked a good question regarding sloped runways and the new apt.dat 850 format:

Am I understanding it right that airport taxiways, ramps and runways are from now on going to be merged into one big mesh, instead of bunch of rectangle pieces overlapping each other? If yes, will this finally allow us turning on sloped runways option in X-Plane without any of the current side effects?

This brings up some interesting questions. First the most basic answer:

• apt.dat 850 prrovides curves, irregular taxiway shapes, which will allow you to create complex taxiway shapes with only one piece of pavement, rather than many overlapping ones.
• X-Plane still honors the order of apt.dat 850 for drawing, so you can also overlap and get visually consistent results.
• We recommend using a smaller number of curved taxiways rather than many overlapping rectangular ones because X-Plane can handles this case more efficiently. It is not necessary to build the entire airport out of one taxiway though.

Now the second part of the second of this question is a little more complex, because the cause of bumps in the scenery changed.

Bumpy Runways in the Good Old Days

Back in X-Plane 806 there was a fundamental problem with the way we did sloped runways that made them virtually unusable: while the corners of each rectangular piece of pavement would sit directly on the terrain (no matter what the terrain's slope), the area of the taixway was formed by a flat plane. This means that the middle of the taxiway might be above or below the terrain.

Now the real problem comes when we have two taxiways that overlap. Because they are only aligned to the terrain at their corners and not centers, there may be differences in their height when one taxiway's corner hits another taxiway's center (which happens a lot). As the airplane travels from one taxiway to another, the elevation of the ground changes instantly, inducing a major jolt to the suspension. At high speeds these damage the airplane's suspension.

Bumpy Runways Now

In X-Plane 850, we break all runways and taxiways (new and old) into multiple pieces each tiem the terrain underneath them has an edge. The resulting taxiways are then aligned to the mesh at their corners. But since no taxiway center goes over a mesh corner, the taxiway "hugs" the mesh perfectly. And since all taxiways hug the mesh in the same way, there is never a height gap between taxiways.


It's the Mesh, Stupid

So why do we still have bumps in X-Plane 850 if we so carefully make sure the taxiways exactly reflect the mesh height? Well, you're effectively driving on the terrain, so any bumps are ones from the terrain. Simply put, even with the new system the usability of sloped runways is only as good as the underlying terrain.

Now our meshes come from SRTM data, which is radar data - it naturally has a certain level of noise and "speckle" which makes it pretty unusable for airports...airplanes are very sensitive to even small bumps during takeoff.

We attempt to "condition" the elevation data for airport use, smoothing out hills and bumps. Unfortunately our algorithm doesn't always work right. The X-Plane 8 US scenery was way too bumpy to be usable. The 7-DVD set is better, but still makes bumpy airports in a few cases:

• If there was no airport in the apt.dat file at the time of scenery creation, no conditioning was applied, and the underlying terrain is probably inappropriate for draping.
• The scenery creator has a bug that causes airport flattening to fail when it's very close to water. For example, the big bump in the runway at KLGA is due to a water-airport interaction.
• I think that flattening across DSF tiles can have problems too.

If you've been thinking -- wow, the diagram for 850 has a lot more triangles (10 vs 4) than the one for 806, you are right. Fortunately, the number of triangles, all part of one taxiway, in an airport layout, doesn't really affect frame-rate, since this is handled by the GPU.

But this is also a case where a few curved polygons can be much more efficient than several overlapping ones - when we cut up the taxiway based on the mesh, if there is ovelapping pavement, each overlapping taxiway must be cut, multiplying the effects of the mesh on triangle count.

It also turns out that in the real case this is somewhat moot: because X-Plane smooths the airports and then induces triangle borders around the edge of the airport. Since the interior area is so flat, it doesn't require a lot of triangles, and therefore the trianglse inside an airport tend to be big, so the number of times we have to cut an actual layout is quite small.

Pictures to Prove It

Mighty Mighty Bostones? Anyone? Nevermind. Anyway...

...some pictures of WED. (These were taken on a Macintosh, but it looks almost the same on Windows except for the title bars of the windows.)


This is the startup window. WED can edit multiple scenery packages (in multiple windows) but does not let you edit a scenery package until you create and name it. So we show this window when you startup to let you choose between a new or existing project.



The left side is the map view, which provides draggable editing of any part of an apt.dat file. This is part of Aussie's KSBD layout, which is part of the X-Plane demo. The right side provides a hiearchy-structure view (top) and more detailed editing or properties (bottom).

With the vertex tool, we can drag control handles for any selected entity.



The map view is a "structural" view, not a photorealistic one. The markings on the lines and pavement color change to reflect the settings you pick, but WED does not attempt to reproduce the final result in X-Plane.

Wiht the marquee tool, we get a rectangular edit box around the selection and can thus resize or move whole sets of entities at once.



This is KBOS from 860 imported into WED. WED will import 810 or 850 layouts, but only exports 850 layouts. (It can thus be used as a simple converter.) That background image is automatically downloaded from terraserver and updated as you zoom and scroll the map. Unfortunately terraserver only covers the US; where there is no coverage you can import any BMP, PNG, JPEG or TIFF file and set it as a background image.

Our New Management Consultant

A long time ago I posted a picture of my immediate supervisor. I am pleased to announce that the home office has brought on Cecelia to help consult on future X-Plane development. Here she is:



A meeting of the management team:

Look Mom, No Tiles

Since before I've been involved in X-Plane, scenery has been broken into tiles. You know the naming scheme...+42-072.XXX is some kind of file for Boston. Perhaps it's a .env or a .dsf, but what is unchanged is that the world is broken up into bite-sized pieces.

Tiling is very necessary for X-Plane...it allows the sim to rapidly load only the information we are interested in. The world is too big to go fishing for the relatively small amount of data that is loaded at one time.

The source data that is used to build global scenery is also tiled - if you look at the SRTM files, they often have names like N42W072. It's no surprise that global data requires tiling to make it manageable.

When we produce global scenery, we work on a per-tile basis. We load the raw tiled data* into source XES files (one per tile), then process and convert that XES file into a DSF. By working one tile at a time, we limit how much RAM we need.

But if you used WorldMaker on an airport or city that spanned a tile boundary, you know how annoying tiling can be. With WorldMaker you couldn't see both halves of an airport at once.

WED will not use tiles. For custom scenery packages, the total data is not so large that we have to tile. When you work on a scenery package in WED, you work on the entire package at once as a single seamless workspace. When you export your work, WED will split the data into tiles as needed. This will mean:

• You do not have to decide in advance where you are going to work. All scenery packages cover the entire planet, and DSFs are created only as needed.
• You will be able to work on more than one "tile" of area at a time, because there are no tiles.
• You will be able to work on airports that span tile boundaries seamlessly.

Hierarchy and WED

I just finished fixing some bugs in WED regarding the hierarchy view. (To get a sense of what a hierarchy view is like, try AC3D...)

In WED, the contents of your airport layout are viewed in a hierarchy view. This is where you reorder your runways and rename entities. You can also group entities (as many times as you want) to help organize your layouts. (The groups will not be visible in the final apt.dat.)

An airport is, in a way, sort of a super-group...it's a group of "stuff" (runways, taxiways, etc.) plus it has its own information.

But this leads to some tricky situations:

• You can't have an airport inside another airport.
• You can't have a runway outside an airport.

The result is that there are a number of grouping and drag & drop combinations in the hierarchy that are simply illegal. Cris and I debated this a bit and what we have now is: commands that would produce illegal results simply cannot be executed. (That is, the group command is grayed out if the new group would violate these rules, and the drag & drop won't show a legal drop if the new order would violate these rules.)

Once we get into beta we'll see how well this works. The alternatives include:

• Putting up a specific error message when an illegal configuration is made (annoying).
• Allowing illegal configurations and putting up an error message on export (by this time the whole file layout could be pretty badly messed up).
• Attempting to silently export the illegal configuration (unpredictable - there's really no good way to export an airport inside an airport).

Also, I am sure there are some illegal configurations I have missed...we'll have to catch those in beta too.

New AC3D Plugin

The new beta of the AC3D export plugin is posted here. There are a bunch of bug fixes, but the two big ones are:

Texture paths are exported without relative directories. This was a new "feature" in the first beta, but virtually everyone said that exporting relative paths was both confusing and annoying. The plugin is now back the way it used to be. (If you need a relative path, there is a "prefix" path that you can specify in the preferences that will be prepended before all textures.)

The popup menu for datarefs has been replaced by a combo-box. This has three advantages:

1. The popup can be built a lot faster.
2. You can type custom datarefs into the combo box.
3. It doesn't crash.

Here's a tip: if you type a part of a dataref into the combo box and then open it, only the matching datarefs are shown. Try typing "flightmodel" into the combo box and then opening it.

"Directories" of datarefs are shown first. So from flightmodel you can pick sim/flightmodel/position and then re-click. Now all of the position datarefs are shown. In this way you can navigate through all datarefs.

You can also use wild-cards in the combo-box. If you search for sim/flightmodel*x you will find all of the datarefs beginning with sim/flightmodel and ending with x. Note that when you use * your pattern must match the entire dataref.

(To search for all datarefs containing flightmodel and local with some text between them, you cannot just use flightmodel*local. You need to use *flightmodel*local*.)

You Can't Unbake a Cake

First, I just want to be clear: I am not announcing any future features for the scenery tools. I am not saying when they will be released, and I am not saying what they will do, because honestly I do not know. There have been too many cases when users have emailed me and said either "I thought you guys were going to do X" or "you guys promised you were going to do X", so now I am officially paranoid.

So this blog post is not about what the future scenery tools will do - it is simply a discussion of the difference between editing source data and compiled DSFs.

When we make the global scenery, we start with a bunch of source data that roughly consists of: road maps, coastlines, elevation, landuse, climate data, airport locations, etc. When we build a DSF out of it, we "bake" these items together into a single file. During this baking process, our tools apply some "integration effects". Here are a few of the more obvious examples:

• Terrain under airports is forced to be an airport grass texture, appropriate for local climate data.
• Roads are removed from airport areas.
• Intersections are computed for highways - that is, a highway and city street form an overpass, but two city streets become a real intersection.
• Generated Buildings are put around the roads, not under them.
• Buildings are oriented to "face" the slope of the ground they are under, based on their shape.

That's not a complete list, but it gives you an idea of some of what goes into making a DSF. All of this information is precomputed and represented in the final DSF. But consider this last point: the DSF contains the actual orientation (north/south/east/west) of each building. It does not contain the slope underneath the building and it contains no information about which buildings need reorientation. In other words, the results of the process, not the inputs to the process, are present.

So consider what would happen if you could simply edit the data in the DSF:

• If you moved an airport, the airport grass would stay in its own location.
• If you moved an airport over a road, the road would still be there.
  
• If you changed a city street to a highway, it would not form an overpass.
  
• If you moved a road on top of a generated building, the building would remain in place.
  
• If you changed the ground elevation, buildings would not change their orientation to face the new slope.

If you changed the source data and re-ran the DSF building process, these effects would occur. But remember, we need elevation, land use, etc. to build a DSF from scratch, and the DSF itself doesn't contain its source data.

So we have two possible strategies for editing DSFs:

1. We could build a DSF "retouching" tool that let us make very small changes to the existing DSF without having any source data. None of these effects would "work" so authors would have to make very small changes and then hand-fix any problems that appeared.
2. We could build a DSF "rebuilding" tool that let authors make new DSFs from source data. All of the effects would look good, but authors would have to get some of the source data. (We could post our source data, or provide links to places where it can be downloaded.)

Note that we can't have our cake and eat it...we cannot get the "integration effects" listed above unless we go back to the source data. If I can stretch my cake metaphor to the breaking point, once we make our cake, we can't easily remove the flour and add another egg - we need to start over with the raw ingredients.

Which strategy will the scenery tools use? I don't know yet. I am focusing on airport and overlay editing, which sidestep this issue a bit (we can easily edit an apt.dat 850 or overlay from the final product). We may do a bit of both strategies - it depends on what users want and what we can code efficiently.

Why don't the finished DSFs contain everything we need to edit them? The answer is size. The finished global scenery was about 56 GB of DSF. When I last checked, the raw data that forms the DSFs was at least 100 GB or more. So for each DVD we shipped of scenery, we'd have to ship two more DVDs of source data, for a 21 DVD set, most of which wouldn't be useful to most users.

It's the little things....

That take the longest to program. This may surprise non-programmers, but won't surprise anyone who has done UI development before.

Example: in WED, there is a spreadsheet-like view (the "property" view) that shows text-based information, usually with a list of runways or taxiways in one direction and different aspects of them (name, displaced threshhold length, surface, etc.) on the other.

In WED, if you press the tab key while editing text, WED will move to the next editable field, first across, then down. Furthermore, it will scroll the view as needed so you don't need to adjust the scrollbars while working. The result is that you can, for example, use the mouse to select all of your taxiways, and then rename every single one of them from the keyboard with the tab key without ever going back to the mouse.

It's a small UI feature that can make a big difference in how long it takes to work on a project, and takes surprisingly long to code.

European Buildings Everywhere

A user reported a bug: the 3-d buildings in all parts of the world outside the United States* are European-style, even in Australia, Canada, China, etc.

It takes two things to see 3-d objects in X-Plane:

1. Location data in the DSF you are flying over.
2. 3-d models (stored in OBJ files) that match the definitions called for by the location data.

Now we have location data for our "generic" objects (objects that are placed pseudo-randomly based on local topography and roads, to synthesize cities) for all urban areas that are covered by the 7-DVD global scenery set. But we only have two sets of artwork: a set of US-style buildings for use in the US, and a set of European-style buildings for the rest of the world.

So we had an unfortunate dilemma. Either we could have European style buildings everywhere, which looks funny, but at least it's 3-d. Or we could use the European-style buildings only in Europe and have no 3-d at all everywhere else. Both options make some users unhappy and neither is very good. We chose to have the European buildings everywhere - my guess is that it generated slightly less complaints.

Well, the silver lining is: it's actually really easy to remove the European-style builldings from the rest of the world (Canada, Australia, China, Russia, Africa, etc.). All you have to do is make a custom scenery pack that applies to the regions where you don't want the buildings and maps them to a blank object. I have built such a custom scenery pack and posted it here. If you install this scenery pack, then Canada will be devoid of 3-d, but at least there won't be mismatching architecture.

This begs the question: why not use the US buildings to populate some of those regions? The answer is that the US buildings are built to a different spec than the European ones, because the road grid data for the US is more detailed than the rest of the world. So amongst our current artwork, only the European buildings will work globally.

* When I refer to the United States, I really mean the continental United States. Sorry to everyone in the beautiful states of Hawaii and Alaska. This is just an artifact of how the data was imported - I could not get the Census data for Hawaii, Alaska, or the territories to import, so I reverted to the generic "global" data.

Turn Off "Draw Hi-Detailed World"

It looks so innocent, that one check box..."draw hi detailed world". What harm could it do?

Well, if you have a GeForce FX graphics card, quite a bit! If you have Vista it may not be a great idea either.

The "draw hi-detail world" setting turns on multiple rendering settings that look nice but hurts fps. There are two I can think of right now:

• With the setting on, we draw 3-d structures for airport lights. This can slow down slower machines, but usually isn't the big problem.
• With this setting on, we use pixel shaders to draw terrain - without it we use the traditional "fixed function" OpenGL pipeline.

It's this second behavior that causes all the misery. X-Plane won't use pixel shaders if your video card doesn't have them. But...what if your card has pixel shaders and they're just not very good?

I should say: I have no first-hand knowledge of how the GeForce FX series works, and what I am repeating is simply conjecture posted on the web, albeit conjecture that explains what we keep hearing from users. The GeForce FX (nVidia's first series of programmable pixel-shader based cards) is a hybrid card - half the card's transistors are dedicated to fixed-function drawing, and only half for shaders. Thus if we go into shader mode, we basically "lose" half the chip, and our performance tanks. (ATI built 100% programmable cards starting with their first entry, the 9700, and nVidia went this way with the 6000 series.)

So if you have an FX card, it tells X-Plane "I can do shaders". With "hi-detailed world" on, we take a huge performance hit. Simple solution for FX users: turn "hi-detailed world" off! Get your fps back!

I've also heard a bunch of reports that the new drivers for Vista have bugs...they seem to come out more when we use pixel shaders . Again - turn "hi detailed world" off and see if it helps!

Bottom line: "draw hi detailed world" is proving to be an aggressive setting - I recommend backing it down as the first step in trouble-shooting performance problems.

Drawing on the Ground - Polygons or OBJs

With X-Plane 860 there are two ways to draw stuff on the ground:

• Use an OBJ and ATTR_poly_os. This is the way most people have built custom taxiways, markings, etc.
• Use a DSF overlay with a bezier polygon (which is controlled by a .pol or .lin file).

Which should you do? Hrm...

• For any geometry that goes over a very large area, the DSF overlay with a bezier polygon is basically a requirement, because the polygon will "drape" over the ground. The OBJ might just stick out in the air.
• If your element repeats a lot, the OBJ may save memory. (OBJs are stored in memory once and used many times - bezier polygons use RAM each time they're used.)
  
• If the overlay is very simple, the DSF overlay may perform better.
• It may be too expensive to build complex lines with DSF overlay lines. They're meant for taxi-lines, not full vector graphics!

So here are a few decisions I'd make:

• Taxi lines - certainly DSF overlays if we're not using apt.dat 850.
• Parking spot at an airport - use one large polygon with a texture, not a bunch of bezier lines.
• Use a DSF overlay for that parking spot, even if we use it 60 or 100 times. The simplicity of the overlay (vs. the compexity of starting and stopping an OBJ from drawing) outweighs the cost of using it many times.
• A very structurally complex overlay on the ground with thousands of triangles (that for some reason cannot be created with a texture) that is used a lot -- well, this is an "artificial" case I've made up, but in this case use an OBJ.
  

The Sordid History of ATTR_poly_os

I've blogged in the past about ATTR_poly_os...it's a tricky topic. ATTR_poly_os is
a feature of the OBJ file format designed to let authors fix z-buffer thrash problems. Unfortunately, the cause of z-buffer thrash is pretty complex. To make things worse, it turns out I never finished my intended documentation on the subject. (I'm an idiot!)

The fundamental problem I think is that what we have now (ATTR_poly_os and ATTR_layer_group) provide a mechanism to correctly fix z-buffer thrash, but they don't in any way enforce good behavior over bad behavior. The two attributes are very flexible, and if used together, can do all sorts of bad things. The problem is that ATTR_poly_os was thought up years before the layer-group mechanism, and thus they don't really reinforce each other.

So...here are a few simple rules to help with z-buffer thrash in X-Plane 860:

1. Never use the names of objects or their order in the DSF to accomplish anything. X-Plane ignores both names and orders when processing your scenery.
2. Do not move your polygons above the terrain to fix z-thrash. This won't work.
3. When possible, divide your objects into ones that are 100% on-the-ground (and thus may z-thrash) and ones that are 100% 3-d above the ground (and will not thrash). I realize that more objects means slower fps...so this applies best when you have many objects and can pick how you divide them up.
4. Always use ATTR_poly_os for any polygons that lie along the ground. Use the smallest number you can to fix the thrash.
5. If you have an object with ATTR_poly_os geometry and non-poly_os geometry, make sure the ATTR_poly_os geometry is first!

Those five rules should keep you out of trouble.

What about ATTR_layer_group? Well, secretly X-Plane 860 will change the layer group of an object that is ATTR_poly_os for you. So as long as your object contains only offset geometry (this is what I recommend in rule 3) it wll always be drawn before the rest of the objects, preventing artifacts.

You'll need ATTR_layer_group if you want to put objects underneath runways, or underneath taxiways, for example.

I am working on more comprehensive documentation on the topic, and appreciate any feedback on stuff that I've written that's unclear...the rules are complicated!

DX10: Why I'm Not an Early Adopter

Before I begin, X-Plane uses OpenGL as its interface to 3-d hardware, not Direct3D. So when I talk about "X-Plane doesn't utilize DX10", isn't that meaningless? I mean, X-Plane has never supported any version of Direct3D.

But I like to use the term "DX9" and "DX10" anyway for this reason:

• For all practical purposes, within the "games space", most advances to the Direct3D and OpenGL APIs that I care about are created for the purpose of exposing new hardware capabilities to applications. That is, the point of DX10 (including the new Direct3D) is to allow games to use the newest video cards more efficiently.

• OpenGL is revised by adding "extensions", that is, independent features that can be mixed and matched. DirectX tends to have whole-API revisions. So I prefer DirectX because it puts a nice "number" on an entire set of technology. Since the graphics cards are revised in generations as new GPUs are designed, these generations match up reasonably well with the hardware.

So in this context, when I say "DX10" I really mean the very newest set of super-programmable cards, of which the GeForce 8800 is the first, and by use them I mean take advantage of some of these really great new features:

• Instancing (the ability to draw a lot of objects with only one command to the card, which could relieve the CPU cost of huge numbers of objects).
• Geometry shaders (the ability to do per-triangle and not just per-vertex calculations on the graphics card) which could move some of the logic for terrain generation to the graphics card. (We precompute this and save it in the DSF in X-Plane, so we use DVD space and RAM, while I believe MSFS does this kind of thing on the CPU.)
• Better management of state changes (good for unloading the CPU).
• A bunch of really interesting ways to work with data strictly on the card (don't know what it's good for yet, but it unlocks a lot of cool possibilities).

So why doesn't X-Plane utilize all of these new features? Or rather, when will we?

Well, my goal in working on X-Plane's rendering engine is to know these things are coming but not be an early adopter. The way I look at the economics of software development is: the amount of labor we can put into a release is somewhat constrained. If we put in more months between releases, we have fewer releases. If we have more programmers, we have to pay them more, and we have to charge more money. There's a lot of things you can say (or people have told us) about our business model. But I tend to view these as the invariant conditions we have to work with.

So what I worry about is efficiency: if we are limited to exactly X man-months of work per release, how can we make the best of them? Is being an early adopter the most efficient use of limited programmer resources when developing X-Plane? (We have to consider opportunity cost: what features won't be implemented because we spent time on early adoption of new graphics technologies.)

I think there are a few things going against early adoption, particularly for a small company like us where labor is at a premium. (Our list of things we can be doing is very long, so any new feature takes away from a lot of other good ideas.)

• New graphics hardware isn't widely distributed among our user base. If we adopt DirectX-10 style features, this work helps a very small numer of our users. Eventually everyone will have hardware like this, but we can cover a case by adapting the new technology later.
• When new features come out, there is often vendor disagreement on how to code for them. It takes time to come up with cross-vendor standards. Consider that ATI hasn't come out with their DX10 hardware, and the OpenGL extensions to use these are all nVidia proposals. My guess is that ATI will have their own extensions, and the real ones that get used will be a mix of each. If we adopt now, we'll be "betting on the wrong horse" a few times -- code that will have to be rewritten, for a total loss of efficiency.
• New drivers can be buggy. It takes a while for support for new features to be both universal and reliable. The earlier we jump in, the buggier the environment we develop in, and thus the more difficult it is for us to develop.

Of course, there's definitely a cost. The 8800 is capable of doing some amazing things, and X-Plane does not yet fully take advantage of it. But I do believe that in the long term we end up delivering more value to X-Plane users by taking a slower wait-and-see approach to new hardware.

(To GeForce 8800 users I can only say that the code we write now while waiting for the right environment may do some cool things too, so we're not just taking a vacation! And the GeForce 8800 also delivers an overall speed boost to the entire system.)

EDIT: the same logic applies to operating systems like Vista and OS X 10.5 to some extent, but I hesitate to bring this up because: a number of users are having problems with X-Plane and Vista, and it is not because we have delayed support for Vista. The real problem is that the graphics card drivers for Vista are still new and have some problems. I believe that the various Vista problems we're seeing will be addressed by code changes by ATI and nVidia.

Will it Ever Be Done (WED)

The internal joke about WED (the new scenery editor) is that it stands for Will it Ever be Done. So I should say that, given my total inability to predict when it will be done, and how many delays and setbacks there have been, I don't expect you to believe anything I say until I actually ship something you can run (or at least post some screenshots*). I realize that I have destroyed my credibility about ship-dates by having no ability to predict ship dates for WED.

With that in mind, I am pleased to report that today I was able to run WED with a few features working in concert:

• Multiple selection (map and hierachy view).
• Multiple undo everywhere.
• Marquee multi-selection tool on the map view.
• Tree-based hierarchy view with editable property fields.

WED still can't do anything remotely useful, but these four features represent a huge amount of infrastructure investment. Basically my goal is to provide some of these user-interface features (multiple undo, multiple select, multiple tools, etc.) for every single type of scenery component that you can edit.

Since apt.dat contains a lot of different components (just look at all the record types in the file format) and WED will also have to edit DSFs with a wide variety of data, it made sense to me to write a generic mechanism for these features that could be used over and over again without writing new code.

So my hope is that I'm just getting over the "hump" of writing infrastructure, and soon will be able to add two dozen more types of editable things to WED (other than my simple test objects) very quickly. We'll see if it pays off.

The plan is still for WED releases to be separate from X-Plane and to be open source (MIT/X11 license). I've been meaning to post in Chris's programming blog about some of the design ideas WED employs - it makes a good laboratory. Hopefully those posts can also provide some guidance if anyone decides to modify/work on/steal the WED source code. (It is my expectation that most programmers will not want to get into the guts of how WED works, as it is a fairly complex application. I think I can make simpler interfaces to things like import/export to make extending the program simpler.)

Anyway, WED is not the only thing I am working on right now, and you don't have to believe me, but the WED codebase is growing, and some of it even works!

* I am not posting screenshots because the program is running right now with "scab art" - that is, ugly green, red and purple boxes that will be replaced with nice PNGs once the layout settles down. I do not want to make our artists draw the UI components more than once, and I don't want to answer 1000 emails about "Why is it so ugly" by posting the scabs, which are perfectly adequate for my own coding purposes and not intended to ship.

I'm not a fan of SLI/CrossFire

When it comes to video cards, I've always been in the "don't spend more than $200" school of thought. My logic is: video card technology moves so fast that paying a lot of money for the "first six months" of any new technology level is very expensive. Bless all of you who are early adopters - you're helping keep nVidia and ATI humming, but it's an expensive hobby to main an up-to-date machine.

This is one of my favorite tables (there is a similar one on Wikipedia for ATI). It shows performance of graphics cards and when they came out. Compare the GeForce 7950 GT2 and the GeForce 8800 GTS. If you want 24,000 MT/S of fill rate, you could buy a top-of-the-line two-cards-in-one-via-SLI 7950, but if you waited six months, a SINGLE intermediate-speed 8800 would give you the same thing while supporting DX10 shaders (e.g. geometry shaders, instancing, and all that awesome stuff). The 7950 GT2 apparently retailed at $600+, which was a real discount compared to actually chaining two separate 7950's together (that'd get you up around $850). Look on newegg.com and you'll see that GeForce 8800 prices aren't that expensive (compared to an SLI combination). And the 7950 GT2's come down a lot from what it used to cost.

For another datapoint, compare the Geforce 7600's to the GeForce 6800's. The 6800 was the monster card when it came out, putting nVidia back in the number one spot. But the next-generation's intermediate range cards can do what was top-end before. (The 7600 can be had for a little over $100. Compare that to several hundred for the 6800 ultra about one year earlier.)

Simply put, you pay a huge premium to get a given performance level when it's new and top-end. Wait one generation of cards (by buying last-year's top end cards or this-year's middle-range cards) and you save a lot.

It's in this context that I don't believe that SLI makes a lot of sense. In an environment where (IMO, and my opinion only) the top-end video cards are already expensive for what they do, SLI simply makes the situation worse, by allowing you to spend double what the already-high-end cards cost to get performance that will be available in one card in the next generation.

To do the math, does it make sense to spend double the price on your video card to extend its useful life by six months? Only if you intend to change cards every six months.

(nVidia makes an argument that SLI allows developers to preview the next-gen hardware, and this is true. My strategy is different: simply run X-Plane slowly and assume that the next-generation hardware will go faster.)

I don't feel good about criticizing nVidia and ATI because overall I feel that their products provide an extraordinary value at a very good price, and the growth of performance in video cards has been astounding. Todays cards just hit it out of the park.

But to me SLI and CrossFire strikes me as a solution looking for a problem. They solve the problem of making the most expensive cards more expensive, but I don't think they're the best way to spend money on a flight simulation system. (Better might be to not buy at the "SLI/Crossfire" level of video cards, meaning spending $700+ on your video cards, but rather to go down a level and upgrade your motherboard/CPU more frequently.)

Some users email me asking for video card recommendations, in particular whether to buy an SLI/Crossfire configuration. The bottom line is, it depends on how much you value your money vs. your graphics card performance. If money is on object, and you want maximum speed, SLI configurations will provide the fastest performance (by some marginal amount). I believe that a good value lies below $200.

On the other side of the equation, I do recommend that everyone spend at least $100 if you're going to buy a video card at all. Below $100 the price cuts come from remaindering really old inventory and removing parts from the card to save cost. For the savings of $25 you might lose half your card's performance or half of its VRAM when you get down to the really cheap cards.

The other thing I tell users is the truth: no one at Laminar Research has an SLI system, so the reports we get on SLI come from users. Some users have told us they've gotten some benefit at very high FSAA levels. But at this point a single 8800 wll do the same thing. And SLI doesn't address CPU speed at all. Consider this list of features - nothing on the CPU side will get even remotely faster with SLI.

And in full disclosure: my two Macs have a Radeon X1600 Mobility, a Radeon 9600, and a GeForce 5200 FX sits on the shelf for testing purposes. (This isn't intentional bias toward ATI, it's what Apple ships.)

Using Layer Groups

I can't say enough good things about Jonathan Harris (Marginal) -- his work on X-Plane has been fantastic, he is one of the most advanced third party scenery authors I know of, and when he sends us a bug report, it is usually so perfectly patched up that I'm looking at the bad line of code in minutes! (One of these days I'll post one of his bug reports -- he always isolates the bug in a simple package that makes it very clear, with no extra "stuff".)

He emailed me a while ago with some questions about layer groups, and I saved them to rewrite into a blog entry. A little bit of background:

X-Plane 850 and 860 introduce the concept of "layer groups", which provide a way to control the draw order of scenery to some extent. Objects naturally fall into a layer group based on their type (e.g. objects go into the "object" layer group by default, and runways always go into the runway layer group). However, some scenery elements let you customize their layer-group placement in two ways:

• By changing which layer group the element goes to entirely, or
• By providing a "bias number", which indicates that within the catagory of scenery elements, this one must be drawn early or late.

Layer groups let you do a number of useful things:

• Make sure that polygons and objects are drawn under runways or over taxiways when needed.
• Make sure that runway markings with polygon offset are drawn before 3-d objects.

Simply changing the order of objects in the DSF is not a reliable way to control draw order! Layer groups are. You can read about layer groups in the OBJ8 spec.

With that in mind, some Q and A. (I will elaborate on my answers from what I originally sent Jonathan.)

J: I assume that objects and polygons within a single layer can be drawn in any order - ie there's no defined drawing order between different scenery types.

B: You assume correctly! Within a layer group, X-plane is free to reorder to improve fps. So you cannot rely on the draw order of any two scenery elements without assuring that they are in different layer groups, either by using different group names, or different relative offsets.

J: But is there are defined drawing order between objects/polygons and apt.dat-generated scenery?

B: Yes because airport scenery goes into specific layer groups! (In fact, all scenery has a "default" layer it goes into, and they usually vary by type of scenery element.)

J: eg if I have an object or polygon with ATTR_layer_group runways 0 will it be guaranteed to be drawn after the runway?

B: You're close. The runways go into the runways group so

ATTR_layer_group runways -1

will always be before runways and

ATTR_layer_group runways 1

will always be after.

ATTR_layer_group runways 0

is the default layer group for runways, so your object or polygon would be in the layer group with all of the other runways, and X-Plane would be free to change the order amongst your object and runways in any way that would optimize framerate.

J: If not, is there any other way to insert objects and/or polygons between the taxiways and the runways?

B: The numeric offset is provided for this. The "spacing" of the layer group numbers is such that you can have up to 5 groups before and after each "named" group. So anything from runways -5 to runways +5 is fair game. (In other words, you can separately control up to 5 different "layers" of elements with a well-defined order for any layer group name.)

X-Plane vs. Reality

A few days ago, Austin posted part of an, um, "animated" discussion between himself and an author regarding blade-theory vs. table-based flight models. (You can find it on the xplane-news yahoo group message archive.)

I'd like to ignore the whole "my flight model can beat up your flight model" thread and look at one of the side effects of physics vs. table based flight models.

In this previous post I commented on the nature of specifications in a flight simulator.

- Some data simulates real-world data. ("Reality-based") The sim has open authority to interpret this data for maximum "quality" and the standard is "how close to reality are we". The specification clearly sites reality as the authority on behavior.

- Some data is arbitrary and has a clearly defined interpretation. ("Specification based".) The behavior of the computer program is unambiguous.

What I find interesting is that a blade theory flight model is a "reality-based" flight model; a table-based flight model is a "specification-based" flight model.

What this means is that, just like reality-based specifications in the scenery system, you can't tune your flight model in X-Plane to achieve a desired end result without understanding the real-world meaning of the parameters you are changing, or you risk a compatibility problem with future versions of X-Plane.

Imagine that, for some reason, your plane seems to feel sluggish when turning. So you increase the area of the control surfaces and the problem goes away.

You can't do that with X-Plane's flight model! The area of the control surfaces mean something other than "a variable you can change to affect how the plane turns". They have to match how the real-life airplane is built. If you increase the area, straying from reality, to "fix" a problem, what really happens is you create a new problem later when X-Plane goes to simulate your model.

Simply put, if you put intentional errors into your plane's flight model to compensate for limitations to the sim, any improvement in the simulation accuracy of X-Plane is almost guaranteed to make your plane fly worse in the future.

So one of the important differences between a table vs. blade-theory flight model is how you talk about "bugs". If your plane doesn't fly the way it used to in a table-based flight model, that's probably a bug (well, depending on how interpolation is done). In a blade-theory model it's not a bug per se.

In a table-based flight model, how the real plane flies is moot - the table is king. In a blade-theory flight model, if the real plane flies differently and the input parameters of the plane are the same, it is a bug, or perhaps a design limitation.

CPU or GPU

If your X-Plane is framerate low, or you want to increase your rendering quality, you might think "time for a new graphcis card But is it?

Some rendering settings actually tax the CPU more than the GPU (graphics card). Here's a simple rule of thumb: if you increase the setting (and restart X-Plane) and your frame-rate does not go down, a new graphics card isn't going to make it go up!

For example, if you have one of those new-fangled GeForce 8800s, you may have noticed that when you turn on FSAA the framerate doesn't dip at all. That's because the 8800 is insanely overpowered for X-Plane (at normal monitor resolutions) and has plenty of extra capacity that will be sitting idle on an older PC. When you turn up FSAA, you are simply wasting less of the card's excess capacity. It goes without saying that if there were a card faster than the 8800, it wouldn't improve your fps any more than the 8800, it would simply be even more bored.

Here's a rough guide to which features tax the CPU vs GPU:

CPU-Intensive

• World Level of Detail
• Number of Objects
• Draw Cars On Roads
• Draw Birds (not that expensive for modern machines)
• Draw Hi Detail World
• World Field Of View (wider view means more CPU work!)

GPU-Intensive

• Texture Resolution (requires more VRAM)
• Screen Resolution
• Full Screen Anti-Aliasing (FSAA)
• Anisotropic Filtering (most cards can do at least 4x)
• Draw Hi-Res Planet Textures From Orbit
• Cloud Shadows and Reflections (not that expensive)
• Draw Hi Detailed World

A few specific framerate-optimization warnings:

• FSAA is equivalent to a higher screen resolution - that is, running at 2048x2048 and no FSA is similar to running at 1024x1024 and 4x FSAA. Both of these tax the video card with virtually no CPU increase. This is probably the only setting that can be helped only with a video-card upgrade.
• Texture resolution: do not worry if the total size of all textures loaded is larger than the VRAM of your card. To find out if more VRAM would help, measure frame-rate with your normal settings, with texture resolution down a notch, and with anisotropic filtering down a notch. If turning texture resolution down increases fps more than turning down anisotropic filtering, more VRAM may help. Machines with faster graphics busses (like PCIe x16) will be less sensitive to VRAM.
• Most Important: do not ever turn "World Detail Distance" beyond the default setting - you will simply destroy your fps and chew up your CPU for no benefit. I strongly recommend trying "low" for this setting - if you like a lot of objects, this setting can make a big difference in performance.
• The number of objects is virtually always a factor of how fast your CPU is, not your GPU -- that is, most GPUs can draw about a gajillion objects if the CPU could only get through them fast enough. If you are unhappy with the number of objects you can draw, do not expect a new graphics card to help - it probably won't.
• Cars on roads hurt fps on machines that don't have the fastest CPU.
• Draw Hi detail World is doubly dangerous - it uses both the CPU and GPU. Quite literally this is where we stash "luxurious" options. Everything this checkbox does chews up framerate. (If these options didn't, we'd leave them on all the time!) So you should not use this option if you aren't happy with fps, if you don't have a fast CPU, or if your graphics card isn't that modern. (HINT: if your nVidia card has "FX" in the title, don't use this!)

Start with the default settings and experiment - turn a setting up one notch, then restart, then turn it down and try another. Different machines will be faster for some things and slower for others.

EDIT: one user correctly determined (by observing CPU utilization relative to settings) that puff-style 3-d clouds bottleneck the GPU, not the CPU! This was not what I expected - when Austin originally wrote that code, our measurement indicating that sorting the puffs from far to near taxed the CPU a lot, making this CPU-intesive. At the time the old Rage 128s would also get bogged down by filling in translucent puffs as you flew right into the thick of the cloud.

Times have changed and neither the sorting nor the alpha-drawing is even remotely expensive on a modern machine. So I was surprised to see the CPU not being used. After some investigation, it turns out that while the CPU and GPU have gotten a lot faster over time, the communciations channel between them has not. The result is that they both do their jobs really quickly, and as a result clog up the communications channel...the CPU simply can't talk to the GPU fast enough to get the clouds out.

This is a great find by this user, as this is something that didn't matter on old machines, but can be optimized in the future for new ones.