Happy New Years

Lori and I are about to leave for a New Years Eve ski trip, but before I shut down the laptop for the last time in 2007 I wanted to say: Happy New Years to everyone in the X-Plane community. I had a lot of fun working on X-Plane in 07 and hopefully the sim brought you enjoyment too. I think 2008 is going to be very exciting - version 9 plants the seeds for a lot of interesting new possibilities during the version run.

When I get back I'll post a bit on panel regions, for which I have the first performance numbers, as well as some of the strange effects FSAA has on the sim. We should have a progress report on Linux soon too.

See you next year!

Script Files and Options

Sometimes we find that users machines cannot run without hiding OpenGL driver features from X-Plane. That is, the computer says it can support VBOs, but when the sim asks for a VBO, something really bad happens. X-Plane (since mid-version 8) accepts a series of command-line options that cause the sim to ignore the given feature.

These kind of bugs come and go as drivers are updated, the sim changes which technology it uses, and hardware cycles through the user base. The biggest one we're seeing right now is that the new iMacs show runway lights as white squares unless the sim is run with the --no_sprites option.

We're trying a new way to address these problems. In the past we would give users the command-line option; now we are building double-clickable script files that launch the sim with the appropriate options. Theoretically this...

• Is less error prone for users.
• Is quicker for users who may have to use the command-line option every time they launch the sim until a driver update becomes available.
• Is quicker for us (in that we spend less time mailing out instructions and helping users who are unfamiliar with a command-line environment).

We did consider some other options, but this seemed like the least evil. The runners-up:

• Just turning off the hardware feature permanently. We ruled this out because the performance hit would be significant and affect all users.
• Attempt to identify and auto-turn-off known bad options. We do this in some cases, but it requires changes to the sim code, so this does no good if a new video card comes out after a given x-plane release and introduces new problems. so it's not a total solution. Also since a bug might be resolved in-field, if X-Plane auto-avoids certain configurations we have to patch the sim once the configuration starts to work again.
  
• Provide a user interface to turn these options off inside the sim. This was ruled out for two reasons: first, some of these options cause the sim to crash before a user can ever get to the rendering settings. Second, turning off these kinds of options can really kill performance, so leaving the options in sight produces a whole new tech support option. (A user tells us their framerate is awful at the lowest settings...perhaps they turned off the hardware acceleration options in an attempt to get the lowest settings...etc.)

We'll see how well this approach works. So far it seems to be working better than handing out command-line options.

(Of course if we can address the problem by working around it with a change to our code, we almost always choose this option.)

The Global Scenery is Special (for now)

In X-Plane 8 the global scenery was installed in the "default scenery" folder, which normally contains most of the built-in artwork to make X-Plane (and the global scenery) work.

In the X-Plane 9 beta it is installed in the custom scenery folder. But the sim recognizes it by name and makes sure custom scenery is higher priority than global scenery. This is imperfect but works. In the future we may do something different. Survival tips:

• The log file tells the priority of all scenery packages.
• Don't rename or move the global scenery - leave it where it was installed.
• Custom scenery will always trump global scenery.

A Road by Any Other Name

In X-Plane 9, road networks can be in DSF overlays. And in X-Plane 8 or 9, road types can be customized using a road.net file.

Within the road definition file, each road type can contain:

• "Segments" (textured quads that make the roads)
• Objects (placed along the road)
• Wires (for powerlines)
• Car lanes (that vehicles drive along)

The interesting thing is: none of these types are mandatory. So you can use any of these features without the others.

"Cars" are just objects too. So one way to make a vehicle drive around an airport would be:

1. Create a road definition file with a new road type. That road type only has a car lane which references your vehicles OBJ model.
2. Make an overlay DSF with this new road type, drawing the path the vehicle OBJ will randomly take.
3. Package the overlay, .net file, and OBJ all together and you've got randomly generated vehicles moving around.

As Marginal pointed out on the .org forums, roads are currently not draped, so using them in overlays is a total PITA. This is something I hope to address in the 9.1 time-frame...draped roads are an important feature, but a little bit too invasive for the v9 betas.

NVidia: 2 Ben: 0

I found the root cause of another NVidia specific bug, and once again it's my own stupid code. If you Google for driver bugs, you'll find plenty of grumpy developers ranting about how card X does this wrong thing and card Y does that wrong thing...I figure it's only fair to follow up and say "yep, that one was mine."

Like the previous nVidia-only crash, this was a case where X-Plane was always doing something wrong, but only some drivers had problems with the behavior. So the crash was NVidia-specific, but X-Plane caused.

I believe that this bug was manfiesting itself either as a message that "scenery shift took more than 30 seconds" or some kind of crash. One of the problems was that the diagnostics for this particular bit of code were really bad. So we've improved things a bunch...

• There is more careful error checking during scenery shift, and those error messages are reported.
• If the sim does crash, some new code will output a crash log on Windows that helps us isolate what actually happened.

Beta 12 will be out soon with the fix that caused problems on NV hardware as well as the improved diagnostics. So you may find that the sim just works better, but if it does still crash or report errors, please tell us - now we'll have log files that will let us diagnose the problem a lot faster!

"Driver Bugs"

I have spent perhaps the last two weeks tracking down driver-related problems. But the term "driver bug" is heavily overused (blog around and you'll see that many OpenGL developers get frustrated...) A few examples of the gray areas:

• Sometimes there will be a bug in application code that shows up only on certain hardware. Drivers are concerned with making video cards go fast, not spanking programmers who don't know what they're doing. This is exactly what happened to me in beta 2 with the crash-on-nVidia-with-C172 bug. This was just plain broken code in X-Plane, but for some reason the ATI drivers didn't have a problem with it (probably because they were performing an optimization which let them ignore the bogus call I made). NVidia specific, but not NVidia's fault!
• On OS X with the Radeon 9600XT, runway lights don't show up. Adding an extra line to the pixel shaders "fixes" the problem. I believe the problem is in the driver (or the shader compiler more specifically) but by changing the code to the shader we work around the problem. A change in X-Plane addresses the problem, but not X-Plane's fault!
• The "36061" errors that some users have been seeing turn out to be because (through a very convoluted chain of events) X-Plane was asking the video card to operate in a mode that it could not operate in. Turns out this can be fixed by changing X-Plane's code (fix will be in beta 12) or by getting new drivers. This one wasn't really a driver bug - more that the drivers were limited in a way X-Plane did not expect. (Our fault for being picky!)

The situation is fundamentally tricky - games are integrators of other people's technology - as such, we get blamed by the end user for a fault anywhere in the system. At the same time, it's way too easy to turn around and blame the part supplier, and unfair when the source of the bug hasn't been identified.

I am looking now at problems on Windows with dual core machines and nVidia cards. The problem goes away both by changing a registry setting that affects the driver and by changing X-Plane's code. So I think it's too soon to tell on this one.

A Tools Update

I've spent perhaps the last two weeks almost entirely looking at driver-related graphics bugs. I should note that these are often not driver bugs, and the issue of whether a bug is in the sim or a driver is a lot stickier than you'd imagine, but I'll blog about that later.

In the meantime, where do we stand on scenery development tools? Here's a rough summary.

General Strategy

The scenery tools are not targeted to X-Plane 8 or 9 - they'll export content for both (but if you use new V9-only features, your content will be v9 dependent).

There's been one structural change in v9 that's worth mentioning: version 9 allows road networks in DSF overlays. This cleans up the scenery model a lot...roads are more like objects and forests than like the base mesh, so now all of the 3-d elements can be in an overlay. This is particularly useful because you might want to adjust the road and objects nearby all at once.

Given that we have no road editing tools right now, I think there is one case where the tools will be v9-centric: I do not think I will ever provide a road-editing tools for DSF base meshes with roads. I do think I will add road editing to the list of overlay features that will eventually end up in WED.

This means that if you want to customize roads, you'll have to do it in version 9. This is an inevitable outcome of the scenery system structure...replacing the entire base mesh to edit the roads is not a great idea.

WED

WED is in beta 4, and I think it will go final as soon as I have time to make it official. At this point most of the open bugs I get are usability issues and enhancement requests, not real bugs. We'll get more usability features into WED in the future.

Please don't send me WED bugs now (or attach them as blog comments). I'll put out a formal last-call for WED bugs when I have time to dig in.

The future plan for WED is to add DSF overlay editing with all legal overlay types (which is now everything except beaches and base mesh).

AC3D

The AC3D export plugin already supports new v9 OBJ features - key frames and panel regions. These tools are available now!

Tools

The "tools" are the set of other scenery apps: DSF2Text, ObjConverter and ObjView. We'll be adding two new tools soon:

• DDSTool is a PNG->DDS converter. There are already other programs (many free) for both platforms that can do this conversion, but we wanted something easy and drag & drop when we were doing the in-house conversion, so I coded DDSTool (based on the Squish library). We'll make it available to authors.
• MeshTool is a base-mesh generation program. We're still working out the details on this, but basically it will generate DSF base meshes from raw DEM data and polygonal coastlines.

I am restructuring the tools to be entirely command-line-based (except for ObjView) with a single "drag & drop" UI that wraps all of the tools. So if you use the mouse, you'll use the one drag & drop application to do all your DSF2Text converions, 3DS->OBJ conversions, PNG->DDS conversion, etc. This is mainly to cut down the number of apps I have to build, but it should also declutter workflow a bit.

MeshTool

I'll post more on MeshTool once we have more details, but here are the main points:

• MeshTool will not have a UI. It will take text files and process them. It is designed to provide a building block on which people can write more advanced tools. In other words, MeshTool gives other programmers our mesh algorithms just like DSF2Text gives other programmers our DSF-packing algorithms.
• MeshTool will not be an editor. It will create NEW meshes from RAW DATA, not from previous meshes. There are a lot of reasons why I do not want the mesh-creation workflow to be "take the global scenery and hack it". So MeshTool lets you build your own.
• This means you will need your own elevation and coastline data for MeshTool! You can't use what shipped in the global scenery! However, since the data that ships the global scenery is freely downloadable and widely available, I don't think this is a problem.
• MeshTool will be able to "burn in" orthophotos into a mesh. This is probably its most important feature. Right now the only way to make orthophoto-based scenery is to use overlay polygons, but the performance of .pol files with a lot of scenery is really poor under version 9. MeshTool will let people build photo-sceneries that take full advantage of the framerate that X-Plane can deliver!

MeshTool should address some of the common issues I'm hearing with orthophoto sceneries (slow performance with a ton of .pol files, no ability to fix the coastlines or remove underlying base terrain, no ability to fix elevation).

MeshTool will create new default landuse terrain where there are no orthophotos or water. This is important because the elevation data can be customized. If you take an existing DSF and somehow edit the elevation points (for example, use DSF2Text and just start editing) and raise a grassy field, you'll have grass going up a cliff. But if you pass the raw elevation data to MeshTool, it will build a rocky cliff over steep elevation, creating more plausible scenery.

MeshTool should be in the post 9.0 final time-frame, as will all the other tool reposts.

I will reply (soon)

At this point my in-box has approximately 180 emails from the month of December regarding X-Plane 9. So while I appreciate all of the feedback we've gotten (bug reports, performance, etc.) it's going to take a while to reply. If you haven't heard from me, don't panic! I hope to answer a whole pile of emails next week.

In the meantime, I've been working on improved crash logging on Windows. Right now when we have a crash on Windows, all we know is that (1) X-Plane crashed and (2) what DLL we crashed in (which is always us or the video driver - not useful).

Coming soon, X-Plane will catch the fatal crash, examine memory to see what was going on, examine its own EXE to figure out the names of the functions going on, and output it all to a crash log that users can send us to get a much clearer picture of what's going on. This information is called a "backtrace" - we've had it for the Mac or a while (OS X provides back-traces automatically with a crash) and it's really useful.*

So my top priority is all of the users who are seeing problems during scenery load, and a new build with a back-trace should help to reveal what's really going on.

I'm also working on putting additional timing and performance information into the sim so we can learn more about why some users have poor performance. So far here's what I'm seeing:

• 8800 users seem to have great performance. If you have this card and don't have good fps, adjust your x-plane settings and NV control panel settings - this card can bring it.
• 8600 users sometimes have performance problems - not sure why.
• Older nvidia GPUs (7600, 6800) sometimes have performance problems with the new eye-candy features - I am investigating.
• The pixel shaders seem to slow down the new HD2x00 Radeons a lot more than expected...I still need to investigate this. This is the most surprising datapoint - the X1600 does very well, so I would expect newer GPUs to at least have that level of performance. I think this is something we might be able to address.
  

However not all of the reports are consistent, so I think it's too soon to make some calls on recommended hardware. The only thing that's clear is that most 8800 useres who we do careful perf experiments with end up with huge framerates.

* Microsoft provides some back-trace facilities, but since we don't use their compiler tools, we had to roll our own.

V-Sync - Problematic in Practice

To those who have sent performance info: thank you, but you probably won't hear for me for a week. I'm up to my eyeballs in reports and it's going to take a while to get through them.

I finally found the code that allows X-Plane to turn off V-Sync. This should help nVidia users who are having framerate problems.

The basic idea is this:

• X-Plane tells the graphics card to draw a lot of stuff.
• The video card accumulates this "todo" list and works on it while X-Plane runs.
• X-Plane indicates that the entire frame to be seen is done and tells the card to show the results.
• Eventually some time later the card finishes the todo list and then shows it to the user.

V-Sync relates to the question of when this last step happens. When V-Sync (vertical sync) is off, the card shows the results as soon as it is done drawing.

But when V-Sync is on, when the card finishes drawing the world, it then waits until the monitor is done drawing its frame, and then shows the results. Without V-Sync we can have a situation where the top half of the monitor is showing a new frame and the bottom half is showing an old frame. (This is called "tearing".)

So normally V-Sync is good because it prevents tearing. But the problem with V-Sync is that a frame can only be shown when the monitor refreshes. The video card has to wait until this happens, and this slows our framerate down.

In particular, most users have their monitors set to 60 hz. If X-Plane can only produce frames at 50 hz, the video card will have to further slow the framerate down to30 hz (one x-plane frame for every two monitor refreshes). If X-Plane falls below 30 hz, we end up with 20 hz (one X-Plane frame for three monitor refreshes), and if X-Plane goes below 20 hz, we would clamp at 15.

So when monitor refresh is on, there can be large framerate hits for small losses of performance in the sim, and a real risk of getting locked around 20 fps.

(The minimum framerate in X-Plane is intentionally set to 19 so that we won't fog up if the video card clamps us at 20 fps.)

So when beta 11 comes out, you may get some framerate back if you haven't already hacked your graphics card's control panel settings. If you still want v-sync, you can always set it this way in the control panel. But most users I've talked to are happy to have it off.

In an only vaguely related note, one of the reasons to have high frame rate is to have a smooth flight model. But Austin has now put a new setting in the operations-and-warnings dialog box: you can pick how many times per graphics frame the physics run. The normal ratio is 1:1, but for fighter and acrobatic pilots, you might find that you can get a nice feel at lower fps (20-30) by setting a higher ratio.

Cores and Drivers and Vsync

Random thoughts:

Update your drivers! Version 9 uses driver features that version 8 does not. Just because version 864 works doens't mean that your drivers are up to date and bug free! First thing to try when weird things happen in version 9 is a driver update.

If you have 60 fps and a rendering setting cuts it to 30, you probably have vsync on - that is, your graphics card can only run at an even divisor of your refresh rate. The next hit will be 20, then fog. Change your monitor refresh rate to 75 or 80 hz. If the framerates all change (to 80, 40, 20, etc.) it must be v-sync. Turn v-sync off for better framerates under heavy load. Nvidia users, you need to turn v-sync to off, not application controlled.

X-Plane 900b7should be able to put sustained load on three cores - if you're recording a QuickTime movie, one core draws the world, one compresses QuickTime frames, and one rebuilds 3-d as you fly. So...I guess we've already hit a point where a quad-core machine has some benefit over a dual-core machine. (I think we'll start to see more central features use more cores during the v9 run.)

The new forests rebuild 3-d very frequently - dual core users who run on "tree hugger" should see utilization of 75% or higher on both cores, depending on video card power. (If your CPU usage isn't 100% then probably your video card is holding you back - turn down FSAA.)

Now we can talk v9-performance

I think we're reaching the point where I can start looking at version 9 performance in detail. Over the past few betas we've fixed some very general and global performance problems and also some crash bugs. Now we're reaching the point where some users have fast performance and others don't, and it doesn't quite match up to the hardware the way we'd expect.

The biggest single complaint I'm hearing is "pixel shaders slow may machine down". This is expected, but the level of slow-down we're hearing about (2x, 3x, 4x fps loss) is a lot higher than we'd expect.

Here's a very basic test case you can try:

1. Start with shaders off, no FSAA, 4x anisotropic filtering, and a texture res that doesn't push your card too far ("very high" for 256 MB cards). Tune objects to get between 30-50 fps all the time. Keep forests to "none" for now - they use the second core, so better not to bring in another variable for testing. Measure fps while on the ground somewhere, so that the camera isn't moving around. Pick a view that has no panel at all, like shift-w (forward-no-HUD view). Use the default 747.
   
2. Now turn shaders on. Turn water reflection to none, turn volumetric fog off. What's the new fps (keeping the same forward-no-HUd stationary 747)?
3. Now turn volumetric fog on (water is still "none"). What are the fps now?
4. Finally turn water reflections to "default". What are these fps?

I'd like to know these four fps numbers because they show the incremental degradation to performance as settings are slowly brought in.

Now if these numbers come out okay, but some real configuration that you tried was terrible, tell me, what's the delta? (For example, did you have FSAA cranked up when you were testing? What was the texture res?) That will tell me the delta between controlled situations and real-world usage cases.

It will probably take me up to a week to get back to performance reports. Best venue right now is probably: send me an email with attachments for your log.txt file. Please don't post performance to the blog comments section, I think the thread will get unreadable.

It's too soon to evaluate whether you need a new video card. I'm hearing reports of GeForce 8800 users with low performance - and yet I have good performance on an X1600. This implies that there's a driver call we make that isn't fast in some cases that we need to replace with something better. Let's get that worked out before you go out shopping!

Key Frames, AC3D, and Compatibility

I've posted the first beta of the AC3D X-Plane export plugin version 3.1 on the scenery web site. This plugin contains support for key frames and panel regions, two new version 9 features.

Generally, the scenery tool code is not part of the X-Plane code. There aren't version 9 and version 8 scenery tools. There will always be one set of scenery tools that let you optionally use the v9 features or not.

An example is the AC3D plugin. Everyone can use the new plugin (and a few small features apply to X-Plane 8 too, like a less sluggish animation UI when editing large objects). If you use key frames, your object will only work with v9 - if you don't, it will work with v8.

More specifically on key frames: a traditional v8 animated object is really just a key framed object with only two key-frames. So the UI has changed very little - basically where you only had two key frames (points of correspondence between the dataref and animation) in the old plugin, you can now use the "add" button to add more.

I will try to add features to the scenery tools to warn when v9 features are being used, so that if you want your scenery to run on v8 and v9, you can check for v9-only feature usage.

Reflective Airplanes

I woke up this morning to one of those funny coincidences that defines the experience of working on X-Plane: two users had emailed me. One was asking whether we'd be extending water reflection technology to airplane fuselages (like some other programs have) and the other made the case that such an extension was not necessary. The two emails arrived in sequence! (Perhaps there was a forum debate on the subject somewhere.)

First, I can tell you that if we ever have reflective airplanes, it won't be that soon. I have a number of features for version 9 that are in progress and need to be finished off before I can start anything new.

Reflective airplanes are on my "investigation list"...that is, a feature where we want to do the initial research to see if it could be implemented in a way that makes sense (for X-Plane this means: would it look good and not kill fps too badly for at least some segment of our users).

I believe the X-Plane 9 version run will start to contain more features for serious 3-d modeling of airplanes. Version 9 already features 3-d lighting in the 3-d cockpit, key frames for animation, and a ton of new datarefs to drive that animation. We're going in the direction of being able to model the plane in absurd detail.

We're also looking at the lighting model in X-Plane. We've only started this work for version 9, but consider pixel-shader-based water. Even in the "no reflections" case, the pixel-shader based water is a reflection of the real sky (as rendered) with a procedural texture to create waves. When you compare this to the version 8 water, you can see how having really close alignment of the coloring scheme for all parts of the sim creates more of a sense of realism.

So reflective airplanes are at least on my list of things to try. I have seen users do wonderful amazing "metal" textures on airplanes, but the one thing that I think holds them back is that all metal airplanes have some kind of tinting assumption to them based on the reflection used...typically these are "blue-based" (meaning they look right on a sunny day) or "gray based" (meaning they look right on a cloudy day). But if you put the plane in the other environment the texture looks a lot less convincing. Reflective textures would let authors really use the real sky color on the plane, for consistent lighting (especially when the plane's orientation changes and the blue side isn't up anymore).

On the other hand, reflections are expensive. Planes reflect light from all sides, so we would need to take reflections from all angles (the water always reflects up, which is a huge savings). For low-quality settings for water, we drop the terrain, and since the terrain only reflects at the water's edge, this is a pretty tolerable omission. An airplane reflection with "sky on the bottom" would look absurd. (Similarly, the water tends to only reflect things that aren't on camera, so the total rendering load of water + the world tends to be static. The plane would pick up a lot of 3-d objects even in orientations where they don't do much good, so plane reflections would become expensive.) And the plane reflection isn't usable for any other plane...do we build them for all planes or just the user's plane?

Certainly right now it's still too soon to tell. Not only have I not done the research into this feature, but we still don't have comprehensive performance data on the water across lots of hardware. A number of users are reporting huge framerate loss on the lowest water settings. This implies that our "render-to-texture" code is slow on some hardware but not others. (The fps loss on my laptop with the lowest water setting is less than 4%.) Render-to-texture is new to v9 and used heavily, so we need to understand how it scales for all users before we go further.

Finally, there is a whole area of 3-d techniques that X-Plane does not yet use that could make sense for airplane modeling: artist controlled fake lighting.

For example, imagine if the airplane contained a single "reflection" texture - this texture would contain a fake ground texture and alpha transparency where the sky color goes. X-Plane could then fill in the sky color (where there is transparency) only when the weather conditions change, and then apply the texture keeping the plane's orientation in mind. Such a proposal would give the plausibility of reflections (correct coloring on all parts of the plane across lighting, orientation and weather conditions) for a fraction of the cost of "real" reflections. I'm not saying this is the best idea, just that there's a lot of intermediate ground between "full reflections" and "make a static texture".

Dr. House

Lori and I are hooked on the TV show "House", where Hugh Laurie plays a really grumpy doctor who solves bizarre medical cases more or less by ESP. The characters are well written, but the medical plot line is somewhat predictable: there are four quarters to the show - in each one House except the last, House will make the wrong diagnosis and the patient will get worse right before the commercial break. (Usually this involves massive bleeding or cardiac arrest going into the long commecial block at 0:30.) None of the symptoms fit until the very end when House finds the simple right explanation they just couldn't see.

This set of nvidia crash bugs felt a lot like that - we had multiple attempted fixes, some of which didn't help at all, until finally after multiple tries I found a bug that explained all of the otherwise completely weird behavior we were seeing.

But I must admit - I have brought shame on the house of X-Plane...the buggy code was mine and the mistake was really stupid. Why nVidia on Windows? As far as I can tell the optimizers present in most OpenGL engines can change whether (and how) the bug manifests itself - different OS/driver pair: different engine with different optimizers.

Now that (at risk of massively jinxing ourselves) we have the crash bug fixed, I will resume performance work. Once we get a build done with all of the immediate performance items I want to cover, we'll start collecting user reports on in-field performance. So I should have more specific instructions on what you can do to help us isolate performance problems in the next few days.

What is a panel region?

X-Plane 9 introduces a new OBJ feature: panel regions. The basic idea is this:

In X-Plane 8 you could use the 2-d panel as a texture in your 3-d cockpit. This allows a plane to have a moving map or glass display in the 3-d cockpit. However, the panel texture is expensive, particularly in v9 for a few reasons:

• You have to take the entire panel, even if you don't use it all. For example, consider all of the "wasted space" from the windows.
  
• The real texture is rounded up to a power of 2 (and in x-plane 9 that could mean 2048x2048 for a a 1600x1600 panel.
• The texture has an alpha channel, which probably isn't usfeul (model your 3-d cockpit windows in 3-d, not using transparency). The alpha channel increases VRAM use by 33% and requires some pixel shader gymnastic in v9 that slow things down.
• In X-Plane 9 this is all twice as painful since we have a panel day and and lit texture.

Panel regions address all of these problems. Here's how they work:

• A panel region is a rectangle within your 2-d panel. It can be placed in any location as long as it is (1) fully within the 2-d panel and (2) its dimensions are a power of 2.
• The cockpit object declares up to four panel regions it wants to use.
• A new attribute lets you use each of the four panel regions as a texture (alpha is not provided - the regions act opaque).

This does exactly what you might expect: it creates between one and four smaller power of 2 textures rather than one huge one and manages those smaller textures. We do have more textures, which is usually bad but we get some big wins:

• Better VRAM use. The panel texture, being a dynamic texture, puts a lot more pressure on VRAM than regular scenery textures. Without this optimization, we could be paying 25 MB of permanent VRAM use just for the 3-d cockpit. Now we don't have to pay to round up to a power of 2.
  
• Faster updates of the 2-d panel into the 3-d texture, since we have to process a lot fewer pixels.
• Efficiency - a clever author can cut down the panel use to only the parts that really matter, which might only be the EFIS at 256x256 pixels.
  

I will try to provide detailed documentation on this in the near future. There will be a new ac3d plugin coming out (hopefully in the next week) that will provide both editing capabilities for key frames and for panel regions.

Better Bug Reports

(Probably I'll be blogging a lot today...the load/change-planes/crash/recompile cycle I am going through while working on the crash bug is a slow one - my old Dell is long in the tooth...it leaves me a lot of time to post.)

Beta can be frustrating for both users (why don't they fix the bugs I reported) and programmers (I need more details in my bug report). Here are a few thoughts on what makes an initial bug report useful:

• Precision of reproduction. This is probably the most important thing - we get a lot of "open an airplane"-type instructions. Which airplane? It turns out that a lot of bugs depend on the particular content being used. So if you know how to make a bug happen, please describe it in the most painfully precise steps possible!
• Short is beautiful. We must know precisely how to reproduce a bug, but a procedure that takes two hours kills our productivity. So please try to determine how to reproduce the bug with the minimum number of precise steps.
• Clean system. Bugs that involve only the default content shipped with the sim are more useful for us because they're quicker to find and more likely to be due to a bug in the sim itself.
• Nuke the prefs. Bug reports that start with "delete your preferences" are good because it means the bug procedure starts from a known state (the sim defaults). We get bugs that we can't reproduce because something is subtly different in our system. Killing prefs is the quickest way to eliminate this case.

As an example, the cleanest, simplest version of the nvidia crash bug would be:
1. Delete prefs.
2. Start the sim.
3. Open the C172 using the "open aircraft" dialog box.
Result: unexpected program termination before the terrain is visible.

Now I have a 6200

We still haven't fixed the crash bug in X-Plane 9 yet, but we have narrowed it down to:

• Windows only
• nVidia only
• Pixel shaders only

We actually know partly what happens, but we don't know why, and we don't know how to fix it yet (short of disabling VBOs or pixel shaders, both of which are unacceptable workarounds). Austin's shipped b4 with shaders off, just to get people running while we work on this.

So I went to BestBuy last night and bought a 6200 (out of desparation) - I can now see the crash bug for myself. My framerates are also terrible. My guess is I can tweak the drivers, but having experience now with the 5200, 6200 and 7300, I can safely say: avoid the lowest end nVidia cards. In particular:

http://en.wikipedia.org/wiki/Comparison_of_NVIDIA_Graphics_Processing_Units

The important thing to look at here is "core config" (although bus width and clock speeds will always be higher in the higher end models). For the last four generations, nvidia has made three levels of cards: a hig end card (with a number like x700, x800 or x900), a mid-range card (usually x600) and a low end card (x200, x300). So for the GeForce 6 series, the 6800 was the flagship card, the 6600 was the mid-range card and the 6200 is the "budget" card.

Each time they go down a level, the number of shaders and other parallel hardware resources gets cut down. This translates directly into lower framerate, particularly in X-Plane 9 where we're shader-bound when you use the shader features. I often recommend the mid-range cards as a good value, but the low end cards aren't worth it - they're too stripped down for not much price savings.

How Framerate Dies - Glitching

Back in the good old days (that would be X-Plane 6), X-Plane's framerate would suffer in two ways as you cranked up the rendering options:

1. For most features (more visibility, more autogen) as the CPU and GPU became more heavily loaded, the framerate would gradually decrease.
2. If you ran out of VRAM (that is, the working set of textures needed per frame was more than your card's VRAM) framerate would really die fast - think 2 fps.

The reason for this second behavior was that computers couldn't shuffle textures between main memory and VRAM fast enough to render a frame in a 30th of a second.

As computers have gotten faster, this second behavior has gone away - modern cards, with fast PCIe 16x busses, can transfer textures from system memory to VRAM pretty fast - fast enough to have the working set be (slightly) larger than VRAM and still fly. So as texture memory increases, framerate decreases more gradually.

However, a new behavior has emerged: "glitching". You may have noticed that when you've got your computer set to the ragged edge of the rendering settings, as you turn the camera, the framerate will stutter for a few frames, then return to a relatively high rate (40-50 fps).

What's happening is: the working set of textures and geometry needed by X-Plane just barely fit in VRAM. But when you turn your head, a different set of textures and geometry are needed. While the card sorts out what is needed and what isn't, it spends some time needlessly shuffling textures, and eventually reaches stability, with only what's needed in VRAM, and framerate stays high.

Glitching has emerged as a mode of performance degredation because over time we've cut down the amount of "stuff" (textures and geometry) x-plane needs to draw a frame to only what's really absolutely needed. This means there is less intersection between the working set in one view and another, and it also means you can get closer to the edge of your hardware.

So my view on glitching is basically "too bad". If the working set weren't as carefully trimmed, you wouldn't have glitches, but the framerate would be entirely low, not entirely high. The only solution is to turn down settings that increase the working set (object density, world LOD, tex res, forests...) until the computer can run without glitches.

An even stranger variant of this: users sometimes report framerate getting "stuck" at 19 fps and then coming back when they change apps. The problem is that the driver doesn't know exactly what the best order of textures to keep in VRAM vs. shuffle is...as the view changes, sometimes the driver ends up with a non-optimal decision about what stays in VRAM and what goes, causing framerate to drop. Changing which app is in the foreground fixes this by temporarily pushing a lot of items out of VRAM, at which point the driver makes a different decision by luck.

Again, the solution is simple: turn down rendering settings to get the working set smaller than VRAM.

Basically, if the working set is smaller than your amount of VRAM, you should have even framerates, proportional to rendering load.

If the working set is greater than VRAM, the driver may find an optimal way to shuffle things and only decrease fps a little, or it may find a non-optimal way to shuffle and you'll get terrible fps, and that shuffle can change on the fly, causing framerate to fluctuate all over the place.

Two Truths of Hardware

I would have to say my track-record at predicting hardware developments in the sim world has been pretty poor. But these two factors seem like they won't change for a bit:

1. The amount of working data a system can crank through in one frame keeps increasing while the total amount of virtual address space stays the same (3 GB).
2. The gap between the fastest and slowest systems from a finite time period keeps widening.

To elaborate on this first point, as video cards get faster (both GPU and internal bus/RAM speeds), system RAM and buses get faster (graphics slot changes are rare, but with enough VRAM this is relatively moot) the amount of texture and geometry data that X-Plane can tell the card to draw in a 30th of a second keeps going up. So users are running with more trees, roads, 3-d, etc. than in the past.

But all of this stuff lives in memory, and even if a user has 8 GB, X-Plane can't load more than 3. Imagine what will happen when a graphics card can draw 3 GB of data in one frame? X-Plane will have to use all of its available memory for things you see right now or it will waste graphics power. This would mean purging from memory anything that isn't on screen!

On the second point, video card power doubles every, um, six to twelve months (perhaps more like 18, now that the card makers are hitting the same fabrication and power limits that the CPU makers have already hit, but this is all seat of my pants). So even if we only supported the last three generations of cards (we support at least seven!!) that gap in performance doubles!

This means that every year it requires a more flexible rendering engine to make a sim with decent frame-rates for old computers and up-to-date graphics on new ones.

Obects and Liveries

In X-Plane 9 betas 2, you can use the livery system to provide alternate textures for misc objects attached to your plane, but not the cockpit objects. Beta 3 will address this, allowing cockpit objects to reference the livery system too. We'll get some examples posted for how this all works soon.

Austin and I were discussing this last night - here's a few thoughts on the difference between the misc. objects and the cockpit object:

• Because you can have multiple misc. objects, you can effectively use more than one texture for your airplane. These days plane designers want a lot more than one texture.
• We're working on optimizations for attached objects - basically by breaking your plane carefully into a few objects, you may be able to optimize frame-rate. A lot of this code is not in the sim yet, but will go into a later beta.
• Using multiple objects with different LODs is a much more efficient way to improve fps than simply having multiple LODs of the entire plane.
  

For example, make an interior object with a low LOD (0-500 meters) and an interior texture. Attach it. Then make an exterior object with an exterior texture and a longer LOD (0-50000 meters). When you back away from the plane, X-Plane can entirely skip the interior object, which means that neither the geometry nor texture have to be moved to VRAM. That's a big performance win.

The cockpit object is also different:

• The cockpit object induces the sim to make a texture out of the 2-d panel. (And it is the only object where the panel texture is legal.)
  
• Mouse-click analysis is only done on the cockpit object. If you load up the cockpit object with all of the animation for your landing gear, X-Plane has to wade through all of that animation to update the mouse cursor in the 3-d cockpit.
• The cockpit object can be swapped for an alternate object in the exterior view (we'll provide more control of this for misc objects soon).

So my general advice is: use the cockpit object only for the real panel itself!

However, we have provided livery support. You can't livery the 2-d panel, and the livery system is not meant to reskin cockpits, but there are probably some planes out there that used the cockpit objet to model exterior plane features (from before we had misc objects). The livery system in beta 4 will let you reskin these planes without changing how your obects work.

A wide range of performance

One trend I'm seeing in the forums is a wide range of reports on X-Plane 9 framerate. I see both "X-Plane 9 runs faster than 8" and "it's so slow I can't fly at all."

In my experience this kind of situation is often due to hardware-specific performance issues...that is, X-Plane is doing something that is fast on one kind of GPU but slow on another. Once we figure out what the problematic operation is, we can replace it with something that's fast for everyone.

If you've got all of the eye candy turned off, everything on minimal, and are still getting 20 fps, that's not X-Plane 9 being a pig, that's something wrong with X-Plane 9 on your particular system. Please...try beta 3 when it comes out, then if it's still slow, report a detailed performance bug and we'll investigate.

Beta 3 - Don't Panic

X-Plane 9 is currently in beta 2. Long-time X-Plane users know that a lot of bugs get fixed between early and late betas, and they also know that a good number of bugs get added between the early betas.*

If all goes well, X-Plane 9 beta 3 will be out in a few days. My advice is: don't panic. X-Plane beta 2 crashes for a number of users, so our top priority is to fix the crash and get the fix out ASAP. If your bug doesn't get fixed in beta 3, it's probably because we're still working on it but didn't want to delay getting the crash fix to users.

Similarly, beta 3 will include some performance improvements, but more are coming. Beta 3 doesn't represent the end of our performance tuning, it represents the first beta that we can do serious analysis with. We only have a fraction of all of the supported video cards within the company, so if your computer is having performance problems, well, we'll figure out what's going on in beta 3 and then fix it.

* Our approach to bug fixing is: if a piece of code is buggy because it's subsystem has a design problem, we go in and fix the design problem, even if we're in beta. Other companies might say "no fixing design problems (which changes more code) during beta." But the way I look at it, badly designed code is just going to cause problems all the time until it's fixed, and it has to be fixed some day, so why wait.

(Why would there be badly designed code in X-Plane, or any computer program? Computer programs change over time, and the functionality they perform changes and grows. As this happens, the designs of the past no longer make sense for future requirements. In my experience most design problems come from code "outgrowing" its framework.

So our approach is to upgrade the framework as soon as it shows signs of growing pains, rather than jamming as many features into the existing framework as possible until it becomes so overcrowded that we can't get anything done.)

Multi-Core and X-Plane 8 and 9

To answer some speculation and questions I've seen on the forums regarding X-Plane and multiple cores:

• X-Plane 8 and 9 will use up to four cores to "shift" the scenery coordinate system when scenery needs loading. This shift causes a pause in both X-Plane 8 and 9, but it is a relatively short pause.
• X-Plane 8 and 9 will use a second core to build 3-d geometry out of road, forest, facade, and airport definition data while you fly, as you move through the loaded scenery area. This is particularly important to smooth flight when there is a lot of 3-d enabled.
• X-Plane 8 and 9 will use a second core to do some of the work of loading textures from disk while you fly. (Right after a scenery shift happens, over time new textures are loaded in.) Some of that texture load is still done on the main thread. Texture compression is done on the main thread, which can cause stuttering. (I expect to address this in the future, but authors can use DDS textures now in v9 to simply not do the compression at all and make everything faster.)
• X-Plane 9 will load new DSFs into memory on a second core. Loading new DSFs takes the vast majority of scenery load time, thus flight with X-Plane 9 and a dual-core machine is a lot smoother than it was in v8. This is a big win for the dual-core machine and is new to v9.

All of our new work on the sim keeps multi-core in mind. We know that more and more users will have multiple cores and that we're leaving performance on the table if we can't distribute rendering load across two cores.

So the important points here I think are:

• X-Plane 9 significantly uses a second core.
• Use of the second core has grown significantly over time, and will increase in the future.
• I like to use bullet points way too much in my blogs and emails.

X-Plane 9 Performance - Let's Wait a Little

When someone posts bad performance with X-Plane 900b2 vs. 864, the urge for me to respond is overwhelming. I spend a lot of time very carefully comparing old and new versions in controlled situations to make sure we haven't let any part of the engine get slower.

For now I will try to hold off - there are a few basic things we can do with beta 3 that will change not only the performance of the sim, but also how the rendering settings are organized.

For now a few immediate thoughts:

• If you are looking at a new graphics card, don't plan your hardware purchases on the beta. Wait for a final build to evaluate.
• There is no free lunch - settings that don't exist in v9 (volumetric fog, reflective water, forests) are going to cause lower fps than 864. (You could set the forest setting to "insane" in v864 with no penalty, as long as you had no forest add-ons installed.)

And finally, something I find myself saying over and over and over: there is no relationship between the fastest computer on the market today and the highest settings in X-Plane. We do not guarantee that the fastest computers can run x-plane "maxed out".

The highest rendering settings represent the most detailed output our engine can make, and the most data we have (based on what fits on the DVDs). We provide this very high rendering setting because:

1. A user might want a lot of detail in one area and not others. You can run on "insane" roads if you leave the other settings dialed down, or reflective water can be on if you don't have a lot of objects or world LOD. So even if having ALL settings on maximum kills virtually any computer made today, having some settings on maximum can be useful.
2. If you buy X-Plane 9 now you are entitled to every patch (and then you may use it long after the version run is over). So you might be using X-Plane 9 for 2, 3, or 4 years. Settings that are impossible to reach on today's hardware may be very usable on future computers.

So we do not penalize users who want some settings very high, or future users, by preventing you from setting the settings in any way you wish.

Please understand that the settings are not calibrated the same as X-Plane 8. If you set the rendering screen to look "as close" to x-plane 8 as possible, you are almost definitely not getting the same amount of detail (and the same load on hardware) as you were in 864. They are not comparable. Trust what you see out the window while flying, not what's on the rendering settings screen.

X-Plane 9 Is Here

There will be a lot more to post about this soon...a few immediate comments:

We have an order in with one of our server providers to get a lot more bandwidth for demo/update downloading...the servers are going to be pretty slammed, but we're doing our best to keep downloads fast. I believe the beta should be about 600 - 700 MB over the net.

X-Plane beta 1 is on the DVD. X-Plane beta 2 is on the net. In other words, we've already done our first patch since the DVDs went master. This patch improves performance and fixes QuickTime movie recording on Windows. (BTW QT movie recording is completely overhauled for version 9.) So once you get DVDs, make sure to run the web updater to get beta 2. With a DVD install this is a tiny download, since the updater only fetches changed files.

The DVD set is "X-Plane 9 beta 1", but the scenery and terrain/earth orbit textures are final. So you can buy DVDs now and simply use the web updater to get the "final" X-Plane 9.00 (plus of course all of the free upgrades and patches for the life of version 9).

This is the first major version release where we've had our updater. In the past we've had to either post a new version without a beta, or put a beta on DVD. With X-Plane 9, the large files that aren't available for web updating are finished, and the rest of the beta can be upgraded over the net. I hope this allows people to participate in the beta program and still enjoy the sim with full scenery.

ANIM_hide is not a framerate optimization

I'm sure I blogged this before, but the blog's gotten old enough that I can't find my old posts. This came up in an internal company discussion, so let me say it again:

ANIM_hide is not a framerate optimization. 99% of the time it doesn't make your object draw faster.

ANIM_hide is provided to let authors make certain animated effects (like swapping out a prop disc for blades based on prop RPM). These are animated effects and cost fps, because they use the CPU and interrupt the graphics cards from just blasting out triangles at warp speed. An example:

You might have a very complex object attached to a plane that doesn't need to be drawn at all, like a landing gear assembly. This object could have its own texture, a ton of animations, and high triangle count. Using a giant ANIM_hide around it to prevent drawing when the gear is retracted does you no good. Here's why it doesn't help.

• Before an object can be drawn, X-Plane prepares the geometry and texture to be used by the card. These are atomic operations (we have to prepare the whole texture and all of the geometry no matter what we will actually use) and they are expensive.*
• X-Plane then must evaluate the animation command on the CPU to decide if the object must be drawn. If the object drawing is simple and the object is drawn, this CPU work is just wasted, so this technique wouldn't even make sense unless the object is hidden a lot of the time.
• If the object is hidden, X-Plane will still run through every command in the object, simply skipping drawing. So if the object is heavily animated, we still pay a lot of CPU for the "hidden" object.

You will actually save some performance with ATTR_hide in one situation: if the geometry that is not drawn with ATTR_hide covers a huge amount of the screen (e.g you are really close to the object) then not drawing the pixels saves frame-rate. This would be extremely unlikely - you'd have to hide an entire cockpit from the inside to see any kind of benefit with this.

Why doesn't ANIM_hide actually skip work? Well, it may someday, but for now it's a question of how we optimize objects. When we load objects, we evaluate their command sequence and attempt to consolidate and remove unnecessary CPU work. We remove unnecessary state change, combine drawing commands when possible, etc. This works because we know at object-load time exactly how the object will be drawn. But consider this bit of OBJ "code":

ATTR_flat
ANIM_begin
ANIM_hide
ATTR_smooth
TRIS 0 30
ANIM_end
TRIS 30 60
ATTR_smooth
TRIS 90 120

If ANIM_hide really skipped work, is the ATTR_smooth routine necessary? Can the last two TRIS commands be combined into one big TRIS command? If ANIM_hide removes an attribute, we don't know until we actually draw the object what attributes will be run. But if we say ANIM_hide doesn't affect state change (which requires us to actually do the state change) we can then realize that none of the state change in this object are actually necessary, saving 3 attributes, and allowing us to combine a triangle batch.

So for now our policy is one of simplicity:

• ANIM_hide is for artistic purposes, but not optimization purposes. It simply stops drawing, it doesn't eliminate work.
• The optimizer then goes to work on your object.
• I think we will someday provide a new mechanism for the "landing gear case", one that's specifically designed to quickly eliminate parts of a plane that are expensive and don't need to be drawn.
  

* X-Plane does try to optimize out this cost in some cases. For the purpose of drawing, an object can get "culled" (not drawn at all) by being too far away for LOD or fully off-screen, or it can get "drawn" (meaning we do look inside the commands). Culling is very very fast, and drawing is usually quite slow. The optimization of avoiding using geometry and textures always work in the culling case, but usually do not work in the drawing case. So having a smart LOD on the landing gear is a huge win.

New SDK coming...

Read the info here:

http://www.xsquawkbox.net/xpsdk/phpwiki/index.php?NewVersionAnnouncement

Please note that if you are a scenery author and not a programmer, this will not directly affect you at all. (But see the section on compatibility - old plugins will still work!)

Version 9 will also feature a lot of new datarefs - this should make things a lot easier for authors doing animation.

The new API will allow programmers to access the terrain mesh. This means that add-ons that want to add dynamic graphics content by drawing (e.g. drawing in baggage trucks) will be able to place that content "on the ground" no matter what the surface of the airport area. I think this will allow for some interesting new add-ons.

Add-on Forests...

It looks like Andras and Albert just released new forests...their work looks really good and makes me go "is that really X-Plane"? Definitely a must-have add-on!

I would also suggest that anyone who wants to know why X is wrong with the global scenery read some of Andras' posts on the X-Plane tech list...a few weeks ago he described in a lot of detail the kinds of problems that both he and I face when working with global datasets - his comments are right on the money!

I think we'll have more announcements regarding forests relatively soon. Things have been quiet because behind the scenes we're in a big push.

The problem with KLGA

With X-Plane 820 global scenery, you can fly with "sloped runways" (this option drapes the runways like a piece of cloth over the terrain no matter how strange the terrain is) at most locations, but not La Guardia, where there is a huge hump in the runway.

It turns out the problem with KLGA is that there are vertices in the mesh that are adjacent to both water and the airport. The field height is a few meters above sea level, so we cannot decide what height those vertices should be? (Zero for the water, or a few meters for the airport?) In the 820 global scenery, the water wins, pullling down any part of the airport that is coastal.

For the version 9 re-render of the global scenery I am trying to fix this by creating a buffer zone of a few meters around coastal parts of the airport. This assures that we will have at least TWO vertices at the corners of the airport. The inner one can then be at field level and the outer one at sea level. The triangles in between these two "rings" form a sloped transition area (more sloped if the airport is far above water level), allowing the inner area to remain flat.

We'll have to see how much this helps coastal airports - in hindsight I'm surprised that a lot more airports weren't as unmanageable as KLGA.

(As a side note, it appears that the SRTM raw elevation data is at its least reliable in airports - there's something about them, perhaps the metal or concrete or heat that causes a ton of drop-outs and false reflections. So we have to pre-process the airport areas pretty heavily to use them at all - and sometimes this pre-processing doesn't even work)

Taxiway Shapes

For this blog entry I will try to show how to put together taxiway segments to avoid WED problems.



Here we have a 3-way junction, that is, one taxiway ending with another. First, what not to do:



First we have an overlapped solution. This is not good because overlap is wasteful of graphics card power. We also have the problem that the edge markings will overlap. So in the second and third picture, we "cut away" the horizontal taxiway markings by adding vertices. (In picture two we have overlap, in picture 3 we try to end the taxiway right at the intersection.)

The problem with technique two is: we still have overlap (and imprecise markings). The problem with picure three is that we can't guarantee that the vertices of the vertical taxiway are exactly "on top of" the horizontal taxiway edge. So you may get something in X-Plane that looks more like this:



(Gap exagerated for clarity.) This is a "slivering" problem. The solution is one that is both efficient and nice looking:



In this situation, I have inserted two vertices in the edge of the horizontal taxiway and then used "snap to vertex" to assure that the corners of the vertical taxiway are exactly the same as the "cut points" in the horizontal one. This gives me a connection that will look clean in x-plane with no cracks or slivering and no overlap. My lines also end at just the right places.

(Hint: once you do this, to move all of the overlapped vertices at once, select them all with the marquee tool and drag.)

Another issue I see sometimes with WED layouts is how to draw the "fillets", or curved taxiways that touch runways. Because the lines are not as smooth in x-plane as WED, often users are surprised when the approximations of the bezier curves cause an overlap and theere is no pavement.

Note that we're stuck - we can't use the technique described above because we can't insert vertices into the runway. So we have to overlap. Less overlap is better - the overlap here is exaggerated to make things clear.



In this curve, we have a control handle only on the bottom vertex. It will overlap as soon as the control handle reaches the other side of the top line. By pulling the control handle down, we restore the curve.



Here we have two control handles - now it is a question of whether the top control handle crosses the top of the taxiway.



Here is one solution - at the expense of more overlap we can simply give ourselves some breathing room with our bezier curve. However, my preference would be to try to keep the control handles below the taxiway top by a wide margin.

Optimal Performance for apt.dat Layout

I don't actually know all of the best ways to make an apt.dat file. The format is new and we haven't had a ton of time to performance test it. But here are some thoughts:

• One thing I know is...it is best to describe the shape you are using with the fewest number of nodes possible. In other words, let bezier curves do their thing, don't add a ton of vertices to them. And please don't add vertices to try to make the curves smoother. The best thing is to leave the layout simple and let X-Plane render it. In the future we will allow the user to set the level of smoothness based on his or her hardware. Adding more nodes will hurt quality - we will be able to pick better nodes at runtime than you can in the apt.dat file.
• Avoid overlapping large areas of pavement. Overlap is bad - it hurts framerate by requiring the graphics card to draw the same pixels over and over. As X-Plane supports more advanced shaders for nicer lighting, this cost will be a lot heavier on users.
• Ignoring overlap, the question of whether to have several smaller chunks of pavement vs. one large one is a difficult question. I would suggest moderation - try not to make too many individual pavement elements, but don't make the entire element one giant pavement either. You may have to experiment to find the best framerate.

How Flat is Flat?

I posted previously in my blog that we use airport boundaries to flatten airports when we make the DSFs, as well as place airport grass. Let me clarify this a bit: there are two stages of flattening that we do.

• The flattening in the DSF is a "soft" flattening - we do not completely flatten the area, but rather we try to remove high frequency spikes in the elevation data, so that the airport maintains its overall shape without being unusable. This flattening process is very precise in that it only happens inside the airport area. Because we decide the mesh when making the DSF, we can shape the mesh to flatten only the airport area.
• If flattening in the sim is enabled, we then do a "hard" flatten of an area including the airport and some surrounding area. This flatten is truly flat and destroys any topography for the airport and a lot of surrounding terrain. Because the mesh is already defined, X-Plane is limited in its choices of where it can choose to flatten or not flatten.

Now that first soft flattening is what we use the airport boundary for. In X-Plane 800 the US DSF flattening worked very poorly. For the 820 flattening it worked well for most cases, but had some very specific bugs. (Typically the confluence of water, airport, and/or a DSF tile in close proximity would cause the three different processors for water, airport, and boundaries to come into conflict.)

I am going to try to further improve the flattening algorithm for the v9 render, with the hope of fixing some of the buggy cases from the 820 render. I am sure there will still be some broken cases - with 14,000+ DSF tiles, one of them is bound to be weird...but my hope is that with each re-render we can get closer to a render where users can run with sloped runways enabled and enjoy realistic non-flat runways.

Airport Layouts and Global Scenery

When we make the global scenery, we pre-process the terrain to make flat grassy areas for the airports, using the latest apt.dat data. We will be recutting the scenery again soon for version 9, so if you have a favorite airport that isn't flat and grassy in v8, you can submit a layout to Robin Peel - notes here.

I do not know exactly when we will be rendering the scenery, so I don't know how long you have to submit a layout. I can't make a promise that "if you submit a layout by X it will be in the scenery" - something can always come up that requires us to not use the layout (especially now that we have apt.dat 850 and layouts can contain all sorts of strange things). So I can only say: please consider submitting your layout soon. If it's only partly done, finish the boundary and submit it to Robin with a note "this is only for global scenery" - you can always finish it and submit again later.

Draped Roads - A Can of Worms

I don't like it when overlay scenery depends on the underlying base mesh...I've been telling authors for a while now on the lists, "don't assume the base mesh is the global scenery", and while this actually has been a safe assumption for version 8, I suspect that once we get v9 out the door I'll have some time to write mesh-editing tools, at which point the user will have a choice of base meshes.

Roads are currently built in MSL space (that is, absolute elevation on the planet) and thus don't change when the base mesh changes. In a previous post I discussed the idea of having them be AGL based (draped) so that they would "hug" whatever base mesh you have.

First I should say: this isn't coming any time soon and won't be in 9.0. But also, upon closer examination, there are a number of ugly issues just below the surface. Consider all of the things that the global-scenery generator does with roads:

• Road type is selected not only from source road data, but from surrounding conditions, some of which are not obviously determined by the final DSF.
• Road bridging is determined based on nearby roads, water, gradient constraints, and available bridge templates.
• Road type is further influenced by bridging considerations.
• Roads are smoothed to try to build the most even access, "hiding" rapid changes in underlying terrain from a high-speed roadway.
• Powerline tower locations are picked from terrain peak to peak with constrained maximum lengths. (The individual segments also controls the span of wires, so introducing new nodes in the draping process might be undesirable.)
  

The problem therefore with a draped road is that the decision to bridge (both how high and what kind) are dependent on the underlying terrain.

The eventual solution would be to do all of the road processing in X-Plane while draping - but this implies that the author of a custom road grid doesn't actually have control over how the roads come out. One advantage of the current system is that total control over the roads is provided in the DSF - given appropriate tools, a modeler could very carefully hand-craft the roads with a lot of detail.

Key Frames in Version 9 (and datarefs)

X-Plane 9 will feature key-framed animation for the rotation and translation animations (but not the show-hide animations). Basically:

• You can have more than two key frames.
• Interpolation between them is linear.
• Extrapolation off the ends is linear (use two equal-value key frames to clamp).

Key frames will be available for all OBJs: scenery, cockpit and aircraft parts. I will try to post a spec soon - the implementation is very similar to the RFC I posted a while ago.

(If you read the .tcl code in the AC3D plugin, you may have guessed that this was coming.)

Trigger

Now I know somebody is going to ask me: "will this allow me to make an animation X start when event Y happens in the sim?" The answer is: no! Key frames allow you to map the relationship between an input dataref and your geometry in any way you want. But if the input dataref is not changing, the output position cannot change either.

Example: you can use key frames to make complex things happen as the gear deploys. Gear deployment is a ratio that starts at 0 and slowly changes to 1 as the gear deploys (over the cycle time as set in PlaneMaker). So you can use this to make the wheels move forward first, then down.

But you cannot use key frames in the cockpit to make the gear handle move slowly. The gear handle dataref is really a switch - it instantly changes from 0 to 1. If you set up a nice key framed animation, we will instantly jump from the animation position at 0 to the animation position at 1.

In other words, whether or not you have key frames, the timing of an animation comes from datarefs, and datarefs alone. I am not planning any extensions to the OBJ system any time soon that will let you trigger animations from events. In other words, you can't synthesize timed sequences from an OBJ. (The purpose of OBJs is to visualize things, not model things.)

If you want to create timing data, write a plugin. If you don't know how to code, I have seen at least one plugin that lets authors "invent" simple datarefs using a text file.

No Spare Datarefs

And while I'm going off on a tangent about authoring and datarefs, we will not be adding any "spare" datarefs. Again, datarefs should be made with plugins, and non-programmers should use a plugin that creates datarefs from text files. We will not be creating any "unused" datarefs in the sim.

Why not? Think of the integration problems. Even if we make 100, what happens if Joe uses "spare dataref 93" in the cockpit of his great 747 and Fred uses "spare dataref 93" for the windmills turning in his scenery pack? Splat.

The naming convention of datarefs start with the company or author who created it - or "sim" for X-Plane itself. This is intentional - it assures that two independent authors won't use the same dataref name and create conflicts between add-ons!

862RC1 - FPS back, maybe

X-Plane 862 RC1 is out. Just to clear up confusion, this is the next RC after 861RC1 and is just more bug fixes (all bug fixes from 861RC1 plus a few more).

There are two things hopefully working in 862:

1. I think we got the optimization settings right - framerates on Windows should match 860. If they don't, please let me know.
2. There is another object-placement bug fix in 862. If you're still seeing problems with object placement depending on which direction you approach a scenery area from, please let me know.
   

Roads: State of the Union

When I posted about roads a few days ago, I didn't realize that people had such, um, "strong" feelings about them. At risk of throwing fuel on the fire, here's the overall state of the road system in X-Plane:

1. Architecture

For all of the problems with the road system, I think the architecture makes sense: DSFs contain directed networks of roads, graphic resources describe how to draw them as scenery, and cars are generated on that road network dynamically. I don't see the system fundamentally changing - rather, we'll have extensions to the rendering definitions, more advanced car behavior, and other additions within the existing framework.

(I think this is true for most v8 scenery technology - we have a lot of room to grow without a huge rewrite.)

2. Default Road Data

This is the weakest part of the road system - the data in Europe comes from horribly old sources (VMAP0) and even the US data (from TIGER) doesn't contain accurate enough flow information to really draw a nice detailed road grid. I don't see this getting a lot better any time soon...but I think there are three ways this could get better:

• We'll keep improving how we process the data in the default scenery. (But to some extent, junk data going in is going to produce poor results no matter what.)
• If we can ship overlay roads in v9, users will be able to customize road data. This would be the source of the best road data possible, at least for the relatively small* area where users customize roads.
• Some one (us or a third party) could build an add-on pack that uses commercial navigation-quality roads. I do not think we will ever have this in the sim itself - the pricing of commercial data makes this impossible. (This is not about LR being cheap - this is about not losing money on every sale of X-Plane.)

3. Road Rendering

I believe that while road rendering leaves a lot to be improved right now, it is appropriate to the level of data we have. That is, we don't have nicely rendered intersections, but with such weird intersection data in the global scenery, even nicely rendered intersections would look absurd.

Once we have better road data (via better generation, or more likely, user-modified data) we'll incrementally provide improvements in the way roads look, probably including extensions to the .net format that defines their look.

4. Car Brains

Right now the AI for the generated traffic is very simple, and the way the cars drive is crude. But here again I think we need to crawl before we can walk - without nicely rendered intersections (from well-formed data), there's not a lot of point in working on this.

Summary

The road system will grow incrementally, first by allowing users to insert customized data, then by providing tools to make that data, and then by improving the way the sim draws and drives on this data. It will be an incremental process, and when it comes to roads, version 9.0 will have some evolutionary steps but nothing revolutionary.

* I think in terms of the whole world (14,000 DSFs), so user-contributed data represents a small fraction of the total scenery system. On the other hand, human beings live in clustered areas, so the numerically small number of contributions we receive will have an outsized benefit by being in "important" areas.

Future Road Features

Disclaimer: nothing you read in this blog is even remotely like a "promise" about what future X-Plane patches or releases will do. This blog contains my ramblings, not a commitment to future features.

I am looking at allowing road networks in overlay DSFs for X-Plane 9. I'm not sure if this will be possible (this is just an investigation right now). But I realize that...

• The road data in the default scenery isn't much good. Even in the US, where we have nice data, the roads are chunky, not smoothed, contain invalid flow information, and can have lateral error of up to 200 meters (enough to cause chaos). Of course the non-US road data comes from VMAP0 and is just as good as you'd expect from digitized 1:1,000,000 maps from WWII (which is to say, they're awful.)
• Therefore there is a lot of interest in replacing the road grid using overlays. This isn't possible in X-Plane 8 (put a road into an overlay and I think X-Plane will crash pretty hard). If roads could be in overlays, this would let you correct them and exclude the entire underlying layer.

There's another hitch to this...for historical reasons, the road system uses MSL heights (that is, it exists in absolute space and is pre-placed to "hug" the road grid). At the time, when all scenery load involved a giant pause and y-testing was expensive and the entire road grid was built at once, this helped performance. A number of those fundamental problems have been solved.

But if a road grid is pre-placed (and not "draped" like a .pol file or OBJ) then the roads must be built to a specific set of global scenery mesh. I'd like to see custom meshes start to emerge, and hopefully I'll have some time in the future to work on tools to make this a lot easier.

So the feature that would go hand-in-hand with overlay roads is AGL-based road heights (that is, the roads only specify when there is a bridge and not exactly how to hug the ground). As far as I can tell from my initial investigation, this is quite a bit more complicated to implement and is probably still a while off. I think that overlay roads could be useful even in their MSL form though.

(One interesting use would be: create roads with no geometry, just car definitions, and use them to overlay routes for ground vehicles in airports.)

X-Plane 9: The Absurdity of Pretending

There have been plenty of rumors and semi-official posts regarding the upcoming major revision of X-Plane (X-Plane 9). I have been trying to keep my mouth shut about it...the problem with pre-announcing anything is that the upside to us is small (at best we do what we said) and the downside is large (at worst we don't do what we said and people get grumpy). No one complains if XP9 turns out to have no-pause scenery load and it's a surprise...but plenty of people complain if we say "there won't be pauses" and then they are.

But...the situation is becoming mildly absurd...plenty of info is out there, and saying "the upcoming major release", etc. just feels political and weaselly. Austin would be disgusted.

So listen: I am going to try to provide some info on X-Plane 9. This info is subject to change. This is what we think is going to happen to the best of our knowledge. The release is still a ways away and enormous changes will happen. When things change, do not bitch to me that "you promised X" would happen. I do not promise anything. This info is provided to try to help those making add-ons for X-Plane plan appropriately.

With that in mind...I will try to post some more details on the authoring environment in the next few days. For now, here's some very basic guidance on compatibility and hardware requirements:

• The hardware requirements will be at least as high as X-Plane 8. If your machine is gasping and wheezing on 8, it's not going to be any better on 9.
• X-Plane 9 will depend more heavily on pixel shaders. If your hardware doesn't have pixel shaders (GeForce 2-4, Radeon 7000-9200) or has really crappy shaders (GeForce FX series) you will miss out on a lot of the cool stuff in v9, and possibly have the scenery look worse (but faster) than v8 (as we move features from the CPU to pixel shaders).
• Scenery that opens in x-plane should open in X-Plane 9 unmodified - if the scenery works in 8 but not 9, report it as a bug!
• Plugins that work in v8 should work in v9 without modification.

Finally, we are trying to finish up X-Plane 861...this is a bug-fix patch for version 8 - it contains no new features, but it does fix a few nasty bugs, some of which cause crashes. So if there is any new feature, it's coming in 9, not 861. Version 8 has been out for a very long time, so I will accept no argument that v8 should have more features than it does now. It's been a long run!

(One of my main goals with 861 is to try to fix any weird behavior for third party scenery add-ons, so that a scenery add-on looks the same in v8 and v9. If we left the bug in 861 and fixed it in v9, authors would have to hack the scenery to make it work with v8, and then remove the hack and republish for v9. By trying to fix the authoring bugs in v8, at least when possible, it lets authors publish one package for both versions. Of course, v9 will have new features, so I expect some v9-only scenery will emerge pretty quickly.)

861 seem slow? Here's what I need to know!

I've heard a few reports that 861 is slower than 860. I think this is rather unlikely given the code changes made in the 861 patch, but I take all performance issues very seriously!!

I can't get X-Plane 861 to run slower than 860, so here's the data I need:

• If you are seeing this, start by getting a simple case, e.g. sitting on the runway staying still, with a known viewpoint. The conditions have to be reproducible and stable -- "overall the sim is slower" isn't quantitative, it's an impression. Ranges of fps aren't precise enough.
  
• If you are using third party add-ons, strip 'em out. Does the fps problem go away? If not, good, let's get a case that uses only the simplest setup. If the fps problem does go away, let me know what add-on it is...perhaps a particular add-on exacerbates a performance problem.
• You can put the 860 x-plane app into an 861 folder - do this for all regressions to make sure you're using the same prefs file, etc. etc.
• Of course most useful is: run the fps test - if you can send me comparative fps-test logs, this is the most useful thing because it controls virtually all variables!

With that in mind, please email me log files, pref files (if you must use a specific prefs combo to see a problem), etc. Please do not just tell me descriptions - that's not precise enough!

Start With Low Settings to Maximize Framerate

In order to maximize X-Plane's framerate, you need to start with the lowest settings and work your way up. Can you start with the highest settings and work your way down? No! Here's the problem:

X-Plane's framerate is determined by the slowest part of the system...whether it's your CPU, AGP bus, or any of the parts of your graphics card (pixel shaders, lack of VRAM, lack of internal card memory bandwidth). Whatever the problem is, the graphics are drawn in an assembly line, and the rest of the hardware sits bored while the slowest part finishes its backlog.

So...why not start with the highest settings and turn them down? The answer is that when the framerate is slowed down by component A, changes in the load on component B won't show as a difference of framerate, because component B isn't the slowest. So as you turn down settings, you're not seeing the real effect. (The safe thing to do would be: turn B back to its old setting, then retry.)

What I recommend is: start on the lowest settings possible - you'll see a framerate that's about as high as your system can go (limited by the FM and the speed the card can turn a frame around). Then turn settings up, one at a time, by one notch. Each time you see a fps hit, back off one setting and continue somewhere else. This way you won't load up one part of the system so heavily that you are limiting the speed despite everything else.

861 Fixes

A few scenery bugs fixed in 861:

• Some apt.dat layouts can hand 860 - I think this is fixed in 861. If your layout caused problems, retry 861RC1 first!
  
• Objects won't change their position as you approach an airport from different sides. If you were seeing inconsistent object positioning, definitely try this release.

ATTR_no_depth deprecation

I am going to take Marginal's suggestion: in the future, ATTR_no_depth will be mapped to ATTR_poly_os 1, and ATTR_depth will be mapped to ATTR_poly_os 0. As far as I can tell, historically the ability to turn off all depth buffering was a misguided attempt to do the kind of things that ATTR_poly_os is meant for.

This implementation will hopefully help any content that is (for some reason) still using ATTR_depth and ATTR_no_depth...modern OBJ generators like the Blender plugin and the AC3D plugin never use this old attribute.

I wish WED looked like this

I get a fair number of emails where users send me a link and say "can you make X-Plane/WED look like this?" I'll beat you to the punch this week...I wish WED looked like this.

Unfortunately it probably never will for three reasons:

1. X-Plane's time to create the structures you see is slow enough that editing the real world in real-time would be difficult. We try to load things on another thread while you fly to help smooth this.
2. Our scenery creators do a lot of work to build the DSFs before you fly - minutes per DSF...again, from the time the raw data changes, a lot of computation happens.
3. We're trying to keep our tool set open source, but the X-Plane rendering code is closed source, so it would have to be rewritten (a huge task), e.g. WED would need an alternate editing engine.

(Of course if it was rewritten, it could be done so to fix points 1 & 2.)

Still, we must all dream. :-) :-)

Flat Shading is Evil (and other OBJ sins)

Almost two years ago, I posted this blog entry, pointing out that some legacy OBJ commands are, well, evil.

My inclusion of ATTR_poly_os in that blog post was a little strange - when used correctly, ATTR_poly_os is the right way to overlay decals on the ground, and it is fully supported. (The message is just, I suppose, that if you need a huge number for your poly_os, something is really wrong.)

Now ignoring ATTR_poly_os, you might notice that the AC3D plugin exports almost none of those dubious attributes. But what about flat shading? Flat shading isn't necessary, but you were allowed to use it.

So I came to my senses and modified the AC3D exporter - the exporter will now never use ATTR_shade_flat - instead it uses smooth shading and per-vertex normals, avoiding an unnecessary attribute no matter what you do in AC3D. 99.99% of the time this will yield fastest framerate?

Why wasn't the plugin always like this? I thought it was, and only discovered today that I hadn't finished this optimization.

RFC: Airport Vehicle System

I am interested in proposals for an airport-vehicle system in X-Plane. I will explain exactly what I am looking for, but first I must note the oddity of this request:

99.99% of the time, users give us too many implementation details on a feature request. Usually we want to know what you want, and not how you want it done. Example:

Bad: please implement VBOs for the 3-d clouds.
Good: please improve frame-rates when 3-d puff clouds are used.

If you ask for something too specific, we can't tell what you really want - there might be a better implementation.

So normally the feature request is what's useful, not the implementation.

In this case, however, I am interested in a proposed system for third party control of airport vehicles. If you have an interest in this, please write up a proposal and email me. Forgive me bashing you over the head with this, but often I get flooded with email on this.

• Please do not email me on this unless you have a full proposal for a system by which authors can control ground vehicles. Please do not email me asking for other features, telling me which type of truck is your favorite, or asking for user-level features. My goal here is to get feedback on how third parties would like to be able to customize a ground vehicle system. If you don't understand the difference, please don't email on this subject.
• I cannot promise that I will do anything you say, in particular, only that I will read and carefully consider anything that's sent to me that makes sense.
  
• and I cannot say when this will be used (not that soon though -- if we're only in the RFC stage we're only beginning to look at this). This is not a feature announcement.

I will try to respond to proposals quickly, but I've been pretty badly swamped by email, so it could take a few weeks!

X-Plane Dark Arts: Airplane Object Translucency II

In my previous post I described the underlying problems that make translucency in airplane cockit objects tricky. That gives us the background to understand how X-Plane draws cockpit objects and why.

The actual draw order for airplane-related objects is this:

1. Far away weapons.
2. All of the airplanes that we are not inside (this means the user's plane in an external view, and AI planes all the time). For each of these planes, the external cockpit object is drawn first, then the parts, then the attached objects in order.
3. Clouds and puffs and other such environmental phenomena.
4. The geometry of our plane, if it is to be drawn and we are in an inside view.
5. The attached objects in order for our plane, if internal geometry is drawn and we are in an inside view.
   
6. Weapons that are very close to the plane.
7. The inside cockpit object, if we are inside the plane.

A few things to note about this draw order:

• Note that weapons appear twice in the category of "near" and "far". This is all about the clipping plane - if a weapon is close to us, it must be drawn with the cockpit object, late in the draw cycle, when we are doing "close" things. Weapons are treated dynamically - they change where they are in the draw cycle depending on their position in space. This is necessary because a weapon starts out real close (just off your wing) and then goes real far away.
• The user's airplane is special-cased when we are in an internal view ... at that point if external things are being drawn, they are done in the later "close" view with the cockpit.
• The external cockpit object is (strangely) quite early in the draw order. There's really no good reason for that. What's particularly annoying is that it's inconsistent with the internal draw order. The main point of internal/external cockpit objects is to let you simplify your cockpit object in external view for performance.

Is there a sane way to set up an X-Plane 860 aircraft with translucent geometry? I'm not 100% sure about this, but it looks to me from the code like the correct thing to do might be:

1. Put any translucent windshield/canopy/etc. in the last attached object.
2. Put only the interior panel (and not the canopy) in the cockpit object.

This works because in the external view the canopy is drawn last, and in the internal view, the panel is more or less guaranteed to be closer than the canopy, which is good enough.