Optimal Performance for apt.dat Layout

I don't actually know all of the best ways to make an apt.dat file. The format is new and we haven't had a ton of time to performance test it. But here are some thoughts:

• One thing I know is...it is best to describe the shape you are using with the fewest number of nodes possible. In other words, let bezier curves do their thing, don't add a ton of vertices to them. And please don't add vertices to try to make the curves smoother. The best thing is to leave the layout simple and let X-Plane render it. In the future we will allow the user to set the level of smoothness based on his or her hardware. Adding more nodes will hurt quality - we will be able to pick better nodes at runtime than you can in the apt.dat file.
• Avoid overlapping large areas of pavement. Overlap is bad - it hurts framerate by requiring the graphics card to draw the same pixels over and over. As X-Plane supports more advanced shaders for nicer lighting, this cost will be a lot heavier on users.
• Ignoring overlap, the question of whether to have several smaller chunks of pavement vs. one large one is a difficult question. I would suggest moderation - try not to make too many individual pavement elements, but don't make the entire element one giant pavement either. You may have to experiment to find the best framerate.

How Flat is Flat?

I posted previously in my blog that we use airport boundaries to flatten airports when we make the DSFs, as well as place airport grass. Let me clarify this a bit: there are two stages of flattening that we do.

• The flattening in the DSF is a "soft" flattening - we do not completely flatten the area, but rather we try to remove high frequency spikes in the elevation data, so that the airport maintains its overall shape without being unusable. This flattening process is very precise in that it only happens inside the airport area. Because we decide the mesh when making the DSF, we can shape the mesh to flatten only the airport area.
• If flattening in the sim is enabled, we then do a "hard" flatten of an area including the airport and some surrounding area. This flatten is truly flat and destroys any topography for the airport and a lot of surrounding terrain. Because the mesh is already defined, X-Plane is limited in its choices of where it can choose to flatten or not flatten.

Now that first soft flattening is what we use the airport boundary for. In X-Plane 800 the US DSF flattening worked very poorly. For the 820 flattening it worked well for most cases, but had some very specific bugs. (Typically the confluence of water, airport, and/or a DSF tile in close proximity would cause the three different processors for water, airport, and boundaries to come into conflict.)

I am going to try to further improve the flattening algorithm for the v9 render, with the hope of fixing some of the buggy cases from the 820 render. I am sure there will still be some broken cases - with 14,000+ DSF tiles, one of them is bound to be weird...but my hope is that with each re-render we can get closer to a render where users can run with sloped runways enabled and enjoy realistic non-flat runways.

Airport Layouts and Global Scenery

When we make the global scenery, we pre-process the terrain to make flat grassy areas for the airports, using the latest apt.dat data. We will be recutting the scenery again soon for version 9, so if you have a favorite airport that isn't flat and grassy in v8, you can submit a layout to Robin Peel - notes here.

I do not know exactly when we will be rendering the scenery, so I don't know how long you have to submit a layout. I can't make a promise that "if you submit a layout by X it will be in the scenery" - something can always come up that requires us to not use the layout (especially now that we have apt.dat 850 and layouts can contain all sorts of strange things). So I can only say: please consider submitting your layout soon. If it's only partly done, finish the boundary and submit it to Robin with a note "this is only for global scenery" - you can always finish it and submit again later.

Draped Roads - A Can of Worms

I don't like it when overlay scenery depends on the underlying base mesh...I've been telling authors for a while now on the lists, "don't assume the base mesh is the global scenery", and while this actually has been a safe assumption for version 8, I suspect that once we get v9 out the door I'll have some time to write mesh-editing tools, at which point the user will have a choice of base meshes.

Roads are currently built in MSL space (that is, absolute elevation on the planet) and thus don't change when the base mesh changes. In a previous post I discussed the idea of having them be AGL based (draped) so that they would "hug" whatever base mesh you have.

First I should say: this isn't coming any time soon and won't be in 9.0. But also, upon closer examination, there are a number of ugly issues just below the surface. Consider all of the things that the global-scenery generator does with roads:

• Road type is selected not only from source road data, but from surrounding conditions, some of which are not obviously determined by the final DSF.
• Road bridging is determined based on nearby roads, water, gradient constraints, and available bridge templates.
• Road type is further influenced by bridging considerations.
• Roads are smoothed to try to build the most even access, "hiding" rapid changes in underlying terrain from a high-speed roadway.
• Powerline tower locations are picked from terrain peak to peak with constrained maximum lengths. (The individual segments also controls the span of wires, so introducing new nodes in the draping process might be undesirable.)
  

The problem therefore with a draped road is that the decision to bridge (both how high and what kind) are dependent on the underlying terrain.

The eventual solution would be to do all of the road processing in X-Plane while draping - but this implies that the author of a custom road grid doesn't actually have control over how the roads come out. One advantage of the current system is that total control over the roads is provided in the DSF - given appropriate tools, a modeler could very carefully hand-craft the roads with a lot of detail.

Key Frames in Version 9 (and datarefs)

X-Plane 9 will feature key-framed animation for the rotation and translation animations (but not the show-hide animations). Basically:

• You can have more than two key frames.
• Interpolation between them is linear.
• Extrapolation off the ends is linear (use two equal-value key frames to clamp).

Key frames will be available for all OBJs: scenery, cockpit and aircraft parts. I will try to post a spec soon - the implementation is very similar to the RFC I posted a while ago.

(If you read the .tcl code in the AC3D plugin, you may have guessed that this was coming.)

Trigger

Now I know somebody is going to ask me: "will this allow me to make an animation X start when event Y happens in the sim?" The answer is: no! Key frames allow you to map the relationship between an input dataref and your geometry in any way you want. But if the input dataref is not changing, the output position cannot change either.

Example: you can use key frames to make complex things happen as the gear deploys. Gear deployment is a ratio that starts at 0 and slowly changes to 1 as the gear deploys (over the cycle time as set in PlaneMaker). So you can use this to make the wheels move forward first, then down.

But you cannot use key frames in the cockpit to make the gear handle move slowly. The gear handle dataref is really a switch - it instantly changes from 0 to 1. If you set up a nice key framed animation, we will instantly jump from the animation position at 0 to the animation position at 1.

In other words, whether or not you have key frames, the timing of an animation comes from datarefs, and datarefs alone. I am not planning any extensions to the OBJ system any time soon that will let you trigger animations from events. In other words, you can't synthesize timed sequences from an OBJ. (The purpose of OBJs is to visualize things, not model things.)

If you want to create timing data, write a plugin. If you don't know how to code, I have seen at least one plugin that lets authors "invent" simple datarefs using a text file.

No Spare Datarefs

And while I'm going off on a tangent about authoring and datarefs, we will not be adding any "spare" datarefs. Again, datarefs should be made with plugins, and non-programmers should use a plugin that creates datarefs from text files. We will not be creating any "unused" datarefs in the sim.

Why not? Think of the integration problems. Even if we make 100, what happens if Joe uses "spare dataref 93" in the cockpit of his great 747 and Fred uses "spare dataref 93" for the windmills turning in his scenery pack? Splat.

The naming convention of datarefs start with the company or author who created it - or "sim" for X-Plane itself. This is intentional - it assures that two independent authors won't use the same dataref name and create conflicts between add-ons!

862RC1 - FPS back, maybe

X-Plane 862 RC1 is out. Just to clear up confusion, this is the next RC after 861RC1 and is just more bug fixes (all bug fixes from 861RC1 plus a few more).

There are two things hopefully working in 862:

1. I think we got the optimization settings right - framerates on Windows should match 860. If they don't, please let me know.
2. There is another object-placement bug fix in 862. If you're still seeing problems with object placement depending on which direction you approach a scenery area from, please let me know.
   

Roads: State of the Union

When I posted about roads a few days ago, I didn't realize that people had such, um, "strong" feelings about them. At risk of throwing fuel on the fire, here's the overall state of the road system in X-Plane:

1. Architecture

For all of the problems with the road system, I think the architecture makes sense: DSFs contain directed networks of roads, graphic resources describe how to draw them as scenery, and cars are generated on that road network dynamically. I don't see the system fundamentally changing - rather, we'll have extensions to the rendering definitions, more advanced car behavior, and other additions within the existing framework.

(I think this is true for most v8 scenery technology - we have a lot of room to grow without a huge rewrite.)

2. Default Road Data

This is the weakest part of the road system - the data in Europe comes from horribly old sources (VMAP0) and even the US data (from TIGER) doesn't contain accurate enough flow information to really draw a nice detailed road grid. I don't see this getting a lot better any time soon...but I think there are three ways this could get better:

• We'll keep improving how we process the data in the default scenery. (But to some extent, junk data going in is going to produce poor results no matter what.)
• If we can ship overlay roads in v9, users will be able to customize road data. This would be the source of the best road data possible, at least for the relatively small* area where users customize roads.
• Some one (us or a third party) could build an add-on pack that uses commercial navigation-quality roads. I do not think we will ever have this in the sim itself - the pricing of commercial data makes this impossible. (This is not about LR being cheap - this is about not losing money on every sale of X-Plane.)

3. Road Rendering

I believe that while road rendering leaves a lot to be improved right now, it is appropriate to the level of data we have. That is, we don't have nicely rendered intersections, but with such weird intersection data in the global scenery, even nicely rendered intersections would look absurd.

Once we have better road data (via better generation, or more likely, user-modified data) we'll incrementally provide improvements in the way roads look, probably including extensions to the .net format that defines their look.

4. Car Brains

Right now the AI for the generated traffic is very simple, and the way the cars drive is crude. But here again I think we need to crawl before we can walk - without nicely rendered intersections (from well-formed data), there's not a lot of point in working on this.

Summary

The road system will grow incrementally, first by allowing users to insert customized data, then by providing tools to make that data, and then by improving the way the sim draws and drives on this data. It will be an incremental process, and when it comes to roads, version 9.0 will have some evolutionary steps but nothing revolutionary.

* I think in terms of the whole world (14,000 DSFs), so user-contributed data represents a small fraction of the total scenery system. On the other hand, human beings live in clustered areas, so the numerically small number of contributions we receive will have an outsized benefit by being in "important" areas.

Future Road Features

Disclaimer: nothing you read in this blog is even remotely like a "promise" about what future X-Plane patches or releases will do. This blog contains my ramblings, not a commitment to future features.

I am looking at allowing road networks in overlay DSFs for X-Plane 9. I'm not sure if this will be possible (this is just an investigation right now). But I realize that...

• The road data in the default scenery isn't much good. Even in the US, where we have nice data, the roads are chunky, not smoothed, contain invalid flow information, and can have lateral error of up to 200 meters (enough to cause chaos). Of course the non-US road data comes from VMAP0 and is just as good as you'd expect from digitized 1:1,000,000 maps from WWII (which is to say, they're awful.)
• Therefore there is a lot of interest in replacing the road grid using overlays. This isn't possible in X-Plane 8 (put a road into an overlay and I think X-Plane will crash pretty hard). If roads could be in overlays, this would let you correct them and exclude the entire underlying layer.

There's another hitch to this...for historical reasons, the road system uses MSL heights (that is, it exists in absolute space and is pre-placed to "hug" the road grid). At the time, when all scenery load involved a giant pause and y-testing was expensive and the entire road grid was built at once, this helped performance. A number of those fundamental problems have been solved.

But if a road grid is pre-placed (and not "draped" like a .pol file or OBJ) then the roads must be built to a specific set of global scenery mesh. I'd like to see custom meshes start to emerge, and hopefully I'll have some time in the future to work on tools to make this a lot easier.

So the feature that would go hand-in-hand with overlay roads is AGL-based road heights (that is, the roads only specify when there is a bridge and not exactly how to hug the ground). As far as I can tell from my initial investigation, this is quite a bit more complicated to implement and is probably still a while off. I think that overlay roads could be useful even in their MSL form though.

(One interesting use would be: create roads with no geometry, just car definitions, and use them to overlay routes for ground vehicles in airports.)

X-Plane 9: The Absurdity of Pretending

There have been plenty of rumors and semi-official posts regarding the upcoming major revision of X-Plane (X-Plane 9). I have been trying to keep my mouth shut about it...the problem with pre-announcing anything is that the upside to us is small (at best we do what we said) and the downside is large (at worst we don't do what we said and people get grumpy). No one complains if XP9 turns out to have no-pause scenery load and it's a surprise...but plenty of people complain if we say "there won't be pauses" and then they are.

But...the situation is becoming mildly absurd...plenty of info is out there, and saying "the upcoming major release", etc. just feels political and weaselly. Austin would be disgusted.

So listen: I am going to try to provide some info on X-Plane 9. This info is subject to change. This is what we think is going to happen to the best of our knowledge. The release is still a ways away and enormous changes will happen. When things change, do not bitch to me that "you promised X" would happen. I do not promise anything. This info is provided to try to help those making add-ons for X-Plane plan appropriately.

With that in mind...I will try to post some more details on the authoring environment in the next few days. For now, here's some very basic guidance on compatibility and hardware requirements:

• The hardware requirements will be at least as high as X-Plane 8. If your machine is gasping and wheezing on 8, it's not going to be any better on 9.
• X-Plane 9 will depend more heavily on pixel shaders. If your hardware doesn't have pixel shaders (GeForce 2-4, Radeon 7000-9200) or has really crappy shaders (GeForce FX series) you will miss out on a lot of the cool stuff in v9, and possibly have the scenery look worse (but faster) than v8 (as we move features from the CPU to pixel shaders).
• Scenery that opens in x-plane should open in X-Plane 9 unmodified - if the scenery works in 8 but not 9, report it as a bug!
• Plugins that work in v8 should work in v9 without modification.

Finally, we are trying to finish up X-Plane 861...this is a bug-fix patch for version 8 - it contains no new features, but it does fix a few nasty bugs, some of which cause crashes. So if there is any new feature, it's coming in 9, not 861. Version 8 has been out for a very long time, so I will accept no argument that v8 should have more features than it does now. It's been a long run!

(One of my main goals with 861 is to try to fix any weird behavior for third party scenery add-ons, so that a scenery add-on looks the same in v8 and v9. If we left the bug in 861 and fixed it in v9, authors would have to hack the scenery to make it work with v8, and then remove the hack and republish for v9. By trying to fix the authoring bugs in v8, at least when possible, it lets authors publish one package for both versions. Of course, v9 will have new features, so I expect some v9-only scenery will emerge pretty quickly.)

861 seem slow? Here's what I need to know!

I've heard a few reports that 861 is slower than 860. I think this is rather unlikely given the code changes made in the 861 patch, but I take all performance issues very seriously!!

I can't get X-Plane 861 to run slower than 860, so here's the data I need:

• If you are seeing this, start by getting a simple case, e.g. sitting on the runway staying still, with a known viewpoint. The conditions have to be reproducible and stable -- "overall the sim is slower" isn't quantitative, it's an impression. Ranges of fps aren't precise enough.
  
• If you are using third party add-ons, strip 'em out. Does the fps problem go away? If not, good, let's get a case that uses only the simplest setup. If the fps problem does go away, let me know what add-on it is...perhaps a particular add-on exacerbates a performance problem.
• You can put the 860 x-plane app into an 861 folder - do this for all regressions to make sure you're using the same prefs file, etc. etc.
• Of course most useful is: run the fps test - if you can send me comparative fps-test logs, this is the most useful thing because it controls virtually all variables!

With that in mind, please email me log files, pref files (if you must use a specific prefs combo to see a problem), etc. Please do not just tell me descriptions - that's not precise enough!

Start With Low Settings to Maximize Framerate

In order to maximize X-Plane's framerate, you need to start with the lowest settings and work your way up. Can you start with the highest settings and work your way down? No! Here's the problem:

X-Plane's framerate is determined by the slowest part of the system...whether it's your CPU, AGP bus, or any of the parts of your graphics card (pixel shaders, lack of VRAM, lack of internal card memory bandwidth). Whatever the problem is, the graphics are drawn in an assembly line, and the rest of the hardware sits bored while the slowest part finishes its backlog.

So...why not start with the highest settings and turn them down? The answer is that when the framerate is slowed down by component A, changes in the load on component B won't show as a difference of framerate, because component B isn't the slowest. So as you turn down settings, you're not seeing the real effect. (The safe thing to do would be: turn B back to its old setting, then retry.)

What I recommend is: start on the lowest settings possible - you'll see a framerate that's about as high as your system can go (limited by the FM and the speed the card can turn a frame around). Then turn settings up, one at a time, by one notch. Each time you see a fps hit, back off one setting and continue somewhere else. This way you won't load up one part of the system so heavily that you are limiting the speed despite everything else.

861 Fixes

A few scenery bugs fixed in 861:

• Some apt.dat layouts can hand 860 - I think this is fixed in 861. If your layout caused problems, retry 861RC1 first!
  
• Objects won't change their position as you approach an airport from different sides. If you were seeing inconsistent object positioning, definitely try this release.

ATTR_no_depth deprecation

I am going to take Marginal's suggestion: in the future, ATTR_no_depth will be mapped to ATTR_poly_os 1, and ATTR_depth will be mapped to ATTR_poly_os 0. As far as I can tell, historically the ability to turn off all depth buffering was a misguided attempt to do the kind of things that ATTR_poly_os is meant for.

This implementation will hopefully help any content that is (for some reason) still using ATTR_depth and ATTR_no_depth...modern OBJ generators like the Blender plugin and the AC3D plugin never use this old attribute.