Fine Print: Water on the Bottom Please

Here's one of those little details that probably isn't in the scenery docs but needs to be: if you are making base DSF meshes and they use water (the special terrain type "water") it must be on the bottom of the stack of terrains - it can't be used as an overlay!

Polygons Part 5 - Orthophotos

I just finished an example scenery package that shows how to place orthophotos using .polygon files. Check out the scenery website (look under tutorials) to download the package and also read the illustrations. (This is an experiment: this is more of an annotated scenery example tha a true step-by-step tutorial.)

The basic anatomy of an orthophoto-in-a-polygon is:

1. A PNG file contains the orthophoto. (In the case of our example, ksbd_alpha.png.)
   
2. A .pol file references the PNG file and defines the physical properties and layering information. (In the case of our example, ksbd.pol.)
3. An overlay DSF references the .pol file. (In our case this is +34-118.dsf.)
4. The DSF overlay contains exactly one usage of the .pol file, with a polygon parameter of 65535 (a flag to indicate that texture coordinates are in the DSF) and each vertex of the polygon contains texture ST coordinates.

This pattern is actually the same design pattern that is used everywhere in the X-Plane scenery system:

• PNG files are never used directly by DSFs, they are always referenced by some kind of "definition" file, like an OBJ, .pol, .ter. Sometimes the definition file has a lot of info (like an OBJ, which contains a 3-d model), and sometimes they just contain some basic attributes (like a .ter or .pol file, which contain info for the physics engine).
  
• The artwork definitions are always separate from the DSF, so that they can be reused easily in a lot of DSFs.
• The DSF says where the polygon goes and the .pol file says what it looks like, just like the DSF says where an object goes and the OBJ says what it looks like.

One decision I made early on in my work on the scenery system was to stop using "magic file names". Prior to X-Plane 8, in a number of cases it was enough to add _LIT to the end of a texture file and a lighting mask would appear. In all modern file scenery format, there is a texture command (usually TEXTURE_LIT) to add this information.

Now this is probably more confusing to new users, and certainly a little bit confusing to anyone who was used to X-Plane 7. To some extent my goal is to have most scenery files be generated automatically - we're still a long way away from that, so my decisions to prefer extensibility to ease-of-use in the low level formats cause problemes for now. (But try the new AC3D X-Plane plugin version 3.0 - with UI to edit all X-Plane properties, does it matter anymore whether OBJ8 files are easy to read? We now have a real UI for editing X-Plane specific properties.)

In the case of file formats my concern was extensibility. There are only so many "tricks" that we can possibly cram into a file name, and the more we try to make the file name do, the less simple it becomes. When textures can only be "lit", tagging _LIT on the end is simple, and haivng to add TEXTURE_LIT can be annoying. But since then we've also added layering information and physics information. Imagine ksbd_LIT_AIRPORTS_-1_GRASS.png. Now we're getting into the domain of confusing file names.

Take a look here at some of the things we'd like to someday do for textures...seasonal textures would really make the texture names crazy. (MSFS9 accomplishes this using four seasons, which keeps the names simple...but what if we want to customize the time range for seasons?)
And dataref-controlled textures? Well, you can't encode a dataref in a file name - the / is reserved!

So all of these text files (.OBJ, .pol, .for, .ter, etc.) provide extensibility - they let us add new features to the way that DSF content is viewed without requiring cahnges to the DSF file format itself.

Since this blog post has gone off into a philosophical rant and the real info is in the tutorial, let me drift totally off subject by mentioning that while you're looking at tutorials, take a look at Kriss's tutorial on OBJ8 animation using the new AC3D plugin. If you've tried using the old system of animation (by typing cryptic goo into the AC3D object naes) you'll find the new plugin will let you work a lot faster!

Polygons Part 4 - Anatomy of a .pol file

Before I do this, an important disclaimer: we want you to use the .pol files that are found in the default scenery using the library system, not to copy the files! I encourage you to look inside the X-Plane resources folder to see how the system works, but don't copy a file if you can use it via the library; the library system saves loading time and VRAM.*

With that in mind, here's a line-by-line of what we might find in a .pol file. This one came from my X-System folder in Resources/default scenery/820 world terrain/pol/apt_desert_hot.pol. We have:

A
850
DRAPED_POLYGON

This is a standard header for a .pol file, which defines the look of a draped polygon.

TEXTURE ../textures/apt/apt_desert.png

This defines what texture the polygon will be drawn with. In this case, we are using our apt_desert.png file, found elsewhere in our package. (The double periods, .. move us up to a higher level directory. This is legal within a single scenery package.)

SCALE 500 500

This defines how large the texture appears in the X-Plane world. In this case, the horizontal span of the entire texture (before it repeats) is 500 meters (the first number), and the vertical span is also 500 meters.

NO_ALPHA

This tells X-Plane to ignore the alpha channel on the PNG file. Often PNG files used for base textures have alpha channels consisting of noise, for the purpose of creating terrain transitions. Since seeing this alpha isn't desirable in an overlay polygon, this command can be used to strip out the alpha. This lets us use one PNG file for two purposes.

SURFACE dirt

This defines the physical surface properties of the polygon. In this case, it is "dirt", which for X-Plane means some dust puffs will come off the wheels and your plane will clonk around. See the OBJ8 specification for a list of all possible surface types. (These names match the ATTR_hard command in X-Plane 8 objects.)

LAYER_GROUP airports -1

This defines where in the drawing stack to draw. In this case, we will draw one layer under airports (-1 means under, +1 would mean over). This means that this surface will be seen right below all aspects of the airport, but above regular terrain, which is what we want. The names here correspond to the same names you'd find in an OBJ8 in ATTR_layer_group - see the OBJ8 spec for the complete list of names.

* Basically any time there are two PNG files on disk, the VRAM is allocated twice, but any time one PNG file is referenced multiple times, the VRAM is shared. Since the library system lets you share other packages' resources, using it almost always means a savings in VRAM.

Polygons Part 3 - Airport Grass Terrain

In my previous post I discussed airports using the apt.dat and polygons and when to use custom artwork in a DSF.

One problem with the X-Plane scenery system (and a topic for another blog) is that, because scenery is fully pre-processed*, introducing a new airport won't put grass underneath the airport surface area. (Only airports that existed at the time the global scenery was created have grass underneath them.)

To make an airport there are two types of processing you need to do to the underlying area:

1. Flatten it, if it has bumps. Since our mesh comes from SRTM, which is "first return" (meaning whatever the radar bounces off of first, that's the height of the terrain, even if it's a tree top), it often is quite bumpy.
   
2. Change the underlying texture to something appropriate for an airport.

With X-Plane 8.60 (in RC as of this writing) you can do both! I've blogged about airport flattening (and why it isn't very good) before; today I'll try to describe how you can fix the base terrain.

You can cover the base mesh with a draped polygon in a DSF overlay. All you need to do is:

• Create a .pol file that references a PNG for the texture and describes the resolution it will be applied at (i.e. how many pixels per meter) and
• Use that .pol file in a DSF with a polygon in the overlay to describe where to put that texture down.

In X-Plane 8.60 we provide .pol files for you for all of the basic airport terrain textures. If you open the file Resources/default scenery/820 world terrain/library.txt you will find the library listing for the global scenery terrain. Toward the bottom you will see a section like this:

EXPORT lib/g8/pol/apt_sdry_cold.pol pol/apt_sdry_cold.pol

What this means is: the library path lib/g8/pol/apt_sdry_cold.pol is apped to the file pol/apt_sdry_cold.pol in this scenery package. You will see a fairly large number of .pol files on the left. Basically these represent the basic "environments" that an airport can exist in.

So to make an overlay DSF that has wet cold grass underneath it, make an overlay DSF with a .pol file using the path "lib/g8/pol/apt_wet_cold.pol".

EDIT: Important -- there are actually four .pol files in this library that do not work!! They are:

apt_desert.pol
apt_dry.pol
apt_grass.pol
apt_sdesert.pol

You will know they don't work because your scenery will not load with them. Basically, they are accidental hold-overs from a previous naming scheme for airport terrain, but were never deleted. Note how they name only an amount of rainfall (except for apt_grass, which tells us nothing). The rest of the files are named things like apt_sdry_cool, which gives us both temperature and rainfall. This 2-dimensional climate grid (with rainfall and temperature) is how most of the X-Plane terrains are organized.

Why use the library system to do this? Three reasons: copywrite and VRAM, and compatibility.

One advantage of using our library is that it gives you a way to use our artwork without copying our artwork. Remember that you cannot copy the X-Plane PNG files into a custom package and then sell it - it's a violation of X-Plane's EULA. But you don't need to! You can simply use the .pol file and X-Plane will load up the appropriate PNG files.

This is also a win for VRAM. If the wet grass texture is being used in the base terrain (for an airport that has wet grass underneath it from the global scenery) and in your overlay DSF (via the library .pol file) X-Plane will only load one copy of the texture, saving VRAM, speeding up load time, etc. If you copied the PNG file, besides being a EULA violation under some circumstances, you'd also be wasting VRAM. So the library system is more efficient!

Finally, by using our polygons, you ensure that your base grass will match the rest of the scenery system. If we update the look of our grass, because the library .pol files reference our artwork, you will see the same thing in your airport that we have in the default ones.

(Of course my warning from last week applies: if you use a wet airport grass polygon for something other than wet airport grass, then in the future our improvements on the artwork may make your scenery look strange. With these airport polygons we are aiming for "as much like an airport as possible", not "as much like 850 used to look as possible"!!)

* By this I mean, all the decisions about how scenery will look are made ahead of time, so the look of scenery does not respond to the combining of multiple packages by changing the underlying packages - only by superimposing the new ones on top. Thus a new apt.dat layout does not change the underlying terrain from urban and forest to grass. This is a design trade-off; full pre-processing has both good and bad aspects.

Warning: Orthophoto Polygons Are Not a Magic Bullet

With the new version of DSF2Text and X-Plane 8.60 it is possible to overlay orthophotos on top of DSFs. I'll get into this in more detail in a future blog post...the purpose of this post is to be a wet blanket.

Using polygons to overlay orthophotos on the base mesh might seem perfect. It's easy to do and you can clip polygons to any shape. Perfect, right?

WRONG!!!!

There are a few major problems with overlay polygons that make them well suited to some cases but not a general purpose cure-all for orthophoto-based scenery. Here's the fine print:

1. Orthophoto polygons draw over the base mesh. In particular the base mesh is still drawn underneath the orthophoto. This means that wherever you use them, X-Plane is drawing twice! This is bad for performance...it will cut the speed of mesh drawing in half. Ouch! (This is not an issue when working on very small areas like airports, but it is an issue when working on very large areas.)
2. Polygons are not built up until you fly near them. Polygons are part of the 3-d "clutter" part of the scenery not the base mesh, so we defer building them until we get near them to save memory. (This is the same as airports - look at an airport in textured map mode when far away from it -- no taxiways!) But this means that if the orthophoto is huge (and thus visible far away) its absense will be quite obvious.
3. Polygons burn CPU time when they are built up. For very small polygons this isn't that noticable, but the larger the polygon, the more work X-Plane does "draping" it over the base terrain. So on single-CPU systems with large polygon orthophotos, the framerate may be adversely impacted.

Basically overlay polygons are great for texturing small areas like airports or special landmarks within a city, but are not appropriate for making wide-scale orthophoto base meshes. For that you really need to replace the base mesh DSF!

The Future of Facades

Sergio and I were discussing the future of facades today. Facades are DSF polygons that are extruded into buildings by pushing up the polygon into a roof and texturing the roof and walls from a single texture using some simple formulas. The rules for facades are very simple - we originally thought them up for the purpose of creating buildings that precisely fit a city grid no matter what the block spacing.

The problem with facades is that the rules make very simple buildings - especially the roofs. So we want more powerful facades and the question becomes: how best to do this? There are two possibilities and I mention them here because they underscore what I think is the largest overarching design decision when creating a flight simulator scenery system.

1. We could make the facade builder in X-Plane smarter (write new rules).
2. We could write a stand-alone tool that converts the DSF polygons (plus a rule file) into OBJs that are then placed directly into the overlay.

This second strategy would turn the facade from a type of scenery element into a tool for making OBJs. We would be "pre-compiling" our facades.

Here are some of the considerations:

SPACE VS SPEED

Precompiling primitives is a space-vs-speed trade-off. For any given algorithm, it's usually faster to pre-build the entities and load them from disk than to build them in the sim. But pre-building means more files on disk, meaning bigger downloads, more DVDs, etc. Building up buildings from facades inside X-Plane is actually a form of compression.

EASE OF DEVELOPMENT

This is one criteria where I think pre-compiling scenery is a clear win: it's easier to build a stand-alone facade-to-OBJ converter than to implement it inside X-Plane. X-Plane is a relatively unfriendly place to build scenery algorithms because it's busy being a flight sim.

STABILITY VS UPGRADABILITY

If we make OBJs out of facades using a tool, the objects will look the same even if we create a new version of the tool. This is good in that it means that custom scenery won't change how it looks, but it is also a limitation because improvements to facade technology require rebuilding a lot of objects to take effect.

(Another aspect of this: if the objects made from the tool use a texture, that texture can't have its shape changed even if we make new rules, because some objects will be on users machines that were built with the old rules.)

X-Plane mostly errs on the side of pre-building scenery, and this is the one area where we really take it on the chin for that decision: the inability to easily create a new composite look from changes to some parts of the scenery. (Airports that don't cause grass to appear under them is an example of this.)

Perhaps some good questions for weighing these options are:

1. Would an author care whether the facade looks exactly the same in the future?
2. Are there enhancements coming along in this technological area that we would not want to miss out on for older scenery?

OTHER CONSIDERATIONS

Information loss: when we turn a facade into an object, the original polygon information is lost. Is it important that we ship the actual base polygons that make up buildings, or are the OBJs that represent them good enough?

Contextual Information: often we know more about a given situation when making scenery than when displaying it. Could we do better at building up a facade inside the sim (where we have context from other scenery packages), or when creating the sim (where we have a much larger dataset)?

Where To Get Help

Just a gentle reminder: if you are having problems with X-Plane (especially the installer), please do not post a comment to this blog. Instead email Randy and Jack, or tech support guys, at info at x-plane dot com. This is the official way to get tech support, and they will work hard to make sure you can use X-Plane.

Polygons Part 2 - Real World vs Custom

In a previous post I discussed polygons, how they can be used, and a little bit about how they relate to X-Plane 850 and the new apt.dat system. I have been working on some demo scenery that will make this all clear, but the great is the enemy of the good, so rather than wait I'll post more on this now and get the demos done as soon as I can.

There are essentially two ways to get at the new polygon code: via the apt.dat system or via an overlay DSF. When should you use apt.dat and when should you use an overlay DSF?

• If you are trying to model something that is directly in the apt.dat spec, use an apt.dat file. For example, use apt.dat if you are making blue taxiway lights.
• Use a custom overlay DSF if you are modeling outside an airport. (Do not make "fake" airports to use apt.dat features.)
• If you need a custom look not supported by apt.dat, use an overlay DSF - it's the only way.
  
• Use a custom overlay DSF if you are modeling something that isn't found in an airport, even if it looks similar. (For example, if you want blue lights to model some unique architecture in an airport, do not use apt.dat taxiway lights - your lights may look the same, but they are not the same!)

This last point is important: do not "abuse" the definitions of apt.dat files just because they look similar.

There are essentially two kinds of features in the scenery system:

1. Features that do not change how they look, ever. For example, we do not change the way a textured triangle looks in an OBJ file.
2. Features that are designed to model the real world. Over time, we change them to look more like the real world. For example, approach lights have changed a lot in 850 to look more like the real world ones.

This second type of feature is the one where I issue caution: if a feature in apt.dat or the sim is meant to emulate how the real world looks, we will change how it looks to improve rendering! This is why it is important not to use apt.dat feature for purposes other than they were intended for. It might be that in X-Plane 8.50 the blue taxi lights look just like some other feature you want to code. But in a future version, we might make them more realistic for an airport and they will look worse for your other application. By using apt.dat features only for their true purpose, you help us ensure that our changes to the base artwork make scenery better, not worse.

(This division of all scenery features between ones that are "stable" and ones that are "based on the real world" can also be seen in most parts of the sim. In particular, the flight model is designed with a "based on the real world" philosophy, a very controversial decision I'll have to blog about some other time.)

Next: fixing airport terrain with polygons.

I'm not dead! (Just coding tools)

I realized when I saw the "view stats" of 0 that I haven't posted in maybe two weeks! Nothing's wrong, I've just been busy coding scenery tools since 860 went into RC. I'll post something useful here soon!

Just a quick random note: the maximum number of vertices in a DSF polygon is 65,535 vertices. If you have a polygon with more sides, you'll need to simplify or subdivide it.

I would also add that the speed with which forests are built up is related ot the polygon complexity (as well as the total trees made). The algorithm is pretty fast, but you may want to consider a simplification algorithm that removes sides, making the polygon smaller, if the total error is less than a few meters (or whatever your tree spacing is), nuke the side!

Popup Blockers and the Installer

For a while we've been getting the rare report from a user that our installer won't download - usually it dies mid-download on a given file.

(Our servers transfer a complete 600 MB demo copy of X-Plane once every 8 minutes, all the time. So if we get a report of a problem from one isolated user, it's almost always specific to the user. Systemic problems like a bad server generate a ton of email.)

One user stuck with us through some via-email debugging and we finally figured out one of the causes of failed downloads: popup blockers!

The X-Plane installer is built around simple off-the-shelf technology where possible. In particular, files are zipped and sent via http. This means we're using openly available existing standards, and we get to leverage free code. (The biggest win is being able to use "plain old apache" for serving the installer.)

To the operating system and any firewalls involved, the X-Plane installer doesn't look that different from a web browser. It goes to a web server and requests a big pile of files, a few at a time.

It turns out some popup blockers will detect patterns in URLs and automatically substitute the web content! So we go out asking for an object called com_120_60_1.obj.zip and the popup blocker decides that (perhaps because it's a zip file and has 120_60 in it) the file shouldn't be made available. It substitutes a 1-pixel black GIF file that the browser can show inconspicuously..

Of course, that one-pixel black file doesn't look anything like a zip file - when our installer gets it, the installer has a fit.

This bug was a mystery before - what would cause our servers to send just one user a tiny GIF file, and only in one case. Now that we know that the software is designed to zap web advertising, it makes perfect sense.

Now We Are 3

With X-Plane 860, we now mark the X-Plane executable as being capable of using 3 GB of virtual memory on Windows. This means that if your copy of Windows can support large address spaces and you have a powerful enough PC, you can now use more add-ons. (Mac and Linux have always gone to 3 GB by default.)

You can read more about setting Windows up to handle large address spaces here. Please don't try this if you don't know what you're doing, and please don't contact our tech support if you destroy your copy of Windows -- we can't support the whole OS.

Please note that this change does not improve the hardware capabilities of your computer - it only enables X-Plane and Windows to take full advantage of what you have. You will not see a performance boost because of this.

Airport Flattening, the Untold Story

I've been meaning to write this blog entry for about a year now. X-Plane 8 allows the airport surface area to be sloped. Here's some of the back story and details.

Back when Austin and I were doing the design work for teh X-Plane 8 scenery system, we made a decision to allow sloped runways. The issue is that flattening the airport area requires the sim to edit the mesh on the fly, something we wanted to avoid.

(X-Plane's scenery system is based on removing the editing of scenery from the sim itself..."X-Plane is not a GIS". I did some slides on this once, I'll try to post them soon.)

Instead we decided to simply drape the airports over the terrain, no matter how it was built. We figured that the sim's engine could handle this (as it turned out, there were bugs that were fixed in the 8.20 patch) and in real life runways are often quite sloped!

Unfortunately when theory meets practice, things can get ugly...the biggest problem we saw was that in the original set of DSFs, the underlying terrain was very bumpy, and the smoothness requirements for a plane to take off are very high. (Flattening is also necessary to match the ground height with other sims for online flight.)

So we retrofitted the X-Plane engine with terrain flattening for airports. The flattening engine is meant as a last resort, to get absolute flatness and repair an already-flattened DSF. Its goal is not visual quality, but rather speed -- that is, we can't take 10 minutes analyzing the DSF each time we load one, or the sim will freeze pretty badly. (If this requirement that the flattener be fast ever goes away, we could do a much nicer job of flattening.)

The current flattening engine has two unfortunate properties designed to keep it fast:
- It flattens an area that is larger than the airport surface area (it rounds up) and
- It flattens vertices that are in the flattening area, not whole triangles.

The first limitation means that it may crush mountains around the airport, and is not appropriate for airports that are embedded in complex, hilly terrain.

The second limitation needs more examination. If a mesh triangle is partly inside the flattening area and partly outside, then the triangle is not flattened - one vertex is moved, and the others aren't, which cause it to be sloped.




In these pictures, the blue lights represent the airport area perimeter, but the red lights show the full area that is flattened. I have artificially set the airport elevation much higher than surrounding terrain, to make the flattening obvious. Notice how some triangles become highly sloped!

When we make the global base scenery, we use the default airports from Robin's database. So even if you do not want to submit your custom airport layout to Robin's database, consider submitting some kind of layout to Robin. If an airport is present in the default scenery, then the area will be pre-flattened, which makes the sim's flattening both work better and maybe even unnecessary.

Also, you can use the 130 code in a custom airport area to increase the airport boundaries, increasing the amount of flattening. But this is a mixed blessing - as you can see the mechanism is very imprecise. If you do use a 130 code in a layout and you submit the layout to Robin, please remove any 130 boundaries that have been set to a large area to flatten an airport.

So Why is WorldMaker Still Around?

I realize this blog post will probably just inflame a bunch of email about how the scenery tools aren't available yet, but I'll answer the question and take the flames, because it's a fair criticism and scenery tools are a fair feature request.

The long term direction of scenery tools is this:

• Scenery tools will be separate from the X-Plane distribution, free, and open source. (This separation allows us to post scenery tool source without posting X-Plane source, and to use GPL code in the scenery tools.)
• A few very basic editing functions (like adding nav-aids) are integrated into the sim to allow instructors to correct nav data during a training session.
• WorldMaker therefore is no longer a scenery editor at all.

So why haven't we killed it? We've been tempted to. But it will serve a long term purpose in the scenery tools ecosystem: it will be a small-footprint 3-d scenery previewer.

Because the scenery tools don't use X-Plane code, the scenery tools will have two limits to their previewing capabilities:

1. There is always the risk that with different code, the tools will preview scenery differently from X-Plane's final render.
2. Because the scenery tools don't use X-Plane's renderer, we basically have to rewrite viewing code in the scenery tools from scratch. That's a lot of code, so for a while the preview in the tools will be limited.

Running X-Plane and the scenery tools at the same time isn't a great option - since X-Plane loads a lot of scenery, and a weather model, and a plane, and then tries to run at max fps, it tends to be a bit of a pig in terms of system resources. WorldMaker will be a viewer that can reload your scenery quickly so you can have a 3-d view of what your end result will look like that will match X-Plane.

It Ain't Over 'Til It's Over

Just a gentle reminder: please do not consider anything in a beta to be final!

By this I mean: there is a risk that file formats that are new to the current sim (860) could further change during the current (860) beta run.

So...if you are working on scenery that depends on new 860 features, please do not ship your final scenery until 860 is finished! This way you can be sure you're compatible with the final formats.

If your scenery uses 850 features and 860 breaks them, please report a bug, as 860 should handle any legal scenery that 850 handles.

(This goes for plugin datarefs too!)

Logitech Joystick - Framerate Fix

I found the reason for frame-rate loss for Logitech sticks...everything in this entry is Mac specific.

X-Plane has (for a long time) done a rather poor job of parsing the HID descriptors that
the driver provides for USB joysticks on OS X. However it turns out that in the case of Logitech, after we found the 4 axes the joystick usually has (pitch, roll, yaw and throttle) we then went on to misparse a bunch of strange stuff at the end of the stick as more axes. It turns out that reading those axes causing some kind of huge framerate loss. I don't fully understand this, but given the HID spec I've seen I'm not hugely surprised.

Why is this x86-Mac specific? I don't know! Why is this so much worse in 860? Well, in 860 we raised the number of axes from 6 to 8. With more axis slots, we tried to read more incorrect parts of the logitech stick, for more fps loss.

The solution is one that's been a long time coming: I've rewritten a bit of our Mac HID-parsing code to do a better job of figuring out what's a joystick element and what's not.

The bad news is: for the next few betas there may be some broken joysticks. If you have a joystick and it stops working in the next beta, please file a bug and include as much specific info about your joystick as you can.

The good news is: I think we will get almost all of the fps back on x86 Macs - Logitech users will not need to get new sticks.

A side effect is that we should no longer have "dead" slots - that is, buttons and axes that didn't do anything. This should allow you to use more of your X-52 if you have one.

A Change in Settings - 3-d Lights

For X-Plane 860 beta 6 I think we may change the way the airport lights setting works. Since
(despite my previous rantings) there is a lot of discussion of performance based on settings (e.g. "I used to get 40 fps with this menu pick, now I need this other menu pick") as opposed to based on what the sim is actually doing, allow me to go off in some detail about what's going on under the hood.

During X-Plane 850 and 860, I tried to fix a number of long-standing "quality" issues, where the sim had small artifacts. There is a fps penalty to fixing these artifacts that I thought was small, but the message is clear:

• Some users are flying X-Plane on lower-end systems and can't spare a single fp.
• A lot of users really don't care about the quality issues and would gladly trade back the visual "improvement" (which doesn't seem like an improvement after living with the issue for 4 years) for those few fps that mean the difference between clear skies and fog.

So...with that in mind, let me explain runway lights for 3 versions of X-Plane.

X-Plane 840 and earlier
In X-Plane 840, lights could be textured or untextured. This was controlled by a simple checkbox "draw textured lights". A few lights (on the airplane, for example) were always textured because we thought they were few in number and important visually. The rest was decided by the checkbox.

No airport light ever had a 3-d "structure" (e.g. you couldn't see the light housing and support rod).

No light was ever drawn using hardware acceleration, even if the graphic card had pixel shaders; we simply didn't have the code. Therefore textured lights ate up a lot of CPU power and thus ate up a lot of framerate. But life was simple!

X-Plane 850
With X-Plane 850 things got a lot weirder.

• We rewrote the lighting engine to optionally use pixel shaders if present. This improves performance of textured lights a lot. In fact, on a given machine with pixel shaders, textured lights with hardware acceleration are faster than non-textured lights without hardware acceleration. (For hardware accelerated lights, the texturing doesn't cost anything compared to hardware accelerated untextured lights.) It's important to note this performance fact!
• Sergio created a bunch of 3-d object lighting fixtures for the runway environment. They have a low LOD, but up close you can see the actual lighting structure.

So what to do about that setting? Well, basically with the check box on we did the nicest looking thing, and with the check box off, we did the fastest thing. (The check box's label doesn't do a good job of representing this.) We thus have four possibilites:

1. Checkbox off, no pixel shaders: we draw untextured lights using software, no objects.
2. Checkbox off, with pixel shaders: we draw textured lights using hardware, no objects.
3. Checkbox on, no pixel shaders: we draw textured lights using software, with objects.
4. Checkbox on, with pixel shaders: we draw textured ilghts using hardware, with objects.

This is plenty confusing if you ask yourself "what is this checkbox going to do" - it depends on your hardware. But the rule is quite simple if you consider the intent: we either give you the fastest (off) or nicest (on) rendering we can.

X-Plane 860
With the next X-Plane 860, we're going to use this checkbox to control multiple aspects of the sim, but always with two choices: either prioritizing visual quality, or prioritizing framerate.

So this option will not only pick the fastest (or nicest) light code and turn the objects on and off, but it will also, for example, turn on or off per-pixel fog (something that looks nice but is slower than the old per-vertex fog).

I'm not sure everyone will like this, but I think it will meet some important needs:

• It will make it obvious how to set the checkbox. Either you want speed, or quality.
• It keeps the config interface simple for new users.
• It gives us a way to tie in a number of small optimizations that add up when taken in concert.

It seems to me that we're seeing the haves and have-nots. If you've got a Core 2 CPU and a 7000-serise GPU, things like per-pixel fog are chump change. If you've got a G4 with a Radeon 9200, you need any speed you can get. So hopefully this change will make the UI clearer and help us meet everyone's needs.

Doing the Numbers

I think the framerate test may be the best piece of code I've ever written for X-Plane. It speeds up my own work checking X-Plane's performance (and removes the risk of careless error), allows us to easily collect user-based stats, and has changed the dialog on performance in the beta from he-said/she-said (the result of having no precise quantitative measurements) to a fact-based dialog box where we can discover problems more quickly.

Today in a noble sacrifice (using this card is nothing less than torture) I put a GF-FX5200
back in my G5 to look at bugs and performance. (The terrain night lighting and landing lights should be back in all nVidia hardware with shaders in the next beta.) Some things I've learned:

Turning off GLSL shaders boosts fps from 46 to 55 fps with my Radeon 9600XT but goes from 22 to 45 fps with the FX5200. From that we can learn:

• The penalty for GLSL shaders is really really high on the FX5200.
• The shader implementation for the FX5200 just doesn't run at the same speeds we see in R300-based cards...it's not in the ballpark.

Looking at a profile of the sim, with the FX5200 running shaders, about 50% of the time is spent waiting for the graphics card to chug.

Other things I've found:

• GLSL shaders are slower on terrain, but still worth it for runway lights - having those be hardware accelerated is definitely a positive.
• Per-pixel fog doesn't affect fps on the 9600XT but is a measurable penalty on the FX5200.
• The landing light also hits the FX5200 about 20% - again it's negligible on the 9600XT.

The moral of the story may be: we need a way to not use shaders for terrain on the FX5200. Between the little things that shaders give us (that we can't use) that cost too much and the overall lack of performance, GLSL terrain and the FX5200 may not be a good idea.

If you have an FX5200 and OS X 10.3, there may not be any benefit to updating to 10.4 - you'll have the option of using GLSL shaders but you may not want to.

If you have a G5 and an FX5200 and the option to put a new card in, you may want to consider it - you'll almost certainly get a fps boost (especially if you use something like an X850).

Musings on Framerate (Just a Test)

I will admit...this post is really just a test to see if the blog is still working. I just got my blogs moved to the new "blogger beta" so I want to make sure the feed still works.

One user posted this comment in response to my rant on framerate...

"What I wonder in X-Plane and FS2k4 as well is, why there is no better automatic regulation which automatically set object density, object quality and so on based on the current frame rate...."

This is a good idea that has been suggested before. However when I tried it, it worked very poorly for two reasons:

1. Because we reduce visibility as well when fps fall, if we also tie object density/distance to framerate, then we have two separate variables responding to framerate. The result was oscillations. Think of framerate and visibility as a (hopefully) dampened oscillator. With two variables, getting the oscilation out is a lot harder.
   
2. the relationship between object density as the sim controls it and framerate is not a static one, because object density (relative to other scenery complexity) varies a huge amount with locale. In the mountains, backing down objects won't save your framerate. In the city it will. So coding the "spring factor" for that oscillator is very difficult.

That doesn't mean we won't try this again in the future - that's just what happened when I tried my first naive attempt.

Other comments reflected users either hoping for more fps from existing hardware (there's a lot of that on X-Plane.org) or saying that X-Plane runs well on their modern shiny computer.

Here's some stats for you:

• Radeon 9200: 1000 MT/sec 6.4 GBi/sec
• GeForce 5200 FX (64-bit version): 1000 MT/sec 2.7 GBi/sec
• Radeon X1950XTX: 10,400 MT/sec and 64 GBi/sec
• GeForce 7900 GTX: 15,600 MT/sec and 51.2 GBi/sec

(These numbers come from Wikipedia - I make no promises.) The first number MT/sec is mega-texels per second and represents how many millions of pixels* the card can fill in. GBi/sec is gigabits per second of data transfer from VRAM to the GPU. Neither of these are perfect predictors of card speed (but they do limit your ability to run with a lot of FSAA) but they do illustrate something:

The difference between the speed of the smallest and biggest graphics cards is over 15x for fill rate and over 25x on memory bandwidth! That's the equivalent of people trying to run with a 3 ghz and a 120 mhz CPU! Anyone running with a 120 mhz Pentium out there? Definitely not - our minimum CPU requirements are ten times that.

In other words, the high rate of technology improvement in graphics cards has left us with a ridiculously large gap in card performance. (In fact the real gap is a lot worse - the 9200 and 5200FX are not even close to the slowest cards we support, and I haven't even mentioned the new GeForce 8 series, or pairings of two cards via SLI or Crossfire.)

* Pixel or texel? Technically a texel is a pixel on a texture, at least by some definitions, but perhaps more important is that these numbers indicate fill rate at 1x full-screen anti-aliasing...you would expect them to fill screen pixels at roughly 1/16th the speed at 16x FSAA. This is actually good - it means that FSAA can help "absorb" some of the difference in graphics power, making X-Plane more scalable.

This One Goes to 11

If I could have a nickel for every time a user has asked me "what machine do I have to buy so that I can max out all of the settings in X-Plane"...

The problem is simple and two-fold:

1. X-Plane's maximum rendering settings can sink even the biggest computers. Why is X-Plane's maximum so "overbuilt"?
2. The rendering settings are not good predictors of performance. Why the hell not?

The answers are, roughly in that order, "Because we can't predict the future" and "because it has to be that way". Here's the situation...

The Maximum Limits
Some of X-Plane's maximum settings are design limitations. For example, your anti-aliasing can't exceed your card's capabilities, and the visibility can't exceed the loaded area of scenery.

Other limits are based on algorithmic limitations - there are as many objects in New York City as we could possibly pack in. We figure you can always turn down object density if there are too many.

Still other limits are based on guesses about hardware. We could have made the mesh or less complex - we chose a mesh complexity for the global scenery that we thought would be a reasonable compromise in frame-rate and quality for the v8 run based on typical hardware. This is just a judgement call.

So...how big of a computer do you need to run X-Plane with maximum objects? I have no idea! We honestly didn't give it any consideration. Our strategy was:

• Make sure the maximum object density the renderer would produce would not exceed our disk space/DVD allocations. It didn't...because objects are only in cities, they contribute only a tiny fraction to the size of global scenery.
• Make sure that we could turn object density up and down easily.
• Ship it!

We figure you'll turn objects up and down until you get a compromise of frame-rate and visual quality that you can live with. If you can't find a compromise, it's time to buy a bigger computer, because we can't give you something for nothing. All those objects require hardware to draw.

And I would go a step further and say it's a good thing that the maximum density of objects is very high. When Randy (our sales/support/marketing guy) got a new PC, a very nice one, but not even the most ridiculous gaming box possible (and this was months ago) he reported that on the default settings he can fly the Grand Canyon at...283 fps!

My point is that the hardware keeps getting faster and we can't predict where it will go. So it seems like a safe bet to leave the maximum settings very high for future users.

So How Insane is Insane
The other problem is that rendering settings don't correspond well to machine power. I can't tell you that a 3 ghz CPU can handle "tons" of objects. The problem is that hardware performance is a composite of a number of subsystems, and scenery complexity varies by region and even by view angle.

Even the slightest change in camera angle can have a pretty big effect on performance. Combine that with different types of scenery elements (objects in some, beaches in other, mountains in others) in different regions and it becomes very difficult to predict how well scenery wil work on a given computer.

A Good Value
Given all this, we don't try to come up with correlations between the settings and the hardware. Instead we simply try to make X-Plane run as efficiently as we can, and try to verify that you get a "good value" - that is, a lot of graphic quality relative to the power of your computer. We have to trust that you'll buy as much computer as you want and/or can afford, and we try to give you the best look we can for it.

So if you are shopping for a new computer, you can go for the "Rolls Royce" strategy, where cash is no object and you maximize every component (even if paying a huge premium for brand new technology) or you can go for a "good value" strategy, trying to find the best price-per-performance points.

I'll close this off with a few tips on buying:

• If you are buying a new computer, be sure you have a PCIe 16x slot! Any decent motherboard should come with this -- if it doesn't, you're not getting a good computer. A PCIe 16x slot for the grahpics card shouldn't raise prices too much.
• Get 1 GB of memory minimum - X-Plane will run with less, but it helps, and memory isn't that expensive these days. A new computer will be fast enough that the system can use the extra memory. I get my memory from Crucial - it's good quality and often cheaper than what you pay if you let the computer manufacturer upgrade you.
• In the US for Windows/Linux, look for a graphics card between $100-$200 - that's the best value range for graphics cards. Get an ATI or nVidia card (nVidia if you're using Linux). I don't buy my cards from PriceWatch but I find it a useful listing of the price point of a lot of graphics cards.
  
• Don't get a video card with HyperMemory or TurboCache.
• Check the "suffix" of your graphics card on Wikipedia - there are tables for the clock speeds for both ATI and nVidia cards. An nVidia "GT" card will cost you more than an "LE" card, but will be clocked a lot faster and given you better performance.

Keyboard Mapping - seemed like a good idea at the time

X-Plane 860 introduces a fundamental change in the way the keyboard is handled in X-Plane.

Before X-Plane 860, keyboard keys are matched to their commands by looking at the character that is prodcued by typing.

For example, spot view is produced by the '@' symbol. On a US keyboard that means shift-2. On an Italian keyboard, the @ symbol is shift + the ò key (ò on its own key, cool!).

But in X-Plane 860, the matching is done on the key itself, not the symbol you get when typing. In other words, the 2 key is the 2 key no matter what. So in X-plane 860, spot view is mapped to shift-2 and shift-2 will induce spot mode whether this gives you @ or " when typing (on Italian keyboards, shift-2 gives the double-qoute).

One advantage of this method is that the keys stay in roughly the same place...the point of those strange punctuation keys for autopilot modes is that on a US keyboard they're all top-of-the-keyboard functions. Now that the keyboard bindings are based on the number keys plus the shift key, this will be true on international keyboards too.

One disadvantage of the new system is that if there is no "primary" key for a symbol, international keyboards can't use it. For example, ; is a key on a US keyboard, but is made by shifiting the , key on an Italian keyboard. So right now in beta 2 Italians can't invoke free-view (ctrl-;) at all...if they type ctrl-shift-, they get, well, ctrl-shift-,. (Of coures, mapping free view to ctrl-shift-, in the keys file does work.)

For now please just keep in mind that 860 is a beta and is still a work in progress. In the long term once we get the quirks worked out, the new keyboard system offers a lot more mapping combinations, because everything can be mapped with any combination of modifier keys. No more running out of keyboard combinations!

Also for hackers, try X-Plane --cmnds=all to see a list of all possible command bindings in the sim.