Going AWOL

It's that time of year - I will be away from all forms of technology next week, enjoying cool mountain air and not having email. When I return I will post an udpate on the status of WED.

Don't Change lights.txt

Some very advanced users have asked: can we change lights.txt. The answer is: please don't.

Lights.txt is not a part of the "scenery SDK", that is, it's not a file whose format we will keep the same and allow you to modify. (The fact that it isn't accessible via the library system is an indication of our intention NOT to make it part of the scenery system.)

The problem is basically this: lights.txt translates named lights into the inputs to our pixel shaders for the hardware-accelerated lights. That pixel shader is really new and likely to change a few times. If the shader changes, we might need new parameters not in lights.txt, requiring a fundamental format change.

For example: those who have poked in the named lights file have noticed that the hardware lights can either have directional or flashing properties. This is because they run on two different shaders, each taking only four input values. This was done a while ago, when we were using low level assembly language shaders. In the future we might merge the two shaders and have 8 params per light. This would give us more flexibility (directional flashing lights), more bus usage (pushing 8 params per light instead of 4) and fewer state changes (we have to change shaders right now).

My point is: we can't predict what will happen, so we can't safely expose these parameters. The best thing to do is: email me and request named light types. We can easily have hundreds of named-light types (see how many there already are just for airports!).

Named lights make our lives easier because it tells us WHAT to draw but not HOW to draw it. So when we put in the next evolution of the lights code, we can remap the named lights to look the best they can for the new technology, instead of worrying about how to map the old params to the new ones.

(I appreciate the input from the users who emailed me about this -- it gives me more insight into what extensions to the scenery system would be useful.)

Liveries Part III: Image Contracts

The livery system I described in my previous blog entry (proposed independently by several users) would imply a contract between the aircarft creator and livery creator. The way I look at it:

• The livery creator agrees to utilize the texture conventions established by the aircraft modeler.
• The aircraft modeler recognizes that changes in the fundamental texturing scheme of the airplane become much more "expensive" because they potentially invalidate a whole set of liveries.

This is an exceptional situation in X-Plane...we have avoided contracts regarding the layout of images in all previous cases. (Examples: you can't use the library to replace an OBJ's texture. You specify terrain via a text file that references the image file, not the image file itself, so information on how to use the texture isn't part of the contract.)

But in the case of liveries, I think the exception is the inevitable outcome - a livery quite literally is a repainting of an aircraft, and the X-Plane community has been living with the limits of this kind of add-on for a while.

What this means is that it would be appropriate to override object textures in a livery system, but this feature is unlikely to appear anywhere else in the sim.

Liveries Part II: A Simple Approach

These ideas are similar to things that both Cormac and Peter have discussed with me...to propose a very simple naive livery implementation:

• An aircraft folder has a "liveries" folder inside it.
• Each folder within "liveries" is a "livery package", that is a folder that can be dropped in to extend how the plane looks.
• When X-Plane is using the livery , the liveries/ folder is searched before the aircraft's folder to find image files and objects.

X-Plane would have to provide some kind of UI to switch between liveries. This would allow authors to publish the base airplane and then publish liveries that simply drop inside, just like custom scenery drops inside the "custom scenery" folder.

Liveries Part I: Contracts

Any time two independently varying components need to interoperate, a contract is required.

Because X-Plane varies independently of third party add-ons (we release patches, you make new add-ons), any extensible part of X-Plane implies a contract. That contract basically says what legal things the add-on can do and how X-Plane will react.

We have to consider this "contract" with third party add-ons any time we modify the sim, and in some cases it means we can't change things. How this applies to liveries will become clear later.

Libraries and PNGs

There was some discussion on X-Plane.org about whether it should be possible to share and/or override PNGs via the library system independent of their objects. I say "no" for this reason:

The library system connects independent third party add-ons, that is, components that vary separately. Therefore there needs to be a "contract" any time the library is used, between the package requesting something from the library, and the package fulfilling it.

My concern with PNGs is that a library PNG would have to have a fixed layout. But realistically the kinds of PNGs that people want to share get reorganized on a regular basis. In particular, people want to reference the default sceney PNGs. But where we have a contract, we are limited in what we can change, and reorganizing the default scenery is critical to our ability to grow our content. I agree with Aussie's comments on x-plane.org that fixed-layout PNGs don't add a lot of value to the library system.

File System Vs. Text Files

There are a number of design trends in all of X-Plane's third party add-on systems. One is the use of the file system to specify modifications to the sim. For example:

• Put a "cockpit" folder in your aircraft, we'll try it first.
• Some files in the aircraft folder must have certain names.
• Sometimes putting _LIT after a texture causes it to be used as a lighting map.

On the other hand, I've gone a very different route with the scenery system:

• Libraries are looked up via a text file, not via filename.
• In all scenery cases, the lit texture is specified in a text file, not by filename.

What's the difference? Well, the file system way seems to be simpler for most users to understand. The text file mechanism is a lot more flexible. (Consider: we have versioning info in a text file, we can have each line in the text file clearly identify a feature, and so there is no risk of a file name being mistaken for a feature.)

Whether we use file names or text files to control extensions, doing so creates a contract, so we must ask: can we easily extend or safely modify the contract later? Can we express what we want using filenames (or text files)?

In the case of the scenery system, I think we need the full expressiveness of text files. Imagine if we wanted to provide a new kind of lighting that uses a different format of texture. Or a way to control at what time of day the lights get turned on? Or what if we want seasonal varying textures, and the rules for how seasons affect the light map aren't simple? In all of these cases, a file name convention rapidly becomes unworkable.

On the other hand, the airplane system has done reasonably well with filenames. Based on what I see, the problems with airplane distribution mostly come from a lack of features (no livery support, no plugins built into airplanes), not file names as a convention running out of flexibility.

Why Objects Kill [Framerate]

Austin and I were running some numbers on the KSBD demo area DSF as part of a discussion of how instancing will someday allow X-Plane to render more objects. (Instancing is the ability to render multiple simple objects with a single instruction to the GPU...the requirement of one command to the GPU per object means that total object count is bottlenecked by the CPU->GPU connection.)

Here's some numbers:

KSBD contains 868,220 mesh vertices - at 32 bytes per vertex, we have about 27 MB of geometry per DSF. In one view we picked, about 20% of those vertices were on screen. (But since there are six DSFs loaded, really only about 3% of a DSF mesh is seen at one time at lower altitudes.)

KSBD contains 153,816 objects. Near the airport, we average about 238 vertices per object. (This is a GOOD number - less than 100 vertices would imply that we aren't sending out enough vertices for each command to draw.) But this means that if we were to simply store all of the objects as one huge object, we would have 1.1 GB of object memory! This is why you can't just make a single huge object for the world. :-)

(X-Plane of course stores each OBJ file once, saving a lot of memory. But then we burn CPU power telling each OBJ to be drawn over and over in many places.)

It also explains why forests keep causing people to run out of memory. Consider that only about 3% of the mesh may be visible (when we're at lower altitudes, where you can see the trees). This means that we need about 30x as much memory to storage geometry as the card can draw. With cards so fast, we easily run out of memory before we max the card out.

Color Matching in X-Plane

Sergio sent me a scenery package with the question: why don't these two textures appear as the same color in X-Plane (they did in Photoshop). The answer is: gamma correction.

Some background (with much hand-waving): gamma refers to how bright the mid-tones of an image appear. (It's a lot more complex than that, but that's what Wikipedia is for!) Basically Macs and some other computers adjust the colors in an image to compensate for the deficiencies of CRTs, while PCs leave them alone. The result is that the same numeric color levels, when sent to Mac and PC hardware, result in brighter images on the Mac than the PC.

Since X-Plane is authored almost entirely on the Mac, the old complaint (around the time of X-Plane 6, with BMPs) was that X-Plane looked too dark on Windows. PNG addresses this issue: the gamma curve of the system an image was created on can be written into the PNG, allowing X-Plane to adjust the colors (making them brighter or darker) depending on what destination system we are running on.

Unfortunately, X-Plane isn't too brilliant about this, in two ways, one of which isn't our fault:

• X-Plane assumes the platform default gamma for both Mac and PC (that would be 1.8 for Mac and 2.2 for PC). If you read that background article, you know that this isn't real clever of us. But it actually is better than doing nothing at all.
• Gamma in PNG files is optional; if we get a PNG file with no gamma information, we make the rather arbitrary assumption that it came from a Mac. Since X-Plane used to be authored on Macs, this seemed like a reasonable thing and in the case of no gamma information, we're going to be wrong half of the time no matter what.

If you open up the default scenery PNG files in Preview, you can see the gamma inforation - 0.4545. This is 1/2.2, meaning the files are encoded to PC gamma standards. It turns out that one of the textures Sergio sent me had no gamma information, so X-Plane assumed Mac gamma (0.5555, or 1/1.8). Thus the brightness of these textures were being adjusted by different amounts.

My recommendation to authors is simple: make sure that all of your PNG files always have gamma values written into them. Otherwise there is a risk that the default gamma guess that X-Plane makes will not be the one you authored under, causing color shifts.