X-Plane 9 is here for Android

It shipped! X-Plane Mobile is now available for Android phones - look in the Android market under "X-Plane 9".

Edit: Chris sent me this QR Code - scan it to go to the store listing.

Edit: if you either cannot see X-Plane in the Android market or you cannot download it, please first look here for trouble-shooting tips, then contact customer support (info at x-plane dot com). Please do not use the comments section of this blog for customer support; if you need help we will need to contact you one-to-one.

X-Plane for Android

As some have noticed on the org and on FaceBook, Randy mentioned that we may be able to ship X-Plane Mobile for Android. Some users were quite befuddled to learn that we were aiming to ship X-Plane Mobile for Android so soon when X-Plane 10 is delayed. Here's the full story.

Chris, the third and most recent addition to the X-Plane programming team, began a port of X-Plane Mobile to Android a while ago; this was the second port of X-Plane Mobile after our port to Palm WebOS. He was able to accomplish most of the port fairly quickly; hence the video floating around the web of X-Plane on a Nexus One back in May.

Unfortunately we ran into some issues that stopped ship; it looks like Google may have them fixed shortly, hence our hope of finally shipping the app. So while Chris has spent a little bit of time recently working on the last few Android issues, our hope is to release a product that we already put development time into a while ago.

X-Plane vs. Blender Normal Maps

Propsman pointed this one out to me yesterday: apparently Blender tangent-space normal maps run from a value of Z=-1 (no blue) to Z=1 (100% blue). This is not how X-Plane normal maps work; our normals go from Z=0 (no blue) to Z=1 (100% blue).

This difference is easy to miss because X-Plane has to renormalize the normal map as the last step of processing the normal map. This turns a big artifact into a small one. The general effect of using the Blender convention rather than X-Plane's is that your normal map will look 'less bumpy' for fairly extreme amounts of bump.

To fix this, simply remap the colors of your blue channel in PhotoShop or some other image editing program. Basically you'll want to set what was 50% blue to 0% blue, and keep 100% blue the same. This will extend the lighter half of the blue channel over the entire blue channel.

If you have any blue less than 50% in the image, um, that's a normal that points backward, and X-Plane doesn't support that.

Updating to OS X 10.6.x

If you have a DirectX 10 or 11 class video card (that is, a GeForce 8nnn or newer or a Radeon HD card) and you're on a Mac, consider updating to OS X 10.6.x if you're still on OS X 10.5.8.

10.6 has performance enhancements in the video drivers that I suspect will benefit X-Plane 9 users, but it will really matter for X-Plane 10. We need OS X 10.6 to expose some of the OpenGL extensions that these cards have. Thus 10.6 will get you faster frame-rate, more realistic lighting, and more efficient VRAM use.

(If you have an older card, I don't know if you'll get any benefit, although I doubt you'll see a performance loss.)

Using Glass Objects in Planes

X-Plane 9 allows you to categorize objects as being on the plane's outside, inside, or glass. X-Plane depends on these flags being right for a few things:

• The draw order of the airplane is determined by the object types - glass is drawn last to avoid translucency artifacts.
• Interior light from the plane is only spilled on the "inside" objects.
• Glass objects are excluded from shadow calculations to avoid having opaque windows in the airplane shadow.

It is important that you use these flags as intended; X-Plane 10 depends on this information as well, and X-Plane 10's global spill and global shadowing algorithms are more sensitive to incorrect categorization of objects than X-Plane 9's forward renderer.

In particular, you should have glass for the airplane windows in an attached object tagged as type 'glass'; do not attach your glass to the cockpit object, which cannot be categorized as glass. If you have an old plane with glass in the cockpit, consider cutting the object in half in a 3-d editor and attaching the glass separately.

(You should also use our prop disc animation, rather than use an OBJ for prop discs; the OBJ format doesn't contain the z-buffer tricks necessary to make the prop look right.)

The Four Minute Mile

Sometimes these posts get off topic, sometimes in the direction of the art of computer programming, sometimes in the nature of the industry, and sometimes with pictures of the pets. This post is going to go off a bit into the subject of project management.

Randy and Tyler posted what was becoming clear (by the lack of an already existing beta): our estimated release date for X-Plane 10 was incorrect. Software project delays are pretty common, and often when a third party add-on is delayed, the community jumps to speculate about "what's going on" inside the project and tries to infer whether the delay is an indication of serious problems.

I'd like to try to reframe the issue of delays in terms of an analogy. You ask me: how fast can you run a mile? I tell you "4 minutes and 15 seconds". I then run a mile and you time me. My time: 6 minutes, 10 seconds.* What can we learn from this episode? I think we can learn two things:

• For a computer programmer, I am surprisingly fast - a six minute mile isn't to be sneezed at when you spend your days sitting on your ass in front of a monitor drinking coffee.
• My ability to predict my own speed is not very good. I was pretty naive to think I could run a 4 minute mile - that's what world class athletes run. My estimate was off by a fairly big error margin.

One thing we should not conclude is that because my mile time was 2 minutes slower than estimated, that I am a slow runner. The estimate sets up an expectation, but if the estimate is wrong, it's not a useful metric of efficiency.

The same applies to X-Plane; we missed our original projected ship date because the estimation of when we would be done was not a very good estimate. This isn't good for a few reasons:

• It creates uncertainty for third parties as to when a platform will change.
• It makes it difficult for marketing to properly plan a roll-out.
• It makes it difficult to balance the value of more features vs. an earlier release date (since we don't know how much "time" we are trading for "features" if the time estimates are wrong).

But the delay is not at all a black mark for our team - on the contrary, they're working their asses off and creating some really great work.

When looking at a project that will be delayed (because the original schedule was wrong) there's a few things you can do:

1. Add more people. This is quite often the wrong thing to do - please read the Mythical Man Month to understand why. Once your team is the right size, adding more warm bodies usually makes schedule delays worse and hurts efficiency.
2. Remove features. This is the only real way to bring in a ship date.
3. Move the date back.

When Austin and I were working on X-Plane 8, we hit a similar scheduling problem - what we had set out to do was going to take a lot longer than we thought. (Like X-Plane 10, we had just doubled the team size and begun a project that involved massive rewrites, which made it hard to ship until the work was fully complete. Sound familiar?) The difference? With X-Plane 8 we had contracted to ship with an external distributor for Thanksgiving, so we had to go for item 2 - we cut scope. What we cut was the world - that is, we shipped new global scenery only for the US, and the existing ENVs for the rest of the world. We also had to ship the artwork we had on hand, despite being unhappy with its quality. We didn't finish the rest of the world and graphics we were happy with for another 11 months.

Option 2, cutting scope is painful and hard. Sometimes it is the right thing to do. In the case of X-Plane, however, we have the luxury to move the date back. With that in mind, we're trying as hard as we can to keep feature-creep minimal and finish what we've already bit off, so we can get the release done and out the door.

* My mile time is not 6 minutes, 10 seconds...I would be astounded, and quite possibly in the ER if I could run that fast for any sustained amount of time.

Curved Roads

At this point I can say with 99% confidence that X-Plane 10 will feature bezier curved roads. In X-Plane 9, a road is a line segment; you can simulate curved roads by using a lot of line segments, but the global scenery roads are pretty chunky.

X-Plane 10 allows for a road to be a bezier curve, allowing the specification of smooth curves with a small amount of data. This sets us up to trade off visual quality and performance using a rendering setting.

A few notes for authors:

• Like all of the new v10 road features (and pretty much all of the new v10 scenery features), you don't have to use bezier curves in your roads. They are there as an option if you want them.
• X-Plane 10 will not make curves for you; road data that is defined as line segments in the DSF will be rendered as line segments. (This follows the principle that DSFs contain pre-processed scenery data, and the sim shows DSFs exactly as they are written.)

Pay No Attention to the Documentation

The DSF specification alludes to bezier curved roads; this "old way" of encoding curves was never supported in the sim - all versions of X-Plane ignore this data. The "old way" was how we thought we might do curves some day.

The version 10 curve encoding is different; the "old way" will continue to be ignored in version 10. So: do not use the DSF spec to try to make curved roads now. I will post detailed documentation on curved roads once version 10 is available to authors.

A Cliff Shader

I have been stingy with pictures of next-gen global scenery for one reason: it's really hard to get a nice shot of the global scenery that doesn't show unfinished features. With something like global lighting I can zoom in and show just the new trick, but with global scenery, I can't take a picture of a new house without showing a city block that looks funky due to a bug and a road that isn't finished. Posting a working shot of the global scenery where some sub-systems have bugs and artifacts would just freak everyone out.

I figure if it's obvious that the shot isn't a production shot, I can get away with posting it though.

A lot of the times when I work on the rendering engine, it is with test textures like this. Our art team does their best to hide the seams between different art assets, so that the scenery looks like one continuous world. The problem for me is that the better they do, the harder it is for me to tell if the underlying shaders are doing what they should do.

So alpilotx sent this test: it's all of the Innsbruck area painted with a test texture. What's new and interesting here is that the flat, hill, and cliff areas are all shaded by a single shader that selects between multiple textures (and rotates the textures) based on the underlying mesh.

We are adding the cliff shader to version 10 for a few reasons:

• Often we can get better cliff and hill definition by processing in the shader than by painting different triangles with different textures; our ability to control the transitions using different .ter files is limited.
• Using one slope-sensitive shader saves over-draw and triangle count, which makes the DSFs faster and smaller.
• Some day we may have the GPU distorting mountains on the fly to make them more mountainous. If we do, we need the GPU to also apply the correct textures; if the cliff areas are precomputed then they won't respond to GPU distortion.

Will X-Plane 10 Have X?

If I could have a dime for every email I have received that asks some form of "will X-Plane 10 have X" (where X is a feature or enhancement), I wouldn't need to actually work on X-Plane anymore. (If you think your email triggered this post, well, there are approximately 100 other users who have asked the same thing.)

Simply put: I have no idea and I'm not going to try to answer these questions any more. Here's why:

For as long as I have been involved with X-Plane, Laminar Research has provided free patches to the simulator throughout a major version run, and those patches have included not only performance enhancements and bug fixes, but also major new features.

So the question "will X-Plane 10 have X" can really mean one of two things:

1. Will X-Plane 10.0 have feature X immediately 'on the DVD'?
2. Will X-Plane 10.x ever have feature X in a free patch before the major version run is over.

I can answer the first question, because we are relatively locked down on what features are still on the table for 10.0 vs. what must wait, but I think it's at best confusing to do so. If a feature isn't on the DVD, it might be in a free patch within weeks; it might be available by the time you get your DVD. Whether a feature is on the DVD is of interest to us as we plan our release, but I don't think it actually makes a huge difference to users with internet connections.

Consider 64-bit - it's something we want to look at during the version 10 run but we're not going dig into it until after we get 10.0 out. So will 10.0 be 64-bit? No. But there will probably be a 64-bit patch available for free. I think you can see why I don't want to post "X-Plane 10 will not be 64 bit."

I cannot possibly answer the second question, because versions run over several years, and what we code for the end of the version run will depend on market conditions and technology that don't exist now. One of the nice things about patching X-Plane frequently is that we can revise our plans as conditions change.

Consider the question "how many cores will X-Plane 8 utilize" had you asked the question during X-Plane 8.0. When X-Plane 8.0 came out, the answer was "only one" and we had no road-map to change that. For that matter, multi-core machines were rare and exotic beasts at the time, so multi-core wasn't a priority.

Within the three years of X-Plane 8's major version run, we ended up supporting multi-core for scenery mesh loading, something that couldn't have been easily predicted at the beginning of the version run.

Finally, a note on release planning: now is absolutely not a good time to ask for features. The features that will ship in X-Plane 10.0 have already been determined, and since we'd like to ship X-Plane 10 sooner rather than later, I don't think there's anything you can say that would make us add a feature to 10.0.

All future features are going into a 10.x "bucket" for planning purposes. Since Austin, Chris and I are up to our eyeballs in code and the art team is red-lined too, we're not spending any time sifting through 10.x buckets right now. If you send us a feature request, the very best case is that we dump it in a holding list for later; the worse case is that we lose track of the request in the chaos.

That doesn't mean that we don't care about 10.x. It's just that we are very much heads down in the 10.0 release now and won't look up until it's done.

Comment Moderation

Just a quick note re: comments. Recently this blog has been attacked by spammers who copy existing legitimate comments and re-post then; their user name is a link to a click-harvesting web-site. I am doing my best to mark these comments as spam and not allow them.

Besides that, I am likely to nuke any comments that contain profanity, spam, repeated requests for tech support via the blog, and off-topic comments particularly on posts that have are meant for a narrow discussion (e.g. global scenery defects).

Draped Object Geometry in X-Plane 10

I have mentioned a few of the scenery engine features coming in X-Plane 10 that will be of interest to authors: global illumination, conditional parts of OBJs (to cope with variable rendering settings). There is another general feature coming that will make authoring scenery a lot easier, I hope.

X-Plane 9's rendering engine has the ability to drape geometry. Draped geometry are meshes that are 'dropped' onto the terrain and hug the underlying base mesh perfectly. The most common example of this is the runways: because the runways 'drape' the ground, the runway shows any curvature and bumps from the underlying base mesh. This is who we create sloping and non-flat runways.

Authors can drape geometry as well, using a draped polygon (.pol) primitive in an overlay. Such draped geometry is useful any time you want to add more "paint" to the ground, e.g. to put down a taxiway, parking markings, dirt, grass, a driveway for a house, you name it.

There is one case in X-Plane 9 where you cannot drape geometry: in an object. In an object, all geometry is aligned to the object, and will only interact nicely with the ground if you get lucky. For example, if you model a house with a sidewalk, the sidewalk won't "sit" on the ground if the ground turns out to be sloped. You can use ATTR_poly_os to hide the artifacts, but ATTR_poly_os really can't cope with mismatches between the OBJ and the terrain under it.

X-Plane 10 will introduce a new object attribute: ATTR_draped. Draped geometry in an object is actually draped down onto the terrain when the object is placed in the scenery. This means that the draped part of the object will hug the ground perfectly with no interference or Z thrash. You get all of the quality of a draped polygon with the convenience of an OBJ.

There are a few possible uses for ATTR_draped:

• Any time a 3-d model needs some ground details attached to it, e.g. the driveway near a house, draped geometry provides a good fit with the ground and good alignment with the object.
• Any time you want to include a pre-made ground decal (E.g. a painted parking spot on a taxiway), the ground detail can be modeled as an object using draped geometry.

ATTR_draped will facilitate creating and sharing custom details for airports and streamline the authoring process.

Scenery Compatibility and Version 10

This is my expectation for scenery compatibility in X-Plane 10:

Scenery based on DSFs, OBJs, and other version 8/9 file formats should work with X-Plane 10 unmodified.

This includes orthophoto scenery based on DSFs - we're not throwing that code out.

The new rendering engine features for version 10 (and there are a lot of them) are extensions - new ways to render things, new types of art assets.

I do believe that we may drop support for ENV scenery files in version 10. We've had DSF for six years now, and ENV's capabilities (a 500m mesh, very limited orthophoto resolution) aren't useful to today's users. You can use DSF2Text/XGrinder to extract custom object placements from an ENV for use in a new overlay.

We may also drop support for OBJ version 2. (Yes, we still load OBJs version 2.) OBJ version 2 is the OBJ file format from X-Plane 6, the one before OBJ 7. If you have any old OBJs (version 2 or 700) you can use XGrinder to automatically batch convert them to OBJ8.

Coping With Variable Rendering Options In X-Plane 10

X-Plane 10 will have rendering options for global illumination and global shadows. This leaves one question: what if the user has these features disabled?

The plan for version 10 is this: the OBJ file format will have some extensions to allow conditional commands based on rendering settings. A few notes on these conditional commands:

• They will only be based on rendering settings.
• They will be evaluated once when the object is loaded. (If rendering settings change, the object will be reloaded.)

The idea is to be able to change which lit texture you use or remove a set of shadow polygons depending on rendering settings.

The conditionals are evaluated once at load time so that the object can be fully optimized based on the particular set of conditionals used. For example, if your drop shadow (with ATTR_poly_os) is fully removed at load time (because global shadows are on) your object now has fewer attributes, which is good for frame-rate.

This is very different from ANIM_hide. The hide animation may or may not hide depending on datarefs; to keep this fast, you cannot "hide" an attribute, only triangles. This means you "pay" for your atttributes no matter what.

The motivation for both designs is this: if the set of attributes in a file never changes (e.g. they are either conditionally removed at file load once, or they are always present regardless of animation) then we can optimize the attributes of an object once knowing how they relate to each other, to create the leanest, meanest OBJ.

No One Has All of X-Plane 10

I commented on this before, but, like the Funniest Joke in the World, no one team member has all of X-Plane 10. We're all working away madly at our own parts, separately. This means that if one of us has code or art that isn't quite ready for prime-time, it hopefully doesn't slow anyone else down too much.

I mention this because I see a lot of commentary on the X-Plane 10 preview pictures where the comment is analyzing a part of the picture that actually isn't X-Plane 10 at all! Those pictures are coming right off of developer machines, and that developer probably has some new stuff and some old stuff. Not only does the developer probably not have everyone else's parts of X-Plane, but that developer probably has everything except his own work turned off or way down to keep sim load time down. I don't fly with 20 AI planes when I work on scenery, and Austin doesn't load full scenery at max res when he works on the flight model.

So when you look at the screenshots, just bear in mind that they are showing one new piece of technology pretty well, and pretty much everything else on screen is going to be hit or miss.

Orthophotos Are Not Going Away

I mean to blog this a while ago, and Austin has moved on to new missives about the future of X-Plane, but:

A while ago Austin posted to the news list describing our approach to global scenery (that is, the scenery that ships with the sim), and he said some, well, rather disrespectful things toward orthophotos:

Orthophotos are garbage. I see this all the time. I am zooming along in an airplane looking that rooftops of WalMarts painted flat onto the ground. And the rooftops are blurry. And pixelated. And with a magenta or purple tint. And with big blurry shears right through the middle of them when they fall between offset satellite passes. It looks just terrible.

So first let me point out a few obvious things:

1. There was never any chance that the global scenery would be based on orthophotos - not in v8, not in v9, not in v10. Simply put, we can't ship you 900 DVDs in a dump-truck. Orthophotos of any reasonable quality are too large for covering the entire world in the base X-Plane product. This is not a change or new to v10.

2. X-Plane is very capable of handling third party orthophoto scenery. We invested a bunch of engineering in this in the v9 run, and that code is not going away in v10. X-Plane will page orthophotos on multiple cores so that you get smooth flight and crisp images. If you want to see some orthophoto that don't look blurry or pixelated, look here.

3. DSF-based scenery that works in X-Plane 9 will work in X-Plane 10, unmodified. We are not getting rid of any modern scenery file formats.

Beating Ourselves Up

Austin continues in his rant^H^H^H^Hdiscussion, with this:

Then, to make the 2-dimensional, blurry, pixellated, mis-colored, distorted roof of a WalMart painted on the ground look even worse, if you throw in some REAL roads or auto-generated buildings, they invariably fall ACROSS the roof of the WalMart painted on the ground, compounding the wretched orthophoto with an Escher-like rendering-error. This looks terrible, and is not even plausible.

This is a critique of the version 8 and 9 global scenery. In fact, it is an observation of the fundamental problem with the urban global scenery: we never found a way to synchronize the real-world-driven and real-world derived 3-d scenery (real roads with plausible buildings and forests in between) with the photo-based land-class textures running underneath.

Ironically, this is not a problem with orthophotos (that is, specific photos placed in the world where they belong) per se. It's really a problem with how to combine 3-d with land class textures. I don't believe anyone has solved this problem yet for global scenery; if you look at FSX, there isn't a lot of real world vector data to interfere with the land classes and their autogen.

In fact, orthophotos can look very good when they are combined with 3-d in a correlated way. For example, take a look at this screenshot of FlyTampa's KBUF . They are using an orthophoto but they are putting matching 3-d on top of it, which makes things look good close up.

The Global Scenery Problem

I'll leave you with this thought: the problem for the version 10 global scenery is to combine:

1. the plausibility that you get from having synchronized 3-d and ground textures.
2. the detail we've come to expect in photo-based scenery textures.
   
3. the realism you get from using real vector data for the real world.

The current global scenery manages points 2 and 3 but fails pretty badly on point 1. That is what we are trying to address in X-Plane 10.

Do You Ever Feel Like the Planet Is Falling Apart?

Put this in the "sometimes bugs make pretty pictures" category...

Tuning the Rasterizer

I'm not sure anyone cares about this kind of thing, but...




These are pictures of a test of the rasterizer. The rasterizer is code in both the DSF creator* and X-Plane that converts polygons into lines or boxes. What good is that? We use it to:

• Plant trees inside a polygonal forest in a DSF.
• Process all of the elevation points inside a polygonal water body when making a DSF.

Etc., etc. These were from a performance test rasterizing the Mississippi delta at 1201 x 1201; because the vector data is insanely detailed (something like 2 million line segments I think) it's a good test of performance. The blue lines represent a line fit, but the white lines are a "box fit" - that is, they ensure that not only are they "inside" the water, but the area above and below them are too.

* Programming geeks can use this code - it's in PolyRasterUtils.

A Global Scenery Squawk List

This post is a bit of an experiment...we'll see how it goes. When we cut the global scenery, we do a number of validations: we manually inspect some of the DSF tiles, we examine the Earth orbit textures (which are derived from the DSFs) as a quick way to look roughly at all of the tiles, we have a number of internal consistency checks in the generator, and we can compare our output to some of the input data (e.g. did we lose all of our water) to look for gross bugs. But we don't have enough resources to manually examine all 18,000+ tiles in detail inside X-Plane.

So: if you have found a defect in a global scenery tile in version 9, please list the tile in the comments section of this post. I will try to give these "previously broken" tiles priority in manual inspection.

Note that the bugs we can expect in version 10 will be very different than in version 9 because we've really changed the global scenery generation process in nearly every important way. The underlying algorithms are heavily rewritten and data sources are very different. The goal here is to simply find areas that might have a higher probability of weirdness.

Let me try to be clear about what constitutes a broken tile and what does not. Please read this definition six or seven times before you post a comment.

• Do not report bugs of inaccurate data. If your favorite road is missing, or the coastline is in the wrong place, don't report that here. That is not a defect in the rendering process; rather it is a limitation in the source data. I am not trying to collect a list of data inadequacies. We have already selected the new data for the new render. "Stuff is in the wrong place" is not a bug for this list!
• Really weird patterns are of interest. For example: I have seen some reports of very long thin bars of land going perfectly north along the edge of the tile, or a comb of mountain and valley, again perfectly vertical. These are bugs in the production process and I do want to know about them!
• Do not report errors in the placement of the 3-d layers (trees, roads, etc.). Since the 3-d layer is being completely rebuilt for version 10, none of these defects will be present in version 10. (They'll be replaced with a totally new set of weird behaviors!)
• If an airport is too bumpy to use (with "flatten " turned off in the sim), report this only if the terrain around the airport is grass. Basically the grass means that we tried to "fix" the airport in the v9 render and failed. If there is no grass (the airport is over forest or city) and it is bumpy, do not report it - that means it was added to apt.dat later.
• Do not report mismatches between the airport shape and the grassy patch; this is just the apt.dat file being updated.

If you have something to report (basically incorrect flattening of grassy airport areas and really weird coastline/terrain bugs that are clearly programming errors) please include three pieces of information:

1. The DSF tile, e.g. +42-072
2. The nearest ICAO of an airport near the bug (e.g. KBED)
3. A very short description of what went wrong (e.g. "tall thin vertical lakes running through the entire DSF").

Please only post defects of the above nature in the comments section of this blog post; to keep things clean I am going to nuke any off-topic comments on this post.

EDIT: see the first Squawk, Arista's report on LOWS. This is a perfect report: it includes the DSF tile, the ICAOs, a brief but clear description, and it is a bug in how we process the data, not the data itself.

Airport Layouts: Your Cutoff is 10/10/10

If you are planning on submitting an airport layout to Robin so that it is used to flatten terrain in the X-Plane 10 global scenery, please submit the layout to Robin no later than October 10th, 2010 (that is, 10/10/10).

If you have an incomplete but useful layout (e.g. the airport border is in place but not the taxiway signs, you can still submit it; we only consider border outlines and the pavement itself when flattening, not markings.

You do need both the border outline and all existing pavement. The reason for this is that the airport border is used to change the land class to grass, but water is only converted to land (if we have a coastline error) base on real pavement.

More info on airport layouts and how to submit data to Robin can be found here.

WED Roadmap

I posted a quick tip on how to create fence-like facades in WED. Basically WED doesn't handle fences and other non-closed polygons very well, but you can work-around this. A future version will address this more completely.

This is my thinking on the WED roadmap:

• WED 1.1 will relatively ship soon, without cosmetic and usability improvements. At this point the basic bugs (import/export) are fixed, so best to get out a finished, stable build so that people can at least edit overlays.

• A future version will provide editing ATC layout information for X-Plane 10. This shouldn't be too hard to get working, as we already have this in WED now so that we can develop test data for the new ATC system.

• A future version will provide usability enhancements (e.g. previews of facades, etc.). I don't have a ton of code done for this yet, but it's important for everyone using WED for pretty much any purpose.

• A future version will provide road editing since X-Plane 10 can drape roads.

I don't know what order those 3 feature will come in, but they will all happen relatively soon after version 10 ships I think.

A Better WED Beta (and friends)

On my way back from South Carolina a few weeks ago I had some time in the airport to fix the WorldEditor 1.1 export bugs. All of the reports pointed to a single cluster of bugs that are hopefully now fixed in beta 2, and today I finally had time to get the betas posted.

I also cut a new build of MeshTool (2.0RC2) while I was doing builds; this fixes a bug when using orthophotos with varying physics.

So first: you can get the new WED 1.1 beta 2 and MeshTool 2.0 RC 2 on the tools page on the wiki.

The wiki? Yeah. Tyler has been helping me migrate the scenery tools to here. At this point I believe that all of the content on scenery.x-plane.com is now duplicated on the wiki (which also has additional articles).

WED Bugs

If you should find additional WED bugs, after you get over your initial surprise, please file bugs in the scenery tools bugbase. Please do not email me bug reports. WED has to take second priority to version 10 work, so I don't have time to process bug reports now, and I will lose them. The bug base keeps your bug safe, keeps a record of what happened to it, and can take attachments.

Please do provide the minimal materials to reproduce the bug; simple packages with an earth.wed file are great. Thanks to all of the WED users who filed bugs with repro cases - this made it very easy to retest the export code.

My Polygon is Poly-Gone

It is illegal to have DSF polygons (or airport polygons) cross themselves; finally with beta 2, WED actually makes this case fairly easy to detect.

1. If a polygon cannot be exported (because it is self-intersecting), an error message will indicate that some polygons were skipped, and those polygons are selected.
2. If you pick "Error-Check Polygons" from the Edit menu, then for every polygon that has a self-intersection, an OBJ is added at the precise point of the self-intersection. Simply select and zoom in on those OBJs - they act as marker pins to show the problem. Delete the OBJs once you have untangled your polygon.

(I did experiment with error checking on the fly, e.g. simply showing red dots in the intersection points as you drag and resize the polygon, but the math is too slow. I am using CGAL to check bezier polygon intersections, and their algorithm is absolutely rock solid, but it can take up to a quarter of a second to recheck the polygon, which is too slow for interactive editing.)

UPDATE: beta 2 hangs when processing beziers. What is very odd is that this happens on the clean release code but not the working copy of WED I fixed the bugs in. Hence, when I checked all of the bug reports, they all passed. I have reopened all bezier-related bugs. I have not yet located the build environment differences causing the problem.

UPDATE 2: the hang on bezier processing was due to a bad compile configuration for a library WED uses, and was Mac specific. Having fixed this, I have recompiled and reposted WED beta 2 for Mac. If you already grabbed WED, re-download it, and make sure you get the version dated September 24th. You can tell you have the right one because the "compiled on" date in the about box will say "Sep 24 2010 19:34:09".

Performance Tuning and Future Proofing

I wrote up some performance tuning notes for OBJs on the wiki. A few notes on how these rules will change with version 10:

• Everything on that note applies to version 10 too. If you've tuned your model for version 9, that effort will be worth it in version 10.

• A few rules are even more important in version 10 than before. In particular, I've done a lot of performance tuning for OBJ drawing, but you don't get those wins if you use ATTRibutes. Clean your objects out for maximum speed.

• One special case: objects with very small vertex count are sometimes extra fast in version 10. For example, in version 9, a tree with 8 vertices and no attributes is horribly slow. In version 10, this tree will be quite fast. So in version 9 you might make the tree have 64 vertices and look nicer; in version 10 by keeping the tree lean and mean, you get a speed improvement.

  
  

Use Every Pixel of Your Normal Maps

Normal maps are expensive. A 2048 x 2048 normal map takes 22 MB of VRAM! So make sure you get every bit of image quality you can out of it. Two things to consider:

• Normal maps are uncompressed (because texture compression really screws up the normal map). So per-pixel detail will be preserved. Use it!
• VRAM is allocated for an alpha channel whether you use one or not. This is because the cards need 32-bit pixels for performance. So include an alpha channel in your normal maps and use it to modulate shininess; this can help create the illusion of different materials.

For scenery objects, do not include an alpha channel if you don't need it. When textures are compressed, the alpha channel does take more VRAM, and X-Plane will hit a faster rendering path for textures with no alpha channel.

Revisiting Texture Compression

For quite a while now, I have been advocating in favor of DDS compression. I am pretty damned obstinate, but eventually if enough people yell at me, I get a clue. I have come to appreciate that there are some cases where DDS compression is not a net win; this blog post explains when it happens and what we might do in X-Plane 10 to work around this.

DDS - The Good, The Bad, the Ugly

DDS is a file format that contains image data pre-mipmapped (that is, the smaller versions of the image that the video driver needs are included) in a format that may or may not be compressed. DDS is virtually always used with a compressed image format (like DXT1 or DXT5). This has three positive effects for X-Plane:

1. Because the image is already compressed, we save CPU time when loading the texture that would be spent compressing while X-Plane is running.
2. Because small versions of the image (the "mipmap pyramid") is already in the file, we save time down-sizing the image with the CPU, another win for load time.
3. Because the image is compressed ahead of time, it can be compressed with a slow high quality compressor rather than a fast low quality compressor, so relative to other compressed images we get an image quality improvement.

The bad is that the DDS file does not contain the original uncompressed file. If the user unchecks "compress textures to save VRAM", DDS files remain compressed. If the image file contains details that don't compress well, they're going to get splatted and stay splatted.

What If VRAM Grew On Trees?

My original heavy arguments for DDS were based on the idea that VRAM is a limited commodity; if we don't compress textures, the user runs out of VRAM faster and has to go down a level of resolution...and once that happens, everything starts to look ugly.

But what if the user has 1 GB of VRAM? At this point, we've limited the maximum quality the user can see because we don't have the original uncompressed image anymore, only the DDS/DXT version. This can be frustrating to authors who spent a lot of time on their textures.

If you ship PNGs with your airplane or scenery, turning off texture compression will reveal this beautiful, uncompressed image, but now when texture compression is on, the compression will be done by the video driver, and that will look extra ugly.

The Best Of Both Worlds

This is my thinking for version 10. (These are just musings, we haven't coded this yet.) Currently DDS are preferred to PNG files. We could relax the load rules in version 10 to prefer PNG over DDS when texture compression is off and DDS over PNG when it is on. This would allow authors to ship both PNGs and DDS files and have the right one be picked for the scenario: the pre-compressed one when texture compression is on and the uncompressed one when compression is off.

Feature Requests for X-Plane 10

Yesterday I got a bit cheeky regarding feature requests for X-Plane 10. Here are a few slightly more serious thoughts regarding the feature request process.

The major features that we are putting into X-Plane 10 were decided, for the most part, a long time ago. Those who we met in France two years ago won't be surprised by the major items on the list: weather, ATC, airports, and new scenery, new shaders. For that kind of major feature work, we have to start an initiative very early on. Heck - global sun shadows (which I believe we will ship in v10, God willing) were in the works before version 9.0 shipped - it took that long to get an implementation that was useful! (In fact, the v9.0 version was not released because the quality was bad, which is why I am always nervous about announcing features in advance; it ain't over until it's over.)

So while it's exciting to see so many people discussing X-Plane 10, realistically if a feature is a "big" feature and we haven't started it now, it's not going into 10.0. That train left the station over a year ago.

There have also been two complicating factors that have cropped up during the X-Plane 9 run:

1. Microsoft closed ACES, which definitely changes what we want our next major release to look like.
2. The iPhone came along - until the first iPhone release, we had no idea that it would be such a big product.

While we have to plan our big features in advance, the market we are going to ship into changes during development.

So bear with us - a lot of what I see are good ideas that we will get to soon, even if they don't go in the initial roll-out.

Feature Requests for Reality

With X-Plane 10 on the way, it seems that the various X-Plane forums are just filled with users listing off their favorite feature requests. If there is one uniting theme for these requests, it is this: they are pretty much all requests for X-Plane to more accurately model some neglected aspect of reality.

I would like this relationship to be two-way, with give-and-take. It doesn't seem fair that in every case, we should have to change X-Plane to model reality. So I humbly submit to the X-Plane user community: my requests for changes to reality to ease X-Plane development.

No Wind Near Water

My first request is a simple one: can we simply not have any more wind over the ocean? Ocean waves are hard to model. They are non-sinusoidal, they don't repeat, they're refractive, and they go for miles. If we could simply eliminate the wind, then we could model the ocean as a nice calm glassy lake, which would be quite a bit simpler to get right.

Only One Building

My second request is a modest one: from now on, I would like everyone to live in the same type of uniform house. It can be a nice house (several stories, with a pool in the house), but everyone's house should be the same. We have some new instancing technology for version 10; it reaches peak efficiency when a single building is repeated ad infinitum. So if you could all agree on a single house type, it would be good for everyone framerate wise.

(I realize that some of you in subdivisions in the US that already do this; hopefully the rest of the world can take a hint.)

No More Intersecting Roads

Alex and I have spent just a crazy amount of time trying to cope with what to do when two roads intersect. So my third request is: no more intersecting roads. I would like all roads to cross each other by some kind of overpass. This will eliminate a lot of artifacts.

In fact, for extra credit, if all roads could be aligned to be precisely north-south or east-west in a grid, that would be good for RAM use.

No Variation in Climate

This one isn't so important for me, but I think the art team might appreciate it. It turns out that different parts of the world have different temperature variations and rainfall, and that in turns makes the local color of grass, trees, etc. a bit different. This just makes work for our artists - they have to draw everything several times for the different climates. It would help us out a lot if you could all live in places with the same annual rainfall and temperature variation.

(Just in case this climate thing turns out to be true, I suggest everyone settle on the "hot dry" climate pattern - we have pretty good textures for that.)

Pave the Bay

Finally, when in doubt, use asphalt. We have some new shaders in version 10 that will make asphalt look pretty good.

Anyway, this is my list - I am sure the other X-Plane developers will have other requests. (I suspect Austin's life would be made easier if weather was always CAVOK, and Chris's life would be greatly simplified if there weren't any other airplanes in the sky.)

Do you have any hints as to when this new version of reality might be released, or what its minimum system requirements are?

More Airplanes

Quick airplane links:

• Carenado's Mooney M20 J is out, available on the X-Plane.org store.
• Javier's T34c Mentor is out, available at X-Aviation.com.

Pictures available in both links.

Multicore and Version 10

Over the last two years, we've been working on the X-Plane rendering engine, setting up the code to take advantage of multiple cores when possible. Most of these changes are in X-Plane 9 already: multi-core creation of 3-d scenery, multi-core loading of scenery, multi-core loading of textures.

X-Plane 10 will leverage this work, pushing even more work to multiple cores. And yet, nine women cannot have a baby in one month. The ultimate limit on framerate will be based on the performance of one core pushing data to your graphics card.

So how many cores do you need, and is it better to have a few fast cores or more slow cores?

I can't give you a firm answer, because I don't know how important money is to you, I don't know which rendering settings you care most about, and X-Plane 10 isn't finalized. (And even if it was, we often improve performance in patches.) But I can suggest our attitude to how cores are used.

A Graphic Analogy

With graphics cards, the companies target different markets. The "enthusiast" market is the top end, where money is no object and maximum performance is the goal. Below that you have "performance" cards (a good value for the money but not as fast) and "mainstream" (which by video game standards means "slower than snot" - main stream users don't need fast 3-d graphics to check email).

When it comes to rendering features, we expect X-Plane to be efficient enough to meet the performance expectations of a given slice of the market. In other words, if you have bought a top-level graphics card, we need to be efficient enough to let you run at 8x FSAA on a huge screen with per pixel lighting - that's what is expected of the enthusiast crowd and that's what the cards are built for.

But if you have a mainstream card and get 5 fps with per pixel lighting, well, too bad. You've got a cheap, low powered card, you need to dial down the sim. Here our goal is only to give you a way to run X-Plane at all. (And frankly, if X-Plane could run at 60 fps on a mainstream card, it means the max rendering settings don't take advantage of top end hardware!)

Core Considerations

At this point we expect pretty much any modern computer to have at least two cores, and users with single core machines are going to have to make serious graphic quality sacrifices to run X-Plane. (This is already true for version 9.)

When it comes to version 10, I think we will categorize "a lot of cores" (e.g. 4, 6, or more) as enthusiast - some of the top level rendering options may not function well with less than four cores, but dual core machines should at least run decently with some rendering options enabled.

How far we go toward utilizing really high core count (Austin upgraded to the new Mac Pro and now has 8 hyper-threaded i7s) I don't yet know. My guess is that we will get at least some measurable benefit from up to 8 cores.

Trading off clock speed for core count is a difficult choice. Obviously if you had a choice of one core that was twice as fast as a 2-core CPU, you'd take the speed - you're getting the same total computing power but the clock speed can help frame-rate. But the trade-off is almost never formulated like this, and it's too soon to have hard data from the sim itself.

For what it's worth, the latest generation of CPUs is really fast, so it is unlikely that you'd upgrade to a new CPU and not get some serious benefit, whether it comes in cores or clock-speed. The users I have heard from with new iMacs seem quite happy with their machines.

How Specific Should a Bug Report Be?

I got this bug report from Austin this morning (found in the in-development version 10 code):
yter bug

0: go to kcae with no scenery installed at all
1: open the 777
2: go to aircraft:aircraft and situations
3: go to second tab
4: set total num acf to 1
5: go to first tab
6: select 'carrier catshot'
7: release the brakes, with throttle at idle
8: enjoy the cat.. ease the plane gently onto the water, gear down, pwer off
9: note that the water is 'hard'... the yter is telling us that we are on land not water. hit brakes. stop. u r walking on water!

The first thing to note is how specific and small each step is. These steps are much more precise than most of the bug reports we get. And yet, they must be that specific - the bug was so tweaky that skipping virtually any of those steps will cause it to not appear.

The biggest problem I see with bug reports sent to me is the use of "any", e.g. "open any aircraft". That's not very specific! And Murphy's law says: the one airplane I pick will just happen to be the one airplane that you haven't tried and happens to not show the bug for some strange reason.

So always be absolutely specific. When we ask for tiny steps, it's not that we don't know how to use the sim - it's just that we need to use the sim exactly the same way you did!

EDIT: bug report pages.

• For X-Plane.
• For the scenery tools.
• For the plugin SDK.

Be sure you understand which bug base you should be using (E.g. are you reporting a bug against the plugin SDK, the sim itself, or a scenery tool like WorldEditor.) If you don't know what program you are filing a bug against, you probably shouldn't file a bug at all - just contact tech support.

WED Export Bugs

There's a real difference between a useless beta (e.g. what we have with WED now) and a useful one. While the purpose of a beta program is to fix bugs, a beta can be useful if the features users must have work. I don't think I will have time to finalize WED 2.0 this year, but I do think I will be able to fix the really nasty bugs, the ones that make it useless.

To that end I have fixed a number of crash-on-export bugs. To the users who filed complete crash-on-export bug reports: thank you! It was quite an education to see what was in some of these projects polygon-structure-wise.

If you filed a WED export bug, I may have requested feedback if you didn't include the WED file. Basically I would like the minimal materials to fully reproduce the bug by opening the package and picking "Export to DSF". For many packs this is just the Earth.wed file, although sometimes textures or text files may be necessary.

A number of the crash bugs are caused by degenerate polygons. For example, if your polygon crosses itself in an X shape, not only is this illegal in X-Plane, but it also drives the exporter crazy. With the next beta, WED will select polygons that failed to export to help you locate such problems.

Does Sponge Bob Square Pants Need Wake Turbulence Separation

I am down in South Carolina with Austin and Chris, working on integration issues with the new ATC system for X-Plane 10. One issue Chris is working on now: how do you fit the wide open design options of Plane-Maker into the narrow FAA categories for aircraft. If someone builds a rocket powered lighter-than-air piano with a tilt-rotor and a lift fan, how many miles of separation does that aircraft need from the 747 in front of it? What SRS category is Sponge-Bob Square-Pants, or a Steinway piano, or Snoopy's dog house? (And don't even get me started on some of the weird creations Propsman has sent me!)

Our current solution is to analyze the structure of the plane and use the most conservative criteria available. If your aircraft weighs a lot, it's a heavy, no matter how you built it. If it has jet engines, it's a jet, etc. This way airplanes modeled after real life will get correct categorization, and custom designs will at least receive some kind of vaguely sane handling.

GA Aircraft Abound

X-Aviation has released the Duchess, pics here.

Carenado is coming to X-Plane, initial pics here.

More Global Illumination Video

I realized I have slightly better test shots of global illumination than the ones that got out a week ago. These are from only a day or two after the last shots.

This is the Cirrus again, with landing lights and strobes; you can see that all of the airplane's lights contribute dynamically to the lighting on the fuselage and doors as they move. (Yes, that is heat blur on the engine; the heat blur still needs a lot of tuning.)

This shows airplane-mounted lights interacting with custom scenery. In this case the standard Cirrus (with global lights attached) casts both strobe and multiple landing light spill on LOWI. One of the powerful results of global illumination is that we get correct lighting interaction between diverse content, including third party content.

Finally, this shows an airport beacon lighting a plane and vice versa. The global lights on the airport beacon are inside an animation group, making them "sweep" the airplane, which can in turn animate the airport beacon. With global illumination, there are no rules about who lights who.

X-Plane 10 and Global Illumination

Thanks to my foolish use of unprotected directories, we have basically announced that X-Plane 10 will feature global illumination. Here is some basic information on global illumination.

What Is Global Illumination?

Global illumination is the ability of any part of an airplane or scenery system to cast light on any other part of the scenery system or airplane. In X-Plane 9, the only lights in the sim that ever actually cast cast light anywhere else are:

• The sun.
• The airplane's landing light. (Even if your plane has many landing light billboards, there is only one spill effect.)
• Three 3-d lights in the 3-d cockpit.

This list was kept short due to the high cost per pixel of each light on all rendering.

When X-Plane 10's global illumination is enabled, a "spill" light attached to any OBJ can shine light on anything near it. Since any OBJ can have a spill light, this means we can have light sources on airplanes, scenery, cars, whatever you want. The spill effects any 3-d scenery nearby, even from another scenery pack.

This kind of still effect can be simulated in X-Plane 9 by careful use of LIT textures. However, real global illumination works between art assets created by separate authors. You can drive your custom airplane up to a custom airport and the landing and logo lights on the airplane will cast light on the terminal; the apron lights from the terminal will cast light on the airplane.

Furthermore, global illumination is fully dynamic - as objects animate or move, the lighting effects are correctly applied in 3-d. This makes effects possible that cannot easily be created using LIT textures.

Requirements for Global Illumination

Like most new rendering tricks in version 10, global illumination will be a rendering option that can be optionally enabled by users who have a video card meeting hardware requirements. In the case of global illumination, that requirement is a DirectX-10 generation video card, e.g. any Radeon HD , nVidia GeForce 8000 or 9000 series, and "100" series (100,200,300,400 series).

For authors: global illumination is applied using named and parametrized lights on your OBJ. Anywhere you can attach a light billboard, you can attach a spill effect as well, with some tuning constants for how wide you want the light, etc.

It will be possible to create two versions of your LIT textures, one to be used when global illumination is enabled, and one when it is disabled. Thus if you are baking lighting into your textures with a 3-d modeling program, you can simply re-bake the lit texture with some lights disabled and add 3-d lights to your model. The result is an airplane with real 3-d lighting where possible, and a close approximation via baking otherwise.

Global illumination can be added to a model incrementally; existing art content will work normally with global illumination enabled or disabled, so authors can choose to add a few light spill effects or add a large number, as time permits.

The Cost of Global Illumination

Global illumination isn't going to be free. The main cost is an increase in VRAM use and fill-rate. The cost of global illumination is mostly a one-time cost to put X-Plane into a new rendering mode. (Graphics nerds: global illumination is implemented via deferred rendering.) The incremental cost of lights isn't that high, although a scene with a lot of lights will have impact.

My expectation is that users with new, highly capable high-end graphics cards will be able to run global illumination easily, but will lose some of the other benefits of fill rate. (For example, running at 2560 x 1024 + 4x FSAA is a lot more painful with global illumination than without.)

Global illumination also introduces two artifacts, both of which I am trying to minimize as best as I can. These artifacts are a function of deferred rendering - all games that use deferred rendering have to address these problems:

• The lighting calculations are shared between multiple translucent surfaces, which can create some strange effects. For example, if a translucent window is in shadow, the scenery behind the window will appear to be in shadow too.
• Traditional full-screen anti-aliasing is not available with deferred rendering. We should be able to offer a simulation of 4x FSAA as well as some kind of cheaper FSAA-approximation, but the cost will be quite a bit higher in fill rate than the 16x-style CSAA available now.
  

(Hardware-based FSAA can make a number of optimizations like CSAA to optimize throughput; this is how such high multiples as 16x are possible. Since our implementation is similar to "super sampling" and costs a real 4x in performance, 4x will be the highest setting.)

Why Global Illumination

Of the new X-Plane 10 rendering engine features (and there are a fair number of them), global illumination is certainly the one that has the most impact on the structure of the rendering engine. With global illumination, X-Plane effectively has two separate modes ("forward" rendering, which is the only mode X-Plane 9 has, and "deferred" rendering, which produces global illumination).

One of the reasons to get global illumination done earlier than other features was that implementing global illumination required rewriting or modifying nearly every piece of low level rendering code. Now that the work is in place, we can safely add new features and test them in both modes.

Global illumination also meets two requirements:

• Sergio has long observed the central importance of lighting and shadows in the look of X-Plane; at some point more polygons and better textures still look synthetic without a realistic illumination model. Global illumination makes a more realistic lighting model possible at night. Airports represent an environment that can hopefully take advantage of such capabilities in a big way.

• As hardware becomes more powerful, authors have to do more work to create content that takes full advantage of the rendering engine. We are reaching a point where artist's time is going to be a limiting factor as well as hardware and engine capabilities. Global illumination thus kills two birds with one stone: it makes the rendering engine's output look better, but it also makes the whole scene look better with less work by the artist.

  (For example, when baking lighting into a model, the author must plan the model's texture UV map to guarantee unique texture space for all spill effects. When lighting effects are dynamic, the author can simply texture so the model looks good without worrying about baking requirements.)

  
  

X-Plane 10: What Has Been Posted So Far

There have been a few posts about X-Plane 10 in a few random places; here is a summary of all of the version 10 material that's been previewed so far, and some notes on what is actually being shown.

First we had Paul's Oshkosh 2010 video:

I think this has been made clear already, but:

• The base simulator shown here is not X-Plane 10, it is X-Plane 9.
• Many of the airplanes shown here will be released for X-Plane 10.
• This is not showing the new X-Plane weather system or global lighting.
• Some of the content shown here are third party add-ons, available today for X-Plane 9.

The main purpose of this video was to show X-Plane off at Oshkosh; at the time we didn't have X-Plane 10 in a state where we could do an exclusive version 10 preview.

Then I accidentally leaked two test videos of global illumination. This was strictly accidental: I was looking for a cheap way to post a large video for Austin and Propsman late at night and didn't think anyone would sift through 191 zip files to find two obscurely named videos. I was wrong, and someone found them on the org. I appreciate that participants in the ensuing discussion withheld judgment; these were early test videos and don't represent the final feature in any useful way. They do, however show off some of what global lighting will mean.

• This is the Cirrus Jet with landing lights implemented via global illumination. We get two distinct landing lights that cast specular hilights on the fuselage. As the door animates, it opens "into the light".
• This is the Avanti Piaggio with strobes and beacons implemented via global illumination. The strobes cast light both on the fuselage and on the runway below the plane.
  

This second movie is typical of the kinds of tests I do: the beacon lighting is just awful - a gross huge red light splatted on the plane for test purposes. When I get the rendering engine code working I usually hand the feature off to Propsman or Sergio to tune the textures and art constants. In this case, the videos are pre-tuning.

Austin has posted three screen-shots of X-Plane 10-related content:

• This is Javier's new shuttle, which I believe will ship in version 10. I believe this shot may have been taken in X-Plane 9. So this is not the new weather system.

  Some of these screenshots and Paul's video were shot in X-Plane 9. By the time the new airplanes are finished, they will not be usable in version 9 - they will be version 10 only.

• Propsman has done work on the lighting system. It can be subtle to see what's going on here because the old runway lights looked pretty good too, but most of these billboard lights are actually rebuilt.

• This night shot shows global illumination in the scenery system. The glow on the highway pavement is not rendered; it comes from the 3-d lamps along the side of the road. Similarly, the car headlights spill light on the pavement and each other as they drive. (Note how the highway lines are visible in the headlight spill even when there is no streetlight.)

  One difficult problem with rendering a lit highway at night is that the lighting from street lamps on a highway tend to spill light on the surrounding terrain, an effect that is impossible to create with a LIT texture. If you look at the right side of the main highway at the bottom of the picture, you'll see that the street light is casting light on the grass to the right of the highway too.

I think that that's all we have posted. At least, it's all I am aware of. I will go into some of the details of global illumination in another post.

Removing Add-Ons is a Diagnostic Step

When we get bug reports or tech support requests, the first thing we usually recommend is: remove third party add-ons (plugins, scenery, airplanes).

This is not intended as a cure, only a diagnostic! The goal of removing add-ons is to identify whether a problem occurs due to an interaction with a particular third party add-on or within the base simulator package.

If an add-on induces the problem, we can then look at whether the add-on had problems on old versions of X-Plane. (If not, we can look at how we broke third party compatibility, and fix it.) If the add-on has always had problems, we can look at the content or contact the author.

If the problem is in the base simulator, we can compare results from a number of different users; the base simulator gets used a lot, particularly in demo downloads, so defects that aren't reported frequently are typically due to configuration issues like new drivers.

So if tech support asks you to remove add-ons, don't despair - it's just a diagnostic step, not a cure.

Sometimes You Can't Change Aircraft-Based Datarefs

A month ago the blog was quiet because there was a lot I couldn't talk about; now it's quiet because I am up to my eyeballs in X-Plane 10. I'll try to get out a few posts I've been meaning to write, but it's definitely crunch time.

I have received a number of emails from plugin developers who wanted to know if they could modify some of the sim/aircraft datarefs, or why modifying them had no effect/an unintended effect.

The short answer is this: in some cases X-Plane will pre-process and cache data that comes from the .acf file. In this case, a sim/aircraft dataref (most of these come from the .acf file) can be read, but writing it will have no effect because the sim has already had its one look at the dataref.

This is a design limitation; it was never the intention of the SDK to allow complete Plane-Maker-level editing of the aircraft on the fly in the sim.

An Older Build for Regression Testing

I'm never thrilled about posting bug-swatting info on this blog; the blog is (among other things) a temporal snapshot of what's going on in X-Plane, as well as an archive; it's my hope that we'll get this problem sorted out soon, at which point this blog post will do nothing but confuse. But I digress.

There have been a number of reports from users of the sim hanging on startup with this configuration:

• A 64-bit Windows (usually Vista or 7).
• A modern ATI card running Catalyst 10-6 or 10-7 drivers.
• X-Plane 9.62rc2.
• Usually a core i7 type system.

However, I haven't been able to reproduce this, and neither has ATI.

I don't know what the problem is, but a number of variables have changed in this equation that need to be isolated: new sim, new video drivers, newer operating systems.

So if you have this configuration and can't run the sim, despite removing all third party add-ons, please download this time demo. If you can run the 945 time demo but cannot run 962, please let me know by email, and we'll isolate a defect in the sim. I have heard from some users that they can run 940, but no confirmation that 945 runs.

Procedural Or Algorithmic

A quick thought on procedural vs. algorithmic scenery: there was some discussion on the X-Plane dev list about procedural terrain; the Outerra screen shots have stirred up interest.

There is a fundamental difference between what you see with Outerra and what we do with our global scenery:

• Typically procedural scenery is based on the idea of 'amplifying data' - that is, making a little bit of data look more interesting by applying a recursive algorithm to "generate" detail.
• Algorithmic scenery involves taking a large amount of unique input data and translating it; the detail comes from not losing information in the source data.

The key difference is whether the resulting scenery comes from a process that "creates" information through a 'procedure' or "translates" information.

Our scenery process does a bit of both, but we are more in the algorithmic camp than procedural camp; the global scenery is produced from hundreds of GB of input data, and we are constantly looking for better input data to create more interesting and accurate output scenery.

In fact, I would say that we are becoming more algorithmic and less procedural. In version 9, the urban roads in non-US cities are "procedural" - an algorithm generates them from the terrain data, an algorithm, and some random noise. For version 10, we are importing OSM.

One thing I have noticed in the work on version 10 global scenery is that we've finally crossed a line. With version 9, the question was 'what is the best data we can get'. With version 9, the question is 'how much information can we keep'; we've reached a point where the resolution of our input data is so much higher than what can go on DVD, that it's a question of filtering down, not synthesizing up the resolution.

Facade Tuning and Tips

I have updated some of the facade documentation on the wiki with new performance tips for using facades.

A few quick notes:

• Your facades must be counter-clockwise (when viewed from above). Do not repeat the first point; X-Plane will "close" your building for you. (A four sided building should have four points, not five.)
• If you turn off two-sided facade drawing and your walls look wrong (and your roof disappears) your facade is wound in the wrong direction.

The performance tips go into a fair amount of detail about saving memory. Most of X-Plane's rendering fall into two categories:

1. Shared meshes (objects), where the geometry of the object is saved once and used lots of times. Objects usually hurt frame-rate by consuming CPU time, because for each drawing of the object, we have to do some setup to draw that shared geometry in a different location. (Version ten should feature some major improvements in object efficiency.)
2. Non-shared meshes (everything else), where every single "instance" of a tree, facade, forest, etc. is uniquely constructed in memory. Non-shared messages are very fast (because we can submit a huge pile of non-shared messages to the video card in one shot) but they consume a lot of memory (because we pay for RAM per building/tree, not just once). Typically non-shared meshes are limited by virtual address space, not by framerate.

Facades are non-shared meshes, so the performance tips focus on how to limit the amount of RAM needed to represent your facade.

Let Your Eyes Adjust

This is a feature I looked at putting into X-Plane 9, but it turned out that it affects so many different parts of the sim (and has to be done all-or-nothing) that it got kicked to v10. Consider these two pictures of the default B777 (the lighting was not adjusted, only the time of day):



The night image looks pretty, but what's wrong with the day image? The answer is: the small panel post lights in the night image are still casting a fair amount of light in the day image. And the result looks silly. But why?

The answer is: in real life your pupils would contract in the sun, letting in less light. The sun is really rather bright, so the daytime panel would still look normal, but the apparent power of those posts lights would be a lot less, because your eyes are less sensitive. In other words, the relative strength of the sun and post lights is wrong in the second image.

Computer monitors don't have a huge dynamic range for how much brightness they can put out. So we can't hope to display the absolute brightness of the scene correctly. Instead we need to make everything brighter at night (to simulate your night vision) and dimmer during the day, like this:




In this set of images, the night image is matched precisely to the previous one, but as the sun comes out, the apparent brightness of all lit textures has been scaled down to simulate the effect of your eye becoming less sensitive due to the flood of sunlight.

What's good about the compensated image is that the weird artifacts from the post lights are gone; the relative strength of the post lights is really low in relative terms.

What happened to the EFIS and moving map? The answer is that they too are not as apparently bright relative to the sun as they would be at night.

There is one hitch here: plenty of real airplanes have light sensors for various avionics; the avionics will automatically turn up their brightness during the day. So it is possible (I am no expert on the 777) that in the real plane, as the sun rises, you might not have to adjust your instrument brightness; the sensor would do it for you. The pictures above illustrate what you would see if no automatic adjustment is made.

Auto-adjustment presents a challenge: currently two wrongs make a right. We don't auto-adjust the brightness of instruments, but we don't simulate the apparent visual brightness relative to the sun, and the result are instruments that look adequately bright at all times of day without user adjustment.

I think in the productized version of this feature, authors will have two options for anything lit:

• Tie the lit instrument/texture to an auto-adjusting rheostat (e.g. brightness 1 + auto adjustment) or
  
• Tie the lit instrument to the "raw" rheostat (e.g. brightness 1).

The tricky part will be finding the right mapping for legacy airplanes into the new system.

Performance of Panel Texture vs. 3-d Cockpit

I sometimes get questions from authors considering how much to rely on a 2-d panel mapped to 3-d via the panel texture vs. a true 3-d panel. I can't comment on what will look best, but I can comment on the relative performance characteristics of both techniques, and the answer might surprise you: in some cases you'll get better performance by modeling directly in 3-d.

The 2-D Way

When you use the panel texture to make an object, X-Plane goes through a lot of steps to create the final result:

1. Your panel has to be rendered in 2-d. We atlas your panel textures, but we don't necessarily order them optimally - we don't know the optimal order. Each generic instrument is at least one batch, perhaps even two. Those batches have very low vertex count, and the vertices are stored non-optimally on the CPU. There may be a fair number of texture changes between instruments.
2. If you use ATTR_cockpit_region, we then go back and do the same thing...again! Why? Well, we need your panel's raw color ("albedo" to graphics nerds) and the emissive light given off by anything self-lit separately, so that we can do correct 3-d lighting.
3. Both of these are rendered to an off-screen texture that the video driver will feeel obligated to preserve at all costs, putting pressure on VRAM.
4. Only when all that is done do we begin drawing your object, with the usual batches to change to panel texture and change back, perform animations, etc.

If this seems expensive, that's because it is. Periodically users send me airplanes to look at their performance, and lately I've been seeing a lot more problems with 2-d panels (that fuel 3-d cockpits) being the performance bottleneck, not the 3-d modeling itself.

The 3-d Way

What if we want to go 3-d? Well, we're going to "eat" a lot more of what your 3-d pit already has:

• You'll need a lot more animations to move all of those parts.
• You'll need new batches with ATTR_lit_level to dial up and down various lighting levels.

But you do get some advantages:

• Geometry in objects is processed about as optimally as we possibly can. All of that work we've done on the rendering engine to make OBJs fast is available in your cockpit. So you can increase 3-d detail 'for free'.
  
• Your lit geometry can be drawn in a single pass (we don't need to prepare two separate lit textures). So for example a needle would take three batches via the panel-texture route (a batch to rotate the needle for albedo, a second batch to draw the rotated night needle, and a third batch to draw the resulting texture in 3-d) but only one if you use the OBJ directly.
• Since you organize your textures for OBJs, you can guarantee that all of the cockpit stuff is together, saving texture thrash.
• You can use normal maps to add per pixel detail to your cockpit; panel textured geometry cannot be normal mapped.
  

A Balancing Act

Given the high cost of panel texture relative to native OBJ drawing, you'd think going native OBJ would be a no-brainer, right? Well, not quite.

A needle is an easy case: you can model a needle using a rotation animation, so your implementation in an OBJ and our generic instrument are quite similar. Same with the throttle lever generic instrument.

But what about a "glass pie indicator"? What about a moving map? What about a rotary?

There are some generic instruments that have "movement" for which there is no equivalent OBJ technique. With these generics, the generic instrument/panel code may be able to render the generic quite a bit more directly than your OBJ can simulate the same effect.

This is my suggestion on a cut-off: if you can directly model a generic instrument with an OBJ (needles, throttles, and other "simple moving things"), consider 3-d. If you would have to use a lot of extra texture space, copies of your mesh, or a lot of show-hides, use the panel texture.

Your goal should not be to eliminate the use of panel texture. But if you can cut panel texture down to a single 1024 x 1024 region from a larger area, you'll probably see a performance win or a reduction in your airplane's system requirements.

Performance Test First

Final thought: before you invest months in a complex cockpit design, mock up the "work-load" X-Plane must do and performance test it! For an OBJ, simply make one moving instrument and duplicate the mesh to get the number of expected animations. For the panel, drag out a bunch of instruments, make custom textures and just paint junk into them with photoshop. The goal is to make X-Plane do the same amount of work as it will in the final version. Then fly your test panel on target computers and observe performance.

Why Don't the Cars Work Quite Right in Replay?

The short answer is: to save memory.

The cars and replay seem to be a case of damned-if-we-do, damned-if-we-don't. If we don't stop the cars and reverse them in replay, we get piles of bug reports. If we do try to replay the traffic, we get bug reports too.

The current implementation is a bit strange: when you replay traffic, the cars will go back a bit in time, but at some point they will just stop and refuse to reverse any more. What's going on?

The answer is that the cars have the memory of a goldfish. They simply don't remember where they came from. Each car knows what "link" it is on, and about when it got onto that link and how fast it is going. (A link is a single straight piece of road.) So when we go into replay, we can easily move the cars along their links as time goes forward and backward.

But when we reach a time earlier than when the car entered the current link, the car has know idea how it got there, so it is forced to stop.

This is a simple case of not wanting to burn four tons of memory on a feature that is mainly visual. To replay the cars, we would have to accumulate a history of every link a car has been on as it drives. For 20,000 cars and a sim that's been running a while, that's a lot of memory to burn just in case you happen to hit the replay button.

In fact it gets worse. The cars are kept in a data structure that tells us who needs to make a driving decision and when.* This structure is optimized for the cars moving forward in time. We'd have to make and maintain an entire second copy of this structure to move the cars backward; again burning CPU and memory while you fly just in case you might hit a replay.

So instead we just provide replay on the current link.

* Programming nerds: the cars are in a priority queue by time to next navigation decision. I consider this to be very clever.

Games Vs. Platforms

Previously I blogged about the difference between integrated first person shooters (FPS) and flight simulators, and how these differences mean that FPS tend to adopt new graphics technology significantly ahead of flight simulators. One of the major differences is that a FPS often will have its content packaged with the rendering engine in a single, unified product, while a general purpose flight simulator is expected to cope with third party content.

The need to be a platform for external content doesn't just impact our ability to optimize for "special cases" (e.g. we can't assume anything about third party); it also puts more pressure on the rendering engine to be robust in the case of error.

X-Plane has low level and high level scenery abstractions.

• Low level: an OBJ is low level. You give us a textured mesh, and we draw it. We don't process the mesh, we don't interpret it, we just draw what you made in Blender, AC3D, etc.
• High level: a forest. You tell us the outline of the forest's area and give us some trees and we fill in the forest, picking trees and placing them.

Now there is always the risk that third party content can look stupid. If you model an airplane and you use 4 quads for each engine, your airplane is going to look bad, and there's nothing the rendering engine can do to fix that.

But with higher level abstractions, the problem is more subtle. If the input data to a high level abstraction has a problem, X-Plane's rendering might look bad. But what constitutes a problem?

In the case of forests, if the polygonal area of a forest is too thin (along certain axes) we will fail to put any trees into the polygon. Exactly what represents too thin isn't particularly well documented or even easy to measure. This is difficult for third parties, because they don't have an explicit set of guidelines for "you will make the rendering engine grumpy if you do X."

This is the kind of thing that, in an integrated FPS, is much easier to cope with. The art team tries a technique, and if it looks bad, they email the rendering engine coder. The coders then either fix the rendering engine or tell the artist "don't do that".

In our case, we need to be more robust in the case of input data problems because we can't tell everyone who tries X-Plane "don't do that", particularly when the edge cases may change with rendering engine improvements. So whereas a rendering engine feature in an integrated FPS might be useful if it looks good when used in a few usage cases , a rendering engine feature in X-Plane is only useful when it looks good under most usage cases.

OSM Tilings

This is two pictures of "tilings" of OpenStreetMap for use in global scenery. I downloaded a OSM new planet extract about a month ago; in the 11 months since I last grabbed it, the data size has grown 56%! The new, larger file required some changes to my extracting code. After much debugging, I was able to see this in QGIS:


The first picture is 1x1 tiles, which are derived from the second picture (10x10s). You'll see some "ragged" edges. This is because the cutting scheme leaves whole roads of interest in one piece even outside the tiling bounds. Later, more sophisticated code crops the road when the actual DSF is built.

The OSM processing tools are part of the open source scenery tools; I will get my changes checked in to source control over the next few days, although my code is only one of dozens of programs for bulk processing OSM.