Multicore and Version 10

Over the last two years, we've been working on the X-Plane rendering engine, setting up the code to take advantage of multiple cores when possible. Most of these changes are in X-Plane 9 already: multi-core creation of 3-d scenery, multi-core loading of scenery, multi-core loading of textures.

X-Plane 10 will leverage this work, pushing even more work to multiple cores. And yet, nine women cannot have a baby in one month. The ultimate limit on framerate will be based on the performance of one core pushing data to your graphics card.

So how many cores do you need, and is it better to have a few fast cores or more slow cores?

I can't give you a firm answer, because I don't know how important money is to you, I don't know which rendering settings you care most about, and X-Plane 10 isn't finalized. (And even if it was, we often improve performance in patches.) But I can suggest our attitude to how cores are used.

A Graphic Analogy

With graphics cards, the companies target different markets. The "enthusiast" market is the top end, where money is no object and maximum performance is the goal. Below that you have "performance" cards (a good value for the money but not as fast) and "mainstream" (which by video game standards means "slower than snot" - main stream users don't need fast 3-d graphics to check email).

When it comes to rendering features, we expect X-Plane to be efficient enough to meet the performance expectations of a given slice of the market. In other words, if you have bought a top-level graphics card, we need to be efficient enough to let you run at 8x FSAA on a huge screen with per pixel lighting - that's what is expected of the enthusiast crowd and that's what the cards are built for.

But if you have a mainstream card and get 5 fps with per pixel lighting, well, too bad. You've got a cheap, low powered card, you need to dial down the sim. Here our goal is only to give you a way to run X-Plane at all. (And frankly, if X-Plane could run at 60 fps on a mainstream card, it means the max rendering settings don't take advantage of top end hardware!)

Core Considerations

At this point we expect pretty much any modern computer to have at least two cores, and users with single core machines are going to have to make serious graphic quality sacrifices to run X-Plane. (This is already true for version 9.)

When it comes to version 10, I think we will categorize "a lot of cores" (e.g. 4, 6, or more) as enthusiast - some of the top level rendering options may not function well with less than four cores, but dual core machines should at least run decently with some rendering options enabled.

How far we go toward utilizing really high core count (Austin upgraded to the new Mac Pro and now has 8 hyper-threaded i7s) I don't yet know. My guess is that we will get at least some measurable benefit from up to 8 cores.

Trading off clock speed for core count is a difficult choice. Obviously if you had a choice of one core that was twice as fast as a 2-core CPU, you'd take the speed - you're getting the same total computing power but the clock speed can help frame-rate. But the trade-off is almost never formulated like this, and it's too soon to have hard data from the sim itself.

For what it's worth, the latest generation of CPUs is really fast, so it is unlikely that you'd upgrade to a new CPU and not get some serious benefit, whether it comes in cores or clock-speed. The users I have heard from with new iMacs seem quite happy with their machines.

How Specific Should a Bug Report Be?

I got this bug report from Austin this morning (found in the in-development version 10 code):
yter bug

0: go to kcae with no scenery installed at all
1: open the 777
2: go to aircraft:aircraft and situations
3: go to second tab
4: set total num acf to 1
5: go to first tab
6: select 'carrier catshot'
7: release the brakes, with throttle at idle
8: enjoy the cat.. ease the plane gently onto the water, gear down, pwer off
9: note that the water is 'hard'... the yter is telling us that we are on land not water. hit brakes. stop. u r walking on water!

The first thing to note is how specific and small each step is. These steps are much more precise than most of the bug reports we get. And yet, they must be that specific - the bug was so tweaky that skipping virtually any of those steps will cause it to not appear.

The biggest problem I see with bug reports sent to me is the use of "any", e.g. "open any aircraft". That's not very specific! And Murphy's law says: the one airplane I pick will just happen to be the one airplane that you haven't tried and happens to not show the bug for some strange reason.

So always be absolutely specific. When we ask for tiny steps, it's not that we don't know how to use the sim - it's just that we need to use the sim exactly the same way you did!

EDIT: bug report pages.

• For X-Plane.
• For the scenery tools.
• For the plugin SDK.

Be sure you understand which bug base you should be using (E.g. are you reporting a bug against the plugin SDK, the sim itself, or a scenery tool like WorldEditor.) If you don't know what program you are filing a bug against, you probably shouldn't file a bug at all - just contact tech support.

WED Export Bugs

There's a real difference between a useless beta (e.g. what we have with WED now) and a useful one. While the purpose of a beta program is to fix bugs, a beta can be useful if the features users must have work. I don't think I will have time to finalize WED 2.0 this year, but I do think I will be able to fix the really nasty bugs, the ones that make it useless.

To that end I have fixed a number of crash-on-export bugs. To the users who filed complete crash-on-export bug reports: thank you! It was quite an education to see what was in some of these projects polygon-structure-wise.

If you filed a WED export bug, I may have requested feedback if you didn't include the WED file. Basically I would like the minimal materials to fully reproduce the bug by opening the package and picking "Export to DSF". For many packs this is just the Earth.wed file, although sometimes textures or text files may be necessary.

A number of the crash bugs are caused by degenerate polygons. For example, if your polygon crosses itself in an X shape, not only is this illegal in X-Plane, but it also drives the exporter crazy. With the next beta, WED will select polygons that failed to export to help you locate such problems.

Does Sponge Bob Square Pants Need Wake Turbulence Separation

I am down in South Carolina with Austin and Chris, working on integration issues with the new ATC system for X-Plane 10. One issue Chris is working on now: how do you fit the wide open design options of Plane-Maker into the narrow FAA categories for aircraft. If someone builds a rocket powered lighter-than-air piano with a tilt-rotor and a lift fan, how many miles of separation does that aircraft need from the 747 in front of it? What SRS category is Sponge-Bob Square-Pants, or a Steinway piano, or Snoopy's dog house? (And don't even get me started on some of the weird creations Propsman has sent me!)

Our current solution is to analyze the structure of the plane and use the most conservative criteria available. If your aircraft weighs a lot, it's a heavy, no matter how you built it. If it has jet engines, it's a jet, etc. This way airplanes modeled after real life will get correct categorization, and custom designs will at least receive some kind of vaguely sane handling.

GA Aircraft Abound

X-Aviation has released the Duchess, pics here.

Carenado is coming to X-Plane, initial pics here.

More Global Illumination Video

I realized I have slightly better test shots of global illumination than the ones that got out a week ago. These are from only a day or two after the last shots.

This is the Cirrus again, with landing lights and strobes; you can see that all of the airplane's lights contribute dynamically to the lighting on the fuselage and doors as they move. (Yes, that is heat blur on the engine; the heat blur still needs a lot of tuning.)

This shows airplane-mounted lights interacting with custom scenery. In this case the standard Cirrus (with global lights attached) casts both strobe and multiple landing light spill on LOWI. One of the powerful results of global illumination is that we get correct lighting interaction between diverse content, including third party content.

Finally, this shows an airport beacon lighting a plane and vice versa. The global lights on the airport beacon are inside an animation group, making them "sweep" the airplane, which can in turn animate the airport beacon. With global illumination, there are no rules about who lights who.

X-Plane 10 and Global Illumination

Thanks to my foolish use of unprotected directories, we have basically announced that X-Plane 10 will feature global illumination. Here is some basic information on global illumination.

What Is Global Illumination?

Global illumination is the ability of any part of an airplane or scenery system to cast light on any other part of the scenery system or airplane. In X-Plane 9, the only lights in the sim that ever actually cast cast light anywhere else are:

• The sun.
• The airplane's landing light. (Even if your plane has many landing light billboards, there is only one spill effect.)
• Three 3-d lights in the 3-d cockpit.

This list was kept short due to the high cost per pixel of each light on all rendering.

When X-Plane 10's global illumination is enabled, a "spill" light attached to any OBJ can shine light on anything near it. Since any OBJ can have a spill light, this means we can have light sources on airplanes, scenery, cars, whatever you want. The spill effects any 3-d scenery nearby, even from another scenery pack.

This kind of still effect can be simulated in X-Plane 9 by careful use of LIT textures. However, real global illumination works between art assets created by separate authors. You can drive your custom airplane up to a custom airport and the landing and logo lights on the airplane will cast light on the terminal; the apron lights from the terminal will cast light on the airplane.

Furthermore, global illumination is fully dynamic - as objects animate or move, the lighting effects are correctly applied in 3-d. This makes effects possible that cannot easily be created using LIT textures.

Requirements for Global Illumination

Like most new rendering tricks in version 10, global illumination will be a rendering option that can be optionally enabled by users who have a video card meeting hardware requirements. In the case of global illumination, that requirement is a DirectX-10 generation video card, e.g. any Radeon HD , nVidia GeForce 8000 or 9000 series, and "100" series (100,200,300,400 series).

For authors: global illumination is applied using named and parametrized lights on your OBJ. Anywhere you can attach a light billboard, you can attach a spill effect as well, with some tuning constants for how wide you want the light, etc.

It will be possible to create two versions of your LIT textures, one to be used when global illumination is enabled, and one when it is disabled. Thus if you are baking lighting into your textures with a 3-d modeling program, you can simply re-bake the lit texture with some lights disabled and add 3-d lights to your model. The result is an airplane with real 3-d lighting where possible, and a close approximation via baking otherwise.

Global illumination can be added to a model incrementally; existing art content will work normally with global illumination enabled or disabled, so authors can choose to add a few light spill effects or add a large number, as time permits.

The Cost of Global Illumination

Global illumination isn't going to be free. The main cost is an increase in VRAM use and fill-rate. The cost of global illumination is mostly a one-time cost to put X-Plane into a new rendering mode. (Graphics nerds: global illumination is implemented via deferred rendering.) The incremental cost of lights isn't that high, although a scene with a lot of lights will have impact.

My expectation is that users with new, highly capable high-end graphics cards will be able to run global illumination easily, but will lose some of the other benefits of fill rate. (For example, running at 2560 x 1024 + 4x FSAA is a lot more painful with global illumination than without.)

Global illumination also introduces two artifacts, both of which I am trying to minimize as best as I can. These artifacts are a function of deferred rendering - all games that use deferred rendering have to address these problems:

• The lighting calculations are shared between multiple translucent surfaces, which can create some strange effects. For example, if a translucent window is in shadow, the scenery behind the window will appear to be in shadow too.
• Traditional full-screen anti-aliasing is not available with deferred rendering. We should be able to offer a simulation of 4x FSAA as well as some kind of cheaper FSAA-approximation, but the cost will be quite a bit higher in fill rate than the 16x-style CSAA available now.
  

(Hardware-based FSAA can make a number of optimizations like CSAA to optimize throughput; this is how such high multiples as 16x are possible. Since our implementation is similar to "super sampling" and costs a real 4x in performance, 4x will be the highest setting.)

Why Global Illumination

Of the new X-Plane 10 rendering engine features (and there are a fair number of them), global illumination is certainly the one that has the most impact on the structure of the rendering engine. With global illumination, X-Plane effectively has two separate modes ("forward" rendering, which is the only mode X-Plane 9 has, and "deferred" rendering, which produces global illumination).

One of the reasons to get global illumination done earlier than other features was that implementing global illumination required rewriting or modifying nearly every piece of low level rendering code. Now that the work is in place, we can safely add new features and test them in both modes.

Global illumination also meets two requirements:

• Sergio has long observed the central importance of lighting and shadows in the look of X-Plane; at some point more polygons and better textures still look synthetic without a realistic illumination model. Global illumination makes a more realistic lighting model possible at night. Airports represent an environment that can hopefully take advantage of such capabilities in a big way.

• As hardware becomes more powerful, authors have to do more work to create content that takes full advantage of the rendering engine. We are reaching a point where artist's time is going to be a limiting factor as well as hardware and engine capabilities. Global illumination thus kills two birds with one stone: it makes the rendering engine's output look better, but it also makes the whole scene look better with less work by the artist.

  (For example, when baking lighting into a model, the author must plan the model's texture UV map to guarantee unique texture space for all spill effects. When lighting effects are dynamic, the author can simply texture so the model looks good without worrying about baking requirements.)

  
  

X-Plane 10: What Has Been Posted So Far

There have been a few posts about X-Plane 10 in a few random places; here is a summary of all of the version 10 material that's been previewed so far, and some notes on what is actually being shown.

First we had Paul's Oshkosh 2010 video:

I think this has been made clear already, but:

• The base simulator shown here is not X-Plane 10, it is X-Plane 9.
• Many of the airplanes shown here will be released for X-Plane 10.
• This is not showing the new X-Plane weather system or global lighting.
• Some of the content shown here are third party add-ons, available today for X-Plane 9.

The main purpose of this video was to show X-Plane off at Oshkosh; at the time we didn't have X-Plane 10 in a state where we could do an exclusive version 10 preview.

Then I accidentally leaked two test videos of global illumination. This was strictly accidental: I was looking for a cheap way to post a large video for Austin and Propsman late at night and didn't think anyone would sift through 191 zip files to find two obscurely named videos. I was wrong, and someone found them on the org. I appreciate that participants in the ensuing discussion withheld judgment; these were early test videos and don't represent the final feature in any useful way. They do, however show off some of what global lighting will mean.

• This is the Cirrus Jet with landing lights implemented via global illumination. We get two distinct landing lights that cast specular hilights on the fuselage. As the door animates, it opens "into the light".
• This is the Avanti Piaggio with strobes and beacons implemented via global illumination. The strobes cast light both on the fuselage and on the runway below the plane.
  

This second movie is typical of the kinds of tests I do: the beacon lighting is just awful - a gross huge red light splatted on the plane for test purposes. When I get the rendering engine code working I usually hand the feature off to Propsman or Sergio to tune the textures and art constants. In this case, the videos are pre-tuning.

Austin has posted three screen-shots of X-Plane 10-related content:

• This is Javier's new shuttle, which I believe will ship in version 10. I believe this shot may have been taken in X-Plane 9. So this is not the new weather system.

  Some of these screenshots and Paul's video were shot in X-Plane 9. By the time the new airplanes are finished, they will not be usable in version 9 - they will be version 10 only.

• Propsman has done work on the lighting system. It can be subtle to see what's going on here because the old runway lights looked pretty good too, but most of these billboard lights are actually rebuilt.

• This night shot shows global illumination in the scenery system. The glow on the highway pavement is not rendered; it comes from the 3-d lamps along the side of the road. Similarly, the car headlights spill light on the pavement and each other as they drive. (Note how the highway lines are visible in the headlight spill even when there is no streetlight.)

  One difficult problem with rendering a lit highway at night is that the lighting from street lamps on a highway tend to spill light on the surrounding terrain, an effect that is impossible to create with a LIT texture. If you look at the right side of the main highway at the bottom of the picture, you'll see that the street light is casting light on the grass to the right of the highway too.

I think that that's all we have posted. At least, it's all I am aware of. I will go into some of the details of global illumination in another post.

Removing Add-Ons is a Diagnostic Step

When we get bug reports or tech support requests, the first thing we usually recommend is: remove third party add-ons (plugins, scenery, airplanes).

This is not intended as a cure, only a diagnostic! The goal of removing add-ons is to identify whether a problem occurs due to an interaction with a particular third party add-on or within the base simulator package.

If an add-on induces the problem, we can then look at whether the add-on had problems on old versions of X-Plane. (If not, we can look at how we broke third party compatibility, and fix it.) If the add-on has always had problems, we can look at the content or contact the author.

If the problem is in the base simulator, we can compare results from a number of different users; the base simulator gets used a lot, particularly in demo downloads, so defects that aren't reported frequently are typically due to configuration issues like new drivers.

So if tech support asks you to remove add-ons, don't despair - it's just a diagnostic step, not a cure.

Sometimes You Can't Change Aircraft-Based Datarefs

A month ago the blog was quiet because there was a lot I couldn't talk about; now it's quiet because I am up to my eyeballs in X-Plane 10. I'll try to get out a few posts I've been meaning to write, but it's definitely crunch time.

I have received a number of emails from plugin developers who wanted to know if they could modify some of the sim/aircraft datarefs, or why modifying them had no effect/an unintended effect.

The short answer is this: in some cases X-Plane will pre-process and cache data that comes from the .acf file. In this case, a sim/aircraft dataref (most of these come from the .acf file) can be read, but writing it will have no effect because the sim has already had its one look at the dataref.

This is a design limitation; it was never the intention of the SDK to allow complete Plane-Maker-level editing of the aircraft on the fly in the sim.

An Older Build for Regression Testing

I'm never thrilled about posting bug-swatting info on this blog; the blog is (among other things) a temporal snapshot of what's going on in X-Plane, as well as an archive; it's my hope that we'll get this problem sorted out soon, at which point this blog post will do nothing but confuse. But I digress.

There have been a number of reports from users of the sim hanging on startup with this configuration:

• A 64-bit Windows (usually Vista or 7).
• A modern ATI card running Catalyst 10-6 or 10-7 drivers.
• X-Plane 9.62rc2.
• Usually a core i7 type system.

However, I haven't been able to reproduce this, and neither has ATI.

I don't know what the problem is, but a number of variables have changed in this equation that need to be isolated: new sim, new video drivers, newer operating systems.

So if you have this configuration and can't run the sim, despite removing all third party add-ons, please download this time demo. If you can run the 945 time demo but cannot run 962, please let me know by email, and we'll isolate a defect in the sim. I have heard from some users that they can run 940, but no confirmation that 945 runs.

Procedural Or Algorithmic

A quick thought on procedural vs. algorithmic scenery: there was some discussion on the X-Plane dev list about procedural terrain; the Outerra screen shots have stirred up interest.

There is a fundamental difference between what you see with Outerra and what we do with our global scenery:

• Typically procedural scenery is based on the idea of 'amplifying data' - that is, making a little bit of data look more interesting by applying a recursive algorithm to "generate" detail.
• Algorithmic scenery involves taking a large amount of unique input data and translating it; the detail comes from not losing information in the source data.

The key difference is whether the resulting scenery comes from a process that "creates" information through a 'procedure' or "translates" information.

Our scenery process does a bit of both, but we are more in the algorithmic camp than procedural camp; the global scenery is produced from hundreds of GB of input data, and we are constantly looking for better input data to create more interesting and accurate output scenery.

In fact, I would say that we are becoming more algorithmic and less procedural. In version 9, the urban roads in non-US cities are "procedural" - an algorithm generates them from the terrain data, an algorithm, and some random noise. For version 10, we are importing OSM.

One thing I have noticed in the work on version 10 global scenery is that we've finally crossed a line. With version 9, the question was 'what is the best data we can get'. With version 9, the question is 'how much information can we keep'; we've reached a point where the resolution of our input data is so much higher than what can go on DVD, that it's a question of filtering down, not synthesizing up the resolution.