Dual Core? Definitely

Austin hinted on the X-Plane news list that X-Plane will in the future use a second core to load scenery much more smoothly than it does in the past. Each time we take a whack at the scenery load code, we reduce the load time by a significant multiple, so we're still not at truly no-pause flying, but we will be getting a lot closer in the future.

A few years ago when people asked "should I get a dual CPU machine" (this was before dual-core chips were ubiquitous) I hesitated to say yes. This is on longer the case - dual core is definitely the way you want to go.

The price performance curve is such that dual core chips are definitely in the sweet spot. In the past a true dual CPU machine costed big bucks, but now unless you buy the absolute cheapest machine, you're going to get a dual core chip. I never recommend buying the absolute cheapest hardware -- usually to get those last few dollars of price savings, the machine (or GPU) has been stripped down of major features, so you're losing tons of performance for really marginal savings in dollars.

X-Plane's usage of the second core has gone up too, already in 860 and the second core will be even more important in version 9.

Our usage of the second core isn't about trying to raise X-Plane's system requirements, it's about not leaving "money on the table". We need to make X-Plane run efficiently as possible on as many combinations of hardware as possible. This means giving you the best flight sim experience you can get whether you have an older single-core machine with a low-end graphics card or a new dual-core chip with the latest huge graphics card and four tons of VRAM.

The important thing here is: usage of dual core chips requires code changes. We have to write new code in X-Plane to teach it how to safely run some work on one core and other work on another. This is all about efficiency - if you have a dual-core chip there is nothing efficient about leaving one core doing nothing! We'd be leaving half the computing power of the chip idle.

So if you don't have a dual-core chip, the sim will still run (just as it does now). But whereas the gap between single and dual-core chips is smaller now (because the dual-core users have half a chip idle a lot of the time), the gap will become larger as we utilize the second core more fully and dual-core users get to really take advantage of all of that computing power.

Multicore and Version 10

Over the last two years, we've been working on the X-Plane rendering engine, setting up the code to take advantage of multiple cores when possible. Most of these changes are in X-Plane 9 already: multi-core creation of 3-d scenery, multi-core loading of scenery, multi-core loading of textures.

X-Plane 10 will leverage this work, pushing even more work to multiple cores. And yet, nine women cannot have a baby in one month. The ultimate limit on framerate will be based on the performance of one core pushing data to your graphics card.

So how many cores do you need, and is it better to have a few fast cores or more slow cores?

I can't give you a firm answer, because I don't know how important money is to you, I don't know which rendering settings you care most about, and X-Plane 10 isn't finalized. (And even if it was, we often improve performance in patches.) But I can suggest our attitude to how cores are used.

A Graphic Analogy

With graphics cards, the companies target different markets. The "enthusiast" market is the top end, where money is no object and maximum performance is the goal. Below that you have "performance" cards (a good value for the money but not as fast) and "mainstream" (which by video game standards means "slower than snot" - main stream users don't need fast 3-d graphics to check email).

When it comes to rendering features, we expect X-Plane to be efficient enough to meet the performance expectations of a given slice of the market. In other words, if you have bought a top-level graphics card, we need to be efficient enough to let you run at 8x FSAA on a huge screen with per pixel lighting - that's what is expected of the enthusiast crowd and that's what the cards are built for.

But if you have a mainstream card and get 5 fps with per pixel lighting, well, too bad. You've got a cheap, low powered card, you need to dial down the sim. Here our goal is only to give you a way to run X-Plane at all. (And frankly, if X-Plane could run at 60 fps on a mainstream card, it means the max rendering settings don't take advantage of top end hardware!)

Core Considerations

At this point we expect pretty much any modern computer to have at least two cores, and users with single core machines are going to have to make serious graphic quality sacrifices to run X-Plane. (This is already true for version 9.)

When it comes to version 10, I think we will categorize "a lot of cores" (e.g. 4, 6, or more) as enthusiast - some of the top level rendering options may not function well with less than four cores, but dual core machines should at least run decently with some rendering options enabled.

How far we go toward utilizing really high core count (Austin upgraded to the new Mac Pro and now has 8 hyper-threaded i7s) I don't yet know. My guess is that we will get at least some measurable benefit from up to 8 cores.

Trading off clock speed for core count is a difficult choice. Obviously if you had a choice of one core that was twice as fast as a 2-core CPU, you'd take the speed - you're getting the same total computing power but the clock speed can help frame-rate. But the trade-off is almost never formulated like this, and it's too soon to have hard data from the sim itself.

For what it's worth, the latest generation of CPUs is really fast, so it is unlikely that you'd upgrade to a new CPU and not get some serious benefit, whether it comes in cores or clock-speed. The users I have heard from with new iMacs seem quite happy with their machines.

Failed Ideas and Two-Core Rendering

I'm pretty gun-shy about posting new features to this blog before they are released.  One reason is that a fair number of the things I code never make it into the final X-Plane because they just don't perform as expected.  But the converse of that is: there should be no problem posting about what failed.

One idea that I believe now will not make it into the sim is dual-core pipelined rendering.  Let me clarify what I mean by that.

As I have blogged before, object throughput is one of the hardest things to improve in X-Plane. That code has been tuned over and over, and it's getting to be like squeezing water from a rock. That's where dual-core pipelined rendering comes in.  The idea is pretty simple.  Normally, the way X-Plane draws the frame is this:

for each object
is it on screen?
if it is tell the video driver, hey go draw this OBJ

Now the decision about whether objects are on screen (culling) is actually heavily optimized with a quadtree, so it's not that expensive.  But still when we look at the loop, one core is spending all of its time both (1) deciding what is visible and (2) telling the video driver go draw the object.

So the idea of the pipelined render is to have one core decide what's on screen and then send that to another core that talks to the video driver.  Sort of a bucket-brigade for visible objects. The idea would be that instead of each frame taking the sum of the time to cull and draw, each frame should take whichever one is longer, and that's it.

The problem is: the idea doesn't actually work very well.  First, the math above is wrong: the time it takes to run is the time of the longer process plus the waiting time.  If you are at the end of a bucket brigade putting out the fire, you waste time waiting until that first bucket goes down the line.  In practice the real problem though is that on the kinds of machines that are powerful enough to be limited only by object count, the culling phase is really fast.  If it takes 1 ms to cull and 19 ms to draw, and we wait for 0.5 ms, the savings of this scheme is only 2.5%.

Now 2.5% is better than nothing, but there's another problem: this scheme assumes that we have two cores with nothing to do but draw.  This is true sometimes, but if you have a dual-core machine and you just flew over a DSF boundary, or there are heavy forests, or a lot of complex airports, or you have paged-texture orthophoto scenery, then that second core really isn't free some of the time, and at least some frames will pick up an extra delay: the delay waiting for the second core to finish the last thing it was doing (e.g. building one taxiway, or one forest stand) and be ready to help render.

And we lose do to one more problem: the actual cost of rendering goes up due to the overhead of having to make it work on two cores.  Nothing quite gloms up tight fast inlined code like making it thread-safe.

So in the end I suspect that this idea won't ever make it into the sim...the combination of little benefit, interference by normal multi-core processing, and slow-down to the code in all cases means it just doesn't quite perform the way we hoped.

I am still trying to use multiple cores as much as possible.  But I believe that the extra cores are better spent preparing scenery than trying to help with that main render.  (For example, having more cores to recompute the forest meshes more frequently lowers the total forest load on the first CPU, indirectly improving fps.)

Multi-Core and X-Plane 8 and 9

To answer some speculation and questions I've seen on the forums regarding X-Plane and multiple cores:

• X-Plane 8 and 9 will use up to four cores to "shift" the scenery coordinate system when scenery needs loading. This shift causes a pause in both X-Plane 8 and 9, but it is a relatively short pause.
• X-Plane 8 and 9 will use a second core to build 3-d geometry out of road, forest, facade, and airport definition data while you fly, as you move through the loaded scenery area. This is particularly important to smooth flight when there is a lot of 3-d enabled.
• X-Plane 8 and 9 will use a second core to do some of the work of loading textures from disk while you fly. (Right after a scenery shift happens, over time new textures are loaded in.) Some of that texture load is still done on the main thread. Texture compression is done on the main thread, which can cause stuttering. (I expect to address this in the future, but authors can use DDS textures now in v9 to simply not do the compression at all and make everything faster.)
• X-Plane 9 will load new DSFs into memory on a second core. Loading new DSFs takes the vast majority of scenery load time, thus flight with X-Plane 9 and a dual-core machine is a lot smoother than it was in v8. This is a big win for the dual-core machine and is new to v9.

All of our new work on the sim keeps multi-core in mind. We know that more and more users will have multiple cores and that we're leaving performance on the table if we can't distribute rendering load across two cores.

So the important points here I think are:

• X-Plane 9 significantly uses a second core.
• Use of the second core has grown significantly over time, and will increase in the future.
• I like to use bullet points way too much in my blogs and emails.

How Many Cores Will You Have?

My last post generated  number of posts from both sides of the "hardware divide" (that'd be the have's and have-not's).  I think everyone at least grasps that developer time is finite and features have to get prioritized at the cost of other features, even if not everyone agrees about what we should be coding.

I think the term "hardware divide" is the right one, because the hardware market has changed. Years ago when I bought myself a nice shiny new Dell (back when that wasn't an idiotic idea) a medium-priced Dell had medium-priced hardware.  Not only did I get a decently fast CPU (for the time), but I got a decent AGP bus, decent motherboard, etc.  The machine wasn't top-end, but it scaled.

When you look at any computer market, you need to consider what happens when consumers can no longer accept "more" and instead want "the same for cheaper".  This change in economics turns an industry on its head, and there are always winners and losers.  (I have claimed in the past that operating systems have turned that corner from "we want more" to "we want cheaper", a shift that is very good for Linux and very bad for Microsoft.)

Desktop computers hit this point a while ago, and the result is that a typical non-gamer computer contains parts picked from the lower end of the current hardware menu.  You're more likely to see:

• Integrated graphics/graphics by the chipset-vendor.
• System memory used for VRAM.
• Slower bus speeds, or no graphics bus.
• GPU picked from the lowest end (with the fewest number of shader units).
• CPUs with less cache (this matters).

Someone commented a few days ago that computers would get more and more cores, and therefore multi-core scalability would be very important to fully utilizing a machine.  I agree. 

But: how many cores are those low-end PCs, aimed for general use (read: email, the web, text editing) going to have?

My guess is: not that many.  Probably 2-4 at most.

These low end PCs are driven by one thing: price - the absence of VRAM or dedicated graphics hardware is all about bringing the hardware costs down - a $25 savings matters!  In that situation, box-builders will want the cheapest CPU, and the cheapest CPUs will be the physically smallest ones, allowing for more chips on a wafer.  A low-end PC will get no benefit from more than 4 cores - the intended use probably doesn't even use one.*

Multiple cores are great because they give us a new way to benefit from smaller transistors (that is, by packing more cores on a chip, rather than clocking it faster, which has real limitations).  But I think you'll start to see the same kinds of gaps in CPU count that you see now with GPUs.

(In fact, the mechanics are very similar.  The main differences between high-end and low-end GPUs of the same family are the number of parallel pixel pipelines - the low-end chip is often a high-end chip with a few defective pipelines disabled.  Thus you can have a 4x or 8x performance difference due to parallel processing between siblings in a GPU family.  Perhaps we'll see the same idea with multi-core chips: build an 8-core chip, and if 4 of the cores fail, cut them out with the laser and sell it as a low-end chip.)

* One advantage of multiple cores is that they can take the place of dedicated hardware.  For example, there is no penalty for doing CPU-based audio mixing (rather than having a DSP chip on the sound card) if the mixing happens on a second core.  Being able to replace a dedicated component with a percentage of a core is a win in getting total hardware cost down, particularly if you were going to have the second core already.

Multi-Core Texture Loading

In a previous post I discussed the basic ideas behind using multiple threads in an application to get better performance out of a multi-core machine.

Now before I begin, I need to disclaim some things, because I get very nervous posting anything involving hardware. This blog is me running my mouth, not buying advice; if you are the kind of person who would be grumpy if you bought a $3000 PC and found that it wouldn't let you do X with X-Plane (where X includes run at a certain rendering setting, framerate, or make your laundry) my advice is very simple: don't spend $3000. So...

• I do not advocate buying the biggest fastest system you can get; you pay a huge premium to be at the top of the hardware curve, particular for game-oriented technologies like fast-clock CPUs and high-end GPUs.
• I do not advocate buying the Mac Pro with your own money; it's too expensive. I have one because my work pays for it.
• 8 cores are not necessary to enjoy X-Plane. See above about paying a lot of money for that last bit of performance.

Okay...now that I have enough crud posted to be able to say "I told you so"...

My goal in reworking the threading system inside X-Plane for 920 (or whatever the next major patch is called) is, among other things, to get X-Plane's work to span across as many cores as you have, rather than across as many tasks are going on. (See my previous post for more on this.)

Today I got just one bit of the code doing this: the texture loader. The texture loader' job is to load textures from the hard drive to the video card (using the CPU, via main memory) while you fly. In X-Plane 901 it will use up to one core to do this, that core also being shared with building forests and airports.

With the new code, it will load as many textures at a time as it can, using as many cores as you have. I tested this on RealScenery's Seatle-Tacoma custom scenery package - the package is an ENV with about 1.5 GB of custom PNGs, covering about half of the ENV tile with non-repeating orthophotos.

On my Mac Pro, 901 will switch to KSEA from LOWI in about one minute - the vast majority of the time is spent loading about 500 PNG files. The CPU monitor shows one core maxed out. With the new code, the load takes fourteen seconds, with all eight cores maxed out.

(This also means that the time from when the scenery shifts to when the new scenery has its textures loaded would be about fourteen seconds, rather than a minute, which means very fast flight is unlikely to get to the new area before the textures are loaded and see a big sea of gray.)

Things to note:

• Even if we don't expect everyone to have eight cores, knowing that the code can run on a lot of cores proves the design - the more the code can "spread out" over a lot of cores, the more likely the sim will use all hardware available.
• Even if you only have two or four cores, there's a win here.
• Texture load time is only a factor for certain types of scenery; we'll need to keep doing this type of work in a number of cases.

This change is the first case where X-Plane will actually spread out to eight cores for a noticeable performance gain. Of course the long-term trend will be more efficient use of multi-core hardware in more cases.

More Threads

X-Plane 940 beta 1 is out...it will take a little bit to get the release notes and docs on the website completely up-to-date. We're still dealing with loose ends from our migration to the new web site, and most of my office is packed up for a move to the Boston area. I'll try to get docs up as fast as I can given the chaos flying about.

Given the interest multi-core stirred up in previous notes, I will mention one change to 940: with this build we've added yet more multi-core to X-Plane.

In 931, X-Plane will use as many cores as you have to load textures, but only one to build "3-d scenery" (a loose category for the work we do when we make airport taxiways and lines, build forests, and extrude roads).

In 940, this "3-d scenery" is also done on as many cores as you have. This should speed up load times a bit, particularly under very heavy tree settings, and hopefully keep the forest engine running faster for users with more cores.

It also sets us up for long-term growth; X-Plane's visual quality is sometimes limited by the time to build 3-d meshes...being able to do this work on many cores means we can use higher quality algorithms.

Consider for example the roads: my original "road extruder" (the code that converts a vector road into a 3-d model, called an extruder because it builds a 3-d road from a cross section like one of those play-dough toys) made beautiful intersections with stop-lines and cross walks and lots of other great stuff.

It was also really slow. And at the time the sim wouldn't fly at all while roads were extruded, so speed was of primary concern. So I replaced it with the much dumber extruder you see today, where intersections are basically ignored.

Now that we have 3-d scenery build on multiple cores, we can begin to provide rendering options that take more CPU time but produce higher quality results. The trees and airport layouts already do this (in that they take more time and produce slower, more detailed, higher triangle count sceneery at higher rendering setting for the same input DSF ad apt.dat file). With more cores, we can continue this strategy with roads and other parts of the sim without worrying about overloading the one core that was doing this work.

Of course just because we can use 8 cores doesn't mean we do...you won't see 8 cores maxed out very often, particularly if you have simple scenery and a very fast machine.

Instability in Version 9

One of the reasons why the X-Plane 9 betas have had so many more crash bugs than version 8 is that we introduced loading DSFs on a second core. This feature makes scenery loads much slower and (by using the second core) impacts fps less while they happen.

The problem is that I'm still fumbling with code that will allow this in all cases. (That would be three operating systems, two hardware vendors, and a myriad of drivers, some new, some quite prehistoric.)

Beta 22 will be out soon, and will contain the fourth major rewrite of the OpenGL setup code for X-Plane 9. So far the initial tests look good, but we never know until we let a lot of users try the code and find the new edge cases.

It's relatively easy to tell if your instability is related to the use of OpenGL with threads: simply run the sim with the --no_threaded_ogl option. If things become a lot more stable, it's a threaded GL problem. Mind you --no_threaded_ogl is more of a diagnostic than a workaround; without threaded OpenGL, the sim will pause when loading scenery.

(Also, to clarify, you'll find talk on discussion groups and game forums about "threaded drivers". Threads are a programming abstraction that can utilize multiple cores. What I am talking about is X-Plane using multiple threads to load scenery - in our case this requires interfacing with OpenGL. But a threaded driver is different - it's just a graphics driver that's been optimized for multcore machines. These two concepts are totally different; you don't need a threaded driver to use X-Plane 9, and a threaded driver won't make X-Plane 8 load without pauses.)

Plausible, Realistic, Procedural and Algorithmic

Alpilotx pointed me toward a thread on the org discussing Austin's work on the weather system. The thread turned into a bit of a he-said-she-said with regards to Outerra and whether it could some day be combined with X-Plane.

This blog post will be a discussion of various general approaches to scenery and the trade-offs we have to consider, e.g. plausibility and realism, procedural vs. algorithmic and data driven design. But first, a brief note on Outerra. As I have said before, we are already aware of Outerra, so there is no need to email us. The bottom line is that we have a set of mostly done features for X-Plane 10, our goal is to finish X-Plane 10, and we are not even spending one brain cell considering putting a new rendering engine into X-Plane while we are trying to get 10.0 done.

Defining Some Terms

One of the problems with comparing scenery system approaches is that a real productized approach to scenery rarely fits into a perfect bucket or matches a single theoretical techniques. So here are some approximate terms, designed to generally describe an approach. They're not going to be perfect fits, and even the definitions will fluctuate in different contexts and forums.

• We can say scenery is plausible when it looks like it might exist somewhere in the world. Plausible means that roads don't go straight up over a cliff, trees don't grow in the ocean, etc. In other words, plausible scenery is scenery where absurd things don't happen. Plausible scenery is great when you don't know what an area should look like. A lack of plausibility is often a bug.
• We can say scenery is realistic when it correlates closely with what is really present at a given location on the Earth. So if there really is a lake behind my house, realistic scenery has that lake. Plausible scenery might have a lake, a forest, or something else believable for where I live (the Northeastern United States). A giant sandy desert would not be plausible for my location.
• We can say scenery is procedural if the detail in the scenery comes from some kind of algorithm that produces results. For example, a fractal coastline is procedural.
• We can say scenery is data driven when the detail comes from some source of external input data. Our mountains are currently data driven - that is, the mountain shape basically comes directly from the DEMs we use.
• We can say scenery is artist driven if the look of the scenery comes from art assets created by an art team.
• We can say scenery is algorithm driven if part of its look comes from the transformational process that converts data from one form to another.

(I'm sort of drawing a line in the sand here with procedural vs. algorithmic, but what I'm trying to contrast is a program that generates 'information' out of thin air vs. a program that creates information out of other information. For example, in X-Plane 9, European capillary roads were procedural. We had no real data, so I wrote an algorithm that made them up in a manner that was consistent with underlying terrain. In version 10, these roads will be algorithmic; we take OSM data and then do some processing to make it suitable for X-plane. This is definitely a line in the sand kind of definition.)

So Are We Plausible or Realistic?

So the first question is: is the goal of X-Plane global scenery plausibility or realism? The answer is: a bit of both. Austin's posts on the subject virtually always bring up plausibility. The reason for this is simple: he is not too worried about the amount of realism we've put into the scenery, but he is not happy with the bugs. He wants the bugs gone. So every time he and I speak, he says "and make sure it's plausible!"

But we're not going to remove realism just to fix plausibility bugs. I expect that the next global scenery render will be at least as realistic as the last - that is, we're going to use better data and we're not going to make up data where we had real information before.

There are limits to realism. We don't expect the global scenery to ever be as realistic as a custom scenery package for a small matter. But realism does matter. Part of the joy of flying in a flight simulator is seeing the real world. Where we can have more realistic global scenery, we consider it to be a win, and we are always looking to be more realistic than the last render.

Plausibility for the version 10 render is going to take two forms:

1. Bug fixes. Any time something screwy happens, it's not plausible. Sometimes these are code bugs that must be fixed, and sometimes they are data conflicts. For example, the water data sasys "water" but the elevation data says "hill". Combine them and you get water going up a hill. We have to write code to resolve this, somehow.
2. We are reworking the way cities are rendered, because even at their best, the old approach, procedural buildings with algorithmic roads over land class photos, did not look plausible, even at its very highest setting. So this is a feature request to fix a plausibility problem.

Algorithmic or Procedural

I've discussed this before (and forgotten about the post). But to expand the discussion, we need to consider not only algorithmic and procedural data processing, but whether we are driven by procedural generation, input data, assets created by artists, or some combination. (In practice, all systems require a mix of data, art assets, and procedures and algorithms, it's a question of the blend.)

I've been working on global scenery for a few years now, and over time I've come to appreciate the importance of artist input (via art assets) into any scenery process. Simply put, if you want scenery to look good, you need to make it reasonably straight forward for people who are good at making pretty pictures to control the look of your visual results. A few years ago I viewed the scenery process as strictly a question of data conversion and visualization, but now I see it as finding a way to merge art assets and data into a cogent final product, with the art assets being used in a way that the artists can control. In practice, this often means making sure that the art assets come in a format that artists are comfortable with or can learn without too much pain.

As I said in the previous post, our approach is becoming more algorithmic and less procedural as higher quality source data becomes available. (For example, we don't have to generate European roads when we can import and reprocess them.) But our approach over time has always been heavily artist driven. By this I mean: our input data is algorithmically processed into a final form that makes sense only in the context of art assets, and we have a pretty good idea of what those art assets will look like when we design the algorithms. To use roads as an example again, our task with OSM is to convert OSM road data into a road network that will visualize nicely with road art assets created by an artist.

Procedural Compression

One way to view procedural scenery is "creating lots of information from little or no information". But another way to think of it is as a compression technology. As was correctly pointed out on the org forums, you use less storage specifying the overall location of a forest than you do specifying every tree individually. The compressed form (store the forest location) can be equally plausible. It will be less realistic if the original tree locations were based on real world data, but it will be equally (unrealistic) if the original tree locations were procedurally generated. Put another way, pushing procedural processes out of the scenery generation process and into the flight simulator makes DSFs smaller.

When I first started working on X-Plane 8 DSF scenery, not only was DVD size a factor, but so was load time; we had one core and it wasn't a very fast core. Anything we could do to make loading faster, we did. Thus we pushed a lot of work into the scenery generation process, including procedural processes, to keep load time down.

Times have changed; we now have dual core machines as a baseline, and often quite a few more cores. Thus over time we are starting to move procedural processes back into the simulator, trading load time (which runs on multiple cores) for generation time and file size. So perhaps a more accurate statement would be: our scenery generation process is becoming more algorithmic and less procedural, and X-Plane itself is becoming more procedural. This is driven both by more input data (which must be processed up front) and more compute power on the host (which lets us shrink file size, and thus use DVD space for other things).

X-Plane 10

Here's how this plays out in practice in version 10:

• Some (but not all) of the building placement work* has been moved into X-Plane; a bit of expensive precomputation is still done at DSF generation time.
• Some (but not all) of road processing has been moved into X-Plane; a lot is still done at DSF generation time.
• Where possible, we are moving from a multi-layered approach to terrain to a pixel-shader-based approach to terrain. This cuts down overdraw and uses the GPU more efficiently. (The simplest example: in X-Plane 8 and 9, cliffs have separate terrains from hills. In X-Plane 10, a single terrain sits on both the cliff and the hill and changes its appearance based on the actual slope; this texture change is computed by the GPU.)

In other words, X-Plane 10 is making the logical evolution to better balance the computing resources we have to improve plausibility and realism.

More Scenery Features

Since X-Plane 9 went final I've been going in about 5 different directions with the code, and as a result my posts have diverged pretty far from my charter within the company, namely scenery. Here's some of what I'm looking at on the scenery front:

Texture Paging

Texture paging is a new feature where X-Plane can change the resolution of orthophotos while you fly. The big limitation of orthophoto-scenery right now is that X-Plane loads every orthophoto at the maximum user-defined resolution when the underlying DSF/ENV is loaded. This can cause X-Plane to run out of virtual address space and crash. With texture paging, only the nearby textures are loaded at high resolution.

Far away textures may be low res, but you'll never notice because they are far away (and thus you're seeing them over only a few pixels on screen anyway.

The cost of this technique is that textures are being loaded over and over and over. But this cost is made less painful by two features:

• DDS texture loads are very fast and cheap, especially at very low resolutions.*
• With the next major patch, texture loading will be entirely on a second core (if you have one) and can even span multiple cores.
  

This feature will require modification of scenery packs in that you'll have to add some "paging" information to your .ter files; I will try to get a preliminary spec posted soon. Because you only have to modify text files, it should be possible to create "paging patches" to existing DSF orthophoto sceneries that are fairly small.

I do not know if paging will be available for .pol files. My guess is yes, but I reiterate that using .pol files for large areas of orthophotos is a bad idea! Make a new mesh!

Improved Multi-Core Performance

This is going to be an on-going process through the entire v9 run and beyond, but basically with every change to our rendering engine (and we make some serious modifications with almost every patch to keep it modern and competitive) we try to move toward using the new hardware (meaning machines with 2-4 cores) more efficiently. Some of this will be in 920, although my guess is we'll need at least one more patch after 920 to really see the improvement.

Tools

It's going to be a little bit before I can put more time into the various scenery tools. My top priority is to keep up with user feedback and work on MeshTool. Hopefully we'll also declare WED final soon. But for now, since I am working on cockpit and airplane modeling features, my next set of work will probably be for the airplane authors.

Shaders

I do want to put additional shader work into v9. I realize I am at the edge of provoking a bunch of rants in the comments section, so let me save you some time:

"Hey Ben, stop working on eye candy and create some more tools. I don't want a shader feature that makes my 1.2 ghz G4 with a GeForce 2 MX run any slower. You should finish the tools so they do exactly what I want!"

Okay, I admit, that was totally unfair...there is a lot of truth in the complaints about shaders vs. tools.

• I really do try to keep an eye on system requirements, particularly once we shipped. I'm going to try to prioritize shader features that improve existing rendering techniques, rather than introduce new rendering techniques, so we don't seriously lower fps during the version run. But also bear in mind that shaders can be turned off, and there are users who have GeForce 9s and such.
• Tools are very important, hence the effort to get MeshTool out. But tools without engine work aren't very useful either; most of the engine work we do is needed to keep performance up so that new scenery (made with those new tools) don't bury the sim. For example, MeshTool without texture paging would be a dead-end...you could easily make a MeshTool scenery that you can't fly.

So in planning what goes out when, I look for clumps of features and tools that can go together to make some sense: WED to use apt.dat 850, texture paging to go with MeshTool. It wouldn't make sense to defer texture paging to make the next tool while MeshTool is waiting for engine enhancements.

* A DDS already has all of the smaller-version pre-compressed textures in the file. So loading a DDS at low res involves loading a small amount of data from disk to memory and sending it to the graphics card.

By comparison, a PNG file only contains the maximum size, so to load a PNG at low res, we load the largest size, shrink it on the fly, then apply DDS compression.

Next-Gen Cities

A few days ago Tyler posted some pictures of the "auto-gen"* for X-Plane 10's global scenery.

First, a few notes on hardware and performance: Propsman took these pictures on a new iMac, so that's a core i5 and a Radeon HD 5000 series GPU - that is, a pretty decent system. Hardware technology continues to advance rapidly (especially on the GPU front), so there's a big difference between a 'decent' system bought today and even a hard core system from two years ago.

I don't know what your performance will be like. I do know that the system is performing well enough so far in its not-really-all-that-optimized form that we think we can ship it, and more importantly I know that we can turn the level of detail down in a number of ways to lighten the load as needed.

The most expensive feature you see here is the real-time 3-d global shadows. Heavy shadowing combined with heavy 3-d does add up and hit the system hard, but I think we'll be able to have intermediate shadow settings that should be more affordable.

X-Plane 10 will use hardware instancing if your GPU is capable of it, and it makes a big difference in the amount of 3-d you can show.

X-Plane 10 is also quite a bit more fill-rate intensive than X-Plane 9; if your GPU is having fill-rate problems with version 9, some version 10 features will be out of reach. In the past, X-Plane has been light on fill-rate, so we've had users running with cut down cards (like a GeForce 8400) without realizing that their card isn't that fast.

Some users have asked about architecture and localization. I expect we will not ship out of the box with multiple local regions; however, the library system allows us (or any third party) to provide new artwork sets for local, architecturally reasonable buildings.

Finally, it might be a bit difficult to see in these pictures (because they are focused in on the detail), but the 3-d buildings you see here work with the real-world roads. In the past, we've had a clash between the buildings and roads vs. the terrain texture. This is a problem we are solving for X-Plane 10.

* Auto-gen, meaning bulk buildings that populate the world in urban areas...whether it's really auto or gen or anything like autogen in the past is a complex discussion that will have to wait.

More Threads - the Installer

Nine women cannot have a baby in one month - that's the classic example that gets thrown around computer science for the difficulty of parallelization - that is, just because we have ten times as many resources doesn't mean we're going to go ten times as fast.

Problems of scalability via parallelization have become very important for graphics engines as everybody and their mother now has at least two cores, and users with more serious hardware are going to have four.

I get asked a lot: "will X-Plane utilize X cores..." where X is a number larger than two. My general answer is: sometimes, maybe, and probably more in the future. I can't make strong predictions for what we'll ship in the future, but the general trend for the last 18 months has been us using more cores every time we go into a piece of code to do major architectural changes.

I've been doing a lot of work on the installer this week - the first major overhaul of the installer since we originally coded it all the way back at X-Plane 8.15. And the new installers and updaters will try to take advantage of multiple CPUs where possible. A few cases:

• The X-Plane updater runs an MD5 checksum over the entire X-Plane folder to determine which version of the various file components you have and whether they need to be updated. This s not a fast process. I am working on threading this so that more CPUs can work on the problem at once. It looks like there will be only modest benefits from this because the process is also highly bottlenecked on the disk drive.
• The installation engine from the DVD will use more than one CPU to decompress files. For zip compression this wasn't very important, but the scenery will be compressed via 7-zip compression to get us down to disk DVDs. 7-Zip compresses DSFs about 10% smaller per file than zip, but it's horribly slow to decompress, so being able to throw twice the CPU at it is a big win.

Now on one hand, our top performance goals are for the sim, not the installer. On the other hand, faster installations are good. But my main point here is: when we wrote new code four years ago, we assumed one CPU and a nice graphics card. We now assume at least two cores and possibly more, and that informs the design of every new feature, not just the rendering engine. If we don't create a multi-core-friendly design, we're effectively leaving at least 50% of the CPU on the table.

A Change in Settings - 3-d Lights

For X-Plane 860 beta 6 I think we may change the way the airport lights setting works. Since
(despite my previous rantings) there is a lot of discussion of performance based on settings (e.g. "I used to get 40 fps with this menu pick, now I need this other menu pick") as opposed to based on what the sim is actually doing, allow me to go off in some detail about what's going on under the hood.

During X-Plane 850 and 860, I tried to fix a number of long-standing "quality" issues, where the sim had small artifacts. There is a fps penalty to fixing these artifacts that I thought was small, but the message is clear:

• Some users are flying X-Plane on lower-end systems and can't spare a single fp.
• A lot of users really don't care about the quality issues and would gladly trade back the visual "improvement" (which doesn't seem like an improvement after living with the issue for 4 years) for those few fps that mean the difference between clear skies and fog.

So...with that in mind, let me explain runway lights for 3 versions of X-Plane.

X-Plane 840 and earlier
In X-Plane 840, lights could be textured or untextured. This was controlled by a simple checkbox "draw textured lights". A few lights (on the airplane, for example) were always textured because we thought they were few in number and important visually. The rest was decided by the checkbox.

No airport light ever had a 3-d "structure" (e.g. you couldn't see the light housing and support rod).

No light was ever drawn using hardware acceleration, even if the graphic card had pixel shaders; we simply didn't have the code. Therefore textured lights ate up a lot of CPU power and thus ate up a lot of framerate. But life was simple!

X-Plane 850
With X-Plane 850 things got a lot weirder.

• We rewrote the lighting engine to optionally use pixel shaders if present. This improves performance of textured lights a lot. In fact, on a given machine with pixel shaders, textured lights with hardware acceleration are faster than non-textured lights without hardware acceleration. (For hardware accelerated lights, the texturing doesn't cost anything compared to hardware accelerated untextured lights.) It's important to note this performance fact!
• Sergio created a bunch of 3-d object lighting fixtures for the runway environment. They have a low LOD, but up close you can see the actual lighting structure.

So what to do about that setting? Well, basically with the check box on we did the nicest looking thing, and with the check box off, we did the fastest thing. (The check box's label doesn't do a good job of representing this.) We thus have four possibilites:

1. Checkbox off, no pixel shaders: we draw untextured lights using software, no objects.
2. Checkbox off, with pixel shaders: we draw textured lights using hardware, no objects.
3. Checkbox on, no pixel shaders: we draw textured lights using software, with objects.
4. Checkbox on, with pixel shaders: we draw textured ilghts using hardware, with objects.

This is plenty confusing if you ask yourself "what is this checkbox going to do" - it depends on your hardware. But the rule is quite simple if you consider the intent: we either give you the fastest (off) or nicest (on) rendering we can.

X-Plane 860
With the next X-Plane 860, we're going to use this checkbox to control multiple aspects of the sim, but always with two choices: either prioritizing visual quality, or prioritizing framerate.

So this option will not only pick the fastest (or nicest) light code and turn the objects on and off, but it will also, for example, turn on or off per-pixel fog (something that looks nice but is slower than the old per-vertex fog).

I'm not sure everyone will like this, but I think it will meet some important needs:

• It will make it obvious how to set the checkbox. Either you want speed, or quality.
• It keeps the config interface simple for new users.
• It gives us a way to tie in a number of small optimizations that add up when taken in concert.

It seems to me that we're seeing the haves and have-nots. If you've got a Core 2 CPU and a 7000-serise GPU, things like per-pixel fog are chump change. If you've got a G4 with a Radeon 9200, you need any speed you can get. So hopefully this change will make the UI clearer and help us meet everyone's needs.

Fun With Servers

Just an FYI: when it rains it pours. Normally betas increase load on our set of update servers. To compound this, one of them is suffering a midlife crisis^H^H^H^H^H^H^H^Hhard drive failure.* We're working on it now; hopefully it will be resolved in the next 24 hours.

EDIT: the update server is back up - our host not only swapped out the drive, but the whole box. We'll have to take it down one more time in the future, but for the most part I think we're out of the woods.

Another note on servers: Chris has restructured X-Plane for Android to separately download the art assets from our servers, rather than contain all art assets in the actual download. What he found after several painful weeks was that the Android store is not yet reliable for large apps. While the official app size limit is 50 MB, many phones have problems with their configuration that cause downloads to fail. When the user buys our app and the download fails, they get angry at us. (X-Plane may have been, until it was restructured, one of the largest Android game APKs. The other games with large amounts of 3-d content were already doing separate downloads.)

We originally wanted to build a monolithic app (everything in the APK) because we thought that this would provide the simplest, easiest configuration to maintain, and thus hassle-free installation for our users. You get the APK, you install it, you fly! Unfortunately, the Android Market isn't reliable for such a large download, so we had to re-evaluate.

The new system downloads only the core app from the Android Market and then pulls the art assets from one of our servers. So far this appears to be an improvement. If/when Google provides an integrated solution, we will probably switch back to it to simplify the process again (right now we have two points of failure: the Android Market and our server farm, which, per the above notes, sometimes does fail). But for now, we'll host the apps and try to give people the best download experience we can.

Finally, I will try to roll out at least a beta of new installers some time this week. The new installer simultaneously downloads from multiple servers, with a more efficient HTTP implementation; this should hopefully result in better download times and also lower server load per demo.

* Chris pointed out: most normal humans don't know what this ^H^H^H^H is about...it's nerd-speak for the delete key, e.g. to undo a text. ^H is control-H, which you may find works just like the delete key. Yes, I'm a huge nerd.