Too Many Computers

The New Mac Pro came today. I'm very excited about this computer because it may be the first computer that can render an entire set of global scenery on its own in under three days. We've always had to apply multiple computers to the problem, sometimes in different cities, and synchronize the data. Being able to do the render on one box is a nice simplification.

A few immediate observations:

• Man is it expensive. If you aren't Warren Buffet and your work isn't buying it, think long and hard about why you need it. The iMac is a more reasonable X-Plane computer. (And from what I understand, both Radeon HDs and GeForce 8's have driver problems on OS X; until Apple fixes this, consider BootCamp. With the new iMacs you can pick which next-gen card with "work-in-progress" drivers you want.)
• Buy third party parts...not only will you save money, but you'll get the joy of installing them. Since the "blue and white G3" Apple's cases have been on par with well-designed PC cases for accessibility. The Mac Pro is nicer though - hard drives and memory are all installed on rail-guided slide-in parts; two hard drives and two DIMMs took about 5 minutes and a single screw-driver.

My office is going to explode. Here's a rough list of all of the Macs we have in the house now:

• Mac Pro (the new scenery rendering machine, also graphics with 8800, will be triple-boot).
  
• Aluminum iMac (mostly for testing graphics code on Radeon HD, but it's now the DVD burning station, runs Ubuntu 7.10).
• Mac Book Pro (old X1600-based, portable dev machine, triple boot but LILO is dead).
• G5 (the old rendering machine, kept around to regress bugs on PPC, R300 chipset, OS X 10.3, etc.).
• Dell P-IV 2.6 ghz - mostly used to record music, but it does have a GeForce 6 in it, and it's a good low-end test system. (A 2.6 ghz P-IV doesn't get you real far, the FSB is slow, and it's only got 4x AGP.)
  
• G4 laptop (800 mhz) - case is falling apart, but if I ever needed to try to run on a really low-end system. Actually at this point the laptop is so far below min specs that it probably isn't worth it.
• Mac Book - got this for my wife, but if I begged she'd let me regress Intel X3100 bugs on it.

So there are two points to note here, and neither of them are "Ben has a lot of computers" and "Ben's office is a mess."

1. X-Plane 9's use of graphics hardware has caught up enough with the bleeding edge that I now basically have to have "one of everything" to really debug the sim. We'll have some major updates to shaders in future patches, so having accumulated all of this hardware in-house (a lot in the last few months) will help me debug these things faster. A lot of these setups are due to "X doesn't work" bug reports specific to certain hardware/OS combinations.
2. It's really handy that Macs now have x86 chips because it cuts down the number of boxes. The new Mac Pro will give me an nVidia DX10-type platform not only for OS X but also for Vista (first Vista machine in the house, not because I want Vista, but because someone in the company should have at least one copy) and Linux too, if I can make it work.*

So the next time we release a beta and it immediately crashes your computer, please bear with us. It really did work bug free on some computers we own...we're getting closer to a complete matrix to catch more of these incompatibilities early on. But even with that huge mess of hardware we're still missing a lot of combinations.

* Triple-booting Mac/Win/Linux is a much bigger PITA than double boot. The problem is that the old MBR-type setups only give you four partitions, of which one gets eaten for the firmware. That leaves you three partitions and three operating systems, but by default Linux wants a second one for swap (a good idea). So you need to either stick a fifth partition
on and fix the MBR by hand or reconfigure Linux to use on-main-volume swap-space. In summary, with only two operating systems, eitiher Windows or Linux "just installs", but to put three on one drive, you get into customization.

Threads and Cores

Now that multi-core machines are mainstream, you'll hear a lot of talk about "threads". What is a thread, and how does it relate to using more cores?

Definitions

A "core" is an execution unit in a CPU capable of doing one thing. So an 8-core machine might have two CPUs, each with four cores, and it can do eight tasks at once.

A "thread" is a single stream of work inside an application - every application has at least one thread. Basically a two-threaded application can do two things at once (think of driving and talking on your cellular phone at the same time).

Now here's the key: only one core can run a thread at one time. In other words, if you have an eight core machine and a one-thread application, only one core can run that application, and the other seven cores have to find something else to do (like run other applications, or do nothing).

Two more notes: a thread can be "blocked" - this means it's waiting for something to happen. Blocked threads don't use a core and don't do anything. For example, if a thread asks for a file from disk, it will "block" until the disk drive comes up with the data. (By CPU standards, disk drives are slower than snails, so the CPU takes a nap while it waits.)

So if you want to use eight cores, it's not enough to have eight threads - you have to have eight unblocked threads!

If there are more unblocked threads than cores, the operating system makes them take turns, and the effect is for each of them to run slower. So if we have an application with eight unblocked threads and one core, it will still run, but at one eighth the speed of an eight core machine.

It's not quite that simple, there are overheads that come into play. But for practical purposes we can say:

• If you have more unblocked threads than cores, the execution speed of those threads slows down.
• If you have more cores than unblocked threads, some of those cores are doing nothing.
  

Trivial Threads

When a thread is blocked, it does not use any cores. So while X-Plane has a lot of threads, most of them are blocked either most or all of the time. For all practical purposes we don't need to count them when asking "how many cores do we use". For example, G1000 support is done on a thread so that we keep talking to the G1000 even if the sim is loading scenery. But the G1000 thread spends about 99.9% of its time blocked (waiting for the next time it needs to talk) and only 0.1% actually communicating.

What Threads Are Floating Around

So with those definitions, what threads are floating around X-Plane? Here's a short list from throwing the debugger on X-Plane 9.0. (Some threads may be missing because they are created as needed.

• X-Plane's "main" thread which does the flight model, drawing, and user interface processing.
• A thread that can be used to debug OpenGL (made by the video driver, it blocks all the time).
• Two "worker" threads that can do any task that X-Plane wants to "farm out" to other cores. (Remember, if we want to use more cores, we need to use more threads.)
• The DSF tile loader (blocks most of the time, loads DSF tiles while you fly).
• At least 3 threads made by the audio driver (they all block most of the time).
• At least four threads made by the user operating system's user interface dode (they block most of the time).
• The G1000 worker thread (blocks most of the time, or all the time if you don't have the G1000 support option).
• The QuickTime thread (only exists when QuickTime recording is going on).
  

So if there's anything to take away from this it is: X-Plane has a lot of threads, but most of them block most of the time.

Core Use Now


So how many cores can we use at once? We only need to look at threads that aren't blocked to add it up. In the worst flying case I can think of:

1. The main thread is rendering while
2. The DSF tile loader is loading a just-loaded tile while
3. One of the pool threads is building forests while
4. You are recording a QuickTime movie (so the QT thread is compressing data).

Yep. If you really, really put your mind to it, you can use four cores at once. :-) Of course, two cores is a lot more common (DSF tile loading or forests, but not both at once, and no QuickTime.

Core Use In the Future

Right now some of X-Plane's threads are "task" oriented (e.g. this thread only loads DSF tiles), while others can do any work that comes up (the "pool threads", it's like having a pool car at the company, anyone can take one as needed). The problem with this scheme is that sometimes there will be too many threads and sometimes too few.

• If you have a dual-core machine, having forests building and DSF loading at the same time is bad - with the main thread that's three threads, two cores; each one runs at two-thirds speed. But you don't want the main thread to slow down by 66%, that's a fps hit.
• If you have a four-core machine, then when the DSF tile is not loading, you have cores being wasted.

Our future design will allow any task to be performed on a "pool thread". The advantage of this is that we'll execute as many tasks as we have cores. So if you have a dual-core machine, when a DSF tile load task comes along while there is forests being done, the one pool thread will alternate tasks, leaving one core to do nothing but render (at max fps). If you have a four-core machine, the DSF load and forests can run at the same time (on two pool threads) and you'll have faster load times.*

* Who cares about load time if you're flying? Well, if you crank up the settings a lot and fly really fast, the loader can get behind, and you'll see trees missing. X-Plane is always building trees in front of you as you fly and deleting the ones behind you. So using more cores to build the forests faster means you're less likely to fly right out of the forest zone at high settings.

901RC1 Already? Try it!

Well, the paint is barely dry and we already have an X-Plane 9.01 RC1. Here's what's going on and how you can try it.

9.01 is the first of what may be a few small internationalization and localization patches. 9.01 adds German localization and updates the French strings. 9.01 also has a handful of very minor bugs; Austin will probably post the bug list shortly. 9.01 is already RC because the code changes are really minor.

After 9.01 we'll do another localization patch (probably in the next month) that will include unicode and true-type font support. That build (902? 905? Who can guess what number Austin will pick!) will probably have a formally named beta because the unicode changes are extensive.

900r3 is still the latest final build and it's what you get if you update your copy or run the web demo installer.

To get 901r1, go to the updater options and check "get betas"; update your sim and 901r1 will be downloaded. Please try it!

Release Candidates and Betas

I just added this article on version numbers and betas to the X-Plane Wiki.  A few notes:

• You can select to update to new betas using a check-box in the X-Plane Web Updater. Previously there was a special updater for betas - now it is a preference, to cut down on the number of applications we have floating around.
• Release candidates do not get re-released or renamed when they go final!  A release candidate means "we think this is totally done and can be released".  It is declared "final" when, after letting people use it for a while, we decide that there probably aren't any bugs left that we will fix in this release.  So 900r3 is the final version of X-Plane 900.  

It took us three tries (900r1, 900r2, 900r3) to get the final version of 900 right.  So we will not rebuild the sim just to remove "r3" from the app version...900r3 is the latest - if you have it, there is nothing to update.

And of course, my usual rant about betas: betas aren't free candy, they're an attempt to work the bugs out of buggy software!  If your goal is to enjoy your X-Plane experience, the safest bet is almost always to not download the beta.  (An exception would be if there is a work-around for a driver bug that affects your hardware, available in a new beta.)

If you are a third party developer/author, please do download the betas, early and often!  It is much easier for us to fix bugs early in beta.

The X-Plane Support Wiki

We have an X-Plane support Wiki: http://wiki.x-plane.com/.

I'm not sure what will end up there eventually, but it will include some airplane development info, a lot of trouble shooting tips, and general instructions.

It will not contain scenery or plugin development info, as they have their own full websites.

I encourage you to edit the wiki and help organize - I am putting information up there as I think of it, but I need help editing it for organization.

Six Or Eight DVDs

I made the mistake of nuking this poor user's comment and I can't figure out how to get it back; it's a question that I think a number of people might have, so:

Ben, I am thinking of buying the dvds. I have heard the new (final)
version fills the whole dvd (6gb) whereas the beta took about 1,5 gb.
What is the difference? If I buy the final version, will I get better
geographic detail (more vectorpoints) or what is the real reason?

First, you will not get more scenery detail (geographic, vector points, or otherwise) in the final than beta. The scenery is the same in beta and final.

Generally speaking, every difference between beta and final is available via web download. In fact, the final DVDs are made by:

1. Installing beta 1.
2. Running the updater to get the final version.
3. Cutting a new DVD off that finished result.

So I am quite sure that the final disks are the same as a beta DVD plus update.

Now the question is then, why did we do eight DVDs before and six now? The answer is two-part:

1. Walmart.
2. Best Buy.

The retailers prefer a six-DVD pack that is commonly used for software; our distributor therefore wanted us to cut from eight to six DVDs. We did this both by using a more powerful compression algorithm on the DSF (7-zip instead of regular zip for DSFs) and by packing the DVDs much tighter. The old DVDs are one continent-per-DVD, plus the sim; the new set is simply a ton of files packed onto 6 DVDs until they're almost full. Since the new installer lets you pick areas to install, the fact that the files are packed is not a huge problem.

So why did Austin have a sim DVD with only 1.5 GB of stuff? Convenience during beta. Austin updated the beta DVDs at beta 11; by having the sim by itself he could make a new sim DVD and not recut all of the scenery DVDs. (Cutting DVDs takes forever; an hour to burn each one, a lot of testing, a few copies, start all over if you find a problem. It can take me a whole long day and into the night to create a master.)

As a final note, there is a difference between the early US retail six-DVD set and the Laminar one; the installers are different. But they are both six DVD packed sets with scenery on the sim disk (disk one). The retail disks had to be finished first; the new installer was not ready. We will be using the new installer from now on.

The operative point here is: it really doesn't matter what you buy; everything we've released can be web-updated to the current version!

No Need to Repurchase

I think Austin has made this clear, but...

IF you got x-plane 9.00 BETA, then you do NOT need buy version 9.00 final... if you just run the updater, then it will UPDATE your beta TO the final version.. there is no need to make another purchase.

BUT, if you WANT to have the FINAL version on DVD for backup purposes, or to get the latest DVD INSTALLER, then you should re- purchase x-plane 9.00 now... though i do not recommend or suggest that at all... you just CAN, if you want the FINAL, NOT BETA, on DVD

The key equation here is:

Beta DVD + Web Updates = Final Sim

Everything that has changed from the beta DVD to the final DVD is available for free using the updater. If you want to buy more DVDs, Austin's not going to stop you, it's a free country. But we made sure that everything that was big and couldn't be downloaded would be there on the beta 1 DVD so that users could buy the beta DVD and have a complete sim.

So if you have a beta DVD, just use the updater to get the latest version (and RC3 is final, so if you have RC3, you're already done) and you're all set.

(Am I beating this to death? Yes...but...people get very angry at the idea that their DVDs are "not the latest" - so we put a lot of effort into making sure that you can buy the DVD early on and still have the latest!)

Updating X-Plane (Take 2)

Now that X-Plane 9.00RC3 is final, we've released a complete set of "next-gen" updaters and installers, identifiable because they run at 800x600. With these new installers and updaters, we've changed the way updating the sim works.

In the past, we had one application (the "net installer") that would:

• Update any existing files it found and
• Get any files that were missing.

This was useful in that you could get a fresh copy of X-Plane or update your existing one.

But it also made life a living hell for our tech support folks; the single biggest tech support item we would get is users who would:

1. Go to update X-Plane using this tool.
2. Pick a new location, not their existing install.
3. Download 700 MB of demo instead of 30 MB of updates.
4. Run the demo and discover they now have no scenery.
5. Become confused and call or email us.

To try to combat this confusion, we've separated the concept of installing a demo from the concept of updating the sim.

The new "web demo installer" always makes a new copy of X-Plane. It will not install into an existing location; if you ask it to install to an existing folder, it will demand you pick a new name. It will always fetch the full 700-MB demo.

The new "updater" always picks from an existing copy of X-Plane (presented as a list of known X-Plane 9 installs, rather than a file picker - one of the big problems was people picking a folder inside the X-Plane folder). It always updates this existing folder, and will thus never fetch a new install or create a new install.

My hope is that users can identify the task they wish to execute (install a demo, or update what they have) and thus use an application that will guide them through a path without pitfalls. Top concern is that the updaters not install second copies.

Advanced users: you can still do anything with these tools that you could before - but the functionality is now split into two apps. The updater will never rename an existing folder name (as this breaks people's shortcuts), and the installer will let you customize both the install name and the install location. You don't have to install "X-Plane 900r3 Demo" to the desktop.

Finally, a note on beta: previously we had a separate set of tools for beta; searching for betas is now a check-box preference that puts nasty red writing up. This is mostly to keep me sane and the number of installer builds down to what I can count on my fingers. So when the next beta comes out, just take your updater and enable "search for betas".

My Opinion: Let's Not Use RAR

I had to do some research into compression algorithms recently, because we had to squeeze the global scenery onto fewer DVDs for retail distribution.  We did this mostly by completely filling the DVDs, but we also had to use 7-zip compression to get about a 10% improvement in compression ratios.

DSFs are not the best test of compression efficiency because the format has been organized to help algorithms like zip compress them - the improvement with 7-zip and RAR was a lot less than you'd get with, say, a text file.

Anyway, my point here is: let's not use RAR - it's the new GIF.  Every now and then a file format comes along with some kind of restriction that keeps everyone from doing everything with it.  In GIF's case, you had to buy the right to create GIFs, and in the case of RAR you have to buy the right to compress RARs.

I think that having these kinds of entanglements in fundamental low level file formats (like how do we compress our data or save our images) is really bad for the software community as a whole; it balkanizes raw materials.  And file formats stick around for a long time - even though GIF is made obsolete by PNG you'll still find them all over the web.

The lure of RAR is of course higher compression ratios than zip.  But 7-zip can do the same thing, and unlike RAR, has the potential to be completely free, which means it can be completely ubiquitous.

Macintosh users understand the problem here: for the longest time "StuffIt" archives were the standard way to compress data on the Macintosh.  The file format was proprietary, so you couldn't even make your own program work directly with StuffIt archives.  Now that zip has taken over on the Mac, getting data between Mac and Win is easy - you can just zip something using the operating system and send it to all your friends.

Let's not go back into the "bad old days" of proprietary utilities and a lack of integration with regular apps.  I say: if you can stand to use zip or bzip instead of RAR, vote for what's open and has a future, not what is slightly better now but will just be a pain in the ass in three years.

Don't Put Non-ASCII In Your Scenery Files!

I don't know how much of a problem this is yet, or how much of a mess it's going to make of people's scenery. Here's the background:

• ASCII defines 128 character values, mostly letters like A-Z. With ASCII, you can write English and that's just about it.
• The byte that ASCII is stored in on all modern computers can store 256 values.
• Clever people got the idea to put some more letters in the other 128 values to create characters like é and å.
• People defined different "codepages" that have different sets of charcters in those "upper 128" slots. So one code page might be good for French, another for Russian.
• Modern software uses unicode characters, which have a lot more than 256 values, and can thus hold all sorts of characters in one string.

Code pages were around for a while, but they're not a good idea. The problem with code pages is that the same numeric values are used for different letters. The result is that a correctly written Russian document, when converted to a different code page, looks like gibberish. And if you want one document with both French and Russian, well, one code page doesn't do you much good.

Now X-Plane's handling of non-ASCII characters is pretty poor in version 9.00 (and all previous versions). It will draw ASCII and take keyboard input from ASCII but not much else. If you hit the é key on your foreign keyboard, probably nothing will work.

But it turns out there is one way to use foreign characters in X-Plane - I just discovered it tonight. If you use Windows and your system's codepage* is set for a foreign language, you can use those foreign language characters in an OBJ file to name a file on disk with the same name. In other words, you can have textures named été.png and it will work.

Sort of. If you then change your system to work in Russian (which changes the code page) your texture will stop working. The reason things stop working is that the file system uses unicode; that is, the OS knows that été requires a Latin character set that's French friendly, but X-Plane is using Russian since the system's set that way. The result is that the file system has no way name the file in Russian and we fail to load the texture.

So using the "high 128" characters from your system's code page to make non-ASCII characters is a bad idea because your scenery won't work on other people's computers.

But it's going to get worse in the future. X-Plane is going to start using UTF8 in a lot of places. UTF8 encodes unicode into one byte characters by using more than one byte for non-ASCII characters, but as a result it uses the "high 128" character codes for very different things. été.png in UTF8 comes out quite different.

I'm not sure how we'll handle this yet (use UTF8 in the scenery system or have some kind of backward compatibility). But for now I can only advise one things: use ASCII only for your file names. In fact, a good guideline for filenames for the scenery system is to use only numbers, letters, and the underscore.

Spam My Wiki, Please

I'll post more about scenery soon; this will be short and not terribly topical.

The X-Plane Plugin Wiki used to have no login requirements - anyone could just click and edit. All was good for a while, and then I logged in one day to find some of the most highly used pages stuffed to the gills with the phrase "Nigritude Ultramarine" over, and over, with links to other sites.

You can read about this phrase, why it was invented and what was going on here.

Our response was to put a user login requirement on the site, and we haven't had a spam problem since (knock on wood) although we do seem to get what appear to be bogus signup requests. (They don't really hurt anything, they just clog the user database. I'm not sure why anyone would sign up if they don't intend to actually do anything.) But a few thoughts on Nigritude Ultramarine and people's attempts to get junk spam sites into the top of Google's search listing:

• I was pleased to see a real site (this FAQ) as the number two search hit for the term...this real link from a real blog to them can be sort of a contribution to their page rank.
• I have faith that Google will continue to fight the technology arms race against seach engine optimizers...Google has gobs of money and an immense motivation to do so.
• Apparently link farmers, in an attempt to raise their page rank, have been using bots to automatically steal blog content. I haven't seen this myself yet and I've never had an X-Plane query go to a junk site. But some blogs I read have complained of this happening.

Only mildly related, there is an X-Plane Wiki, and I'll try to point people toward it next week; I've been posting things there rather randomly in an attempt to get underdocumented stuff written down somewhere.

The Relationship Problem

I just finished about 15 pages of emails, mostly to Andrew McGregor (who is the very first MeshTool user) and also Benedikt Stratmann (whose x737 is on the bleeding edge of plugin-based aircraft) and AlpilotX (we all know about his forests). Probably all three are wondering how the hell I have time to write so much on weekends. (The answer is of course that my frisbee game got rained out. Foo!)

In the meantime, probably about 300 other people who have emailed me in the last few
months are wondering why the hell they have heard nothing from me. My in-box looks like a mail server exploded. It's not pretty.

So let me blog for a moment about the "relationship problem". Simply put, there are two of us (Austin and myself) and about a thousand of you (third party developers doing cool and interesting things with X-Plane) plus significantly more users, some of whom have some very weird tech support problems.

In this environment, our algorithm for who gets "developer attention" is pretty broken and subject to total thrash...there is a huge element of random luck (who emails me when I am recompiling the sim vs. debugging a nasty bug).

I'm aware of both how hard the task Austin and I face and how frustrating it is for a third party developer because I've been on both sides. Before I worked for LR, I was a third party and I was always astounded that Austin couldn't remember what we talked about last week.

Then I started working for the company and saw what it's like. Imagine sitting at a train station watching the trains go by* (at full speed, not stopping) and someone says "last week I waved to you out the window and you waved back, remember me?"

So I would advise three things to the neglected third party:

1. Be firm - you may need to ping us again because at busy times we can't always keep track of who has asked for what.
2. Be patient - if you need something the week we're burning DVD masters for a second time (because the first set failed at the factory) then you're going to have to wait.
3. Don't take it personally...a lack of a response usually indicates overload inside the company, not a poor opinion of your work!

This blog post has rambled enough, but it may feed well into the next one.

* This analogy is totally stolen from "How Doctors Think" by Jerome Groopman - he uses it to describe the task of primary care physicians trying to spot the early signs of a very rare illness among a fast-moving train of patients who are almost entirely healthy. I strongly recommend this book particularly for Americans - we need to understand the forces at work in shaping the quality of our medical care!

Things That Are Not Jokes

Just to clarify, these things are not April Fools Day jokes.

• RC3 coming out.
• The installer installing to the desktop.

And as always, relax, you will be able to put the install anywhere you want via the "destination" button.

Instaling to the Desktop (Let's Start a Riot)

I am intentionally announcing a decision that might make experience X-Plane users unhappy - my goal here is to solicit arguments against it (if there are any) since it's a sort of a strange change.

The proposal is to make the default installation location for a new copy of X-Plane be...the desktop.

EDIT: to clarify some of the blog comments, this is a default installation location; the user will be able to customize both the folder name that is created to contain X-Plane (default will be "X-Plane 9") and the folder into which this new X-Plane 9 folder will be placed.

The desktop? What are you hacks thinking? Well, here's what we're thinking:

1. Our goal is to minimize tech support calls during installation. This means making an installer where the last computer-savvy users will not get stuck in the installation process. If you know what you're doing, you're not who we're aiming at. (I reiterate, we get a lot of calls about installation problems from users who have never used a computer before.)
   
2. We need a location that the user installing X-Plane has access to, guaranteed. That rules out places like the Applications folder on OS X - we expect the least sophisticated users to not customize the install location, so we need one that will work.
3. We need a location that the user can find. This rules out their home folder (the user may not use their home folder or know it exists) as well as the C drive and the Program Files folder on Windows (both can have their files hidden by default on Windows XP to keep users from breaking tings).

The desktop solves all of these problems. Every user has write-access to his or her own desktop, and every user knows where it is.

If, like me, you don't want X-Plane on your desktop, you can simply click the "destination" button in the installer to put it somewhere else.

As a final note, the strongest alternative to this on Windows was to put the app in program files and build a start menu short-cut. But this starts the sweater unraveling...if we have a shortcut, we need an uninstaller rather than trashing the folder...if we have an uninstaller, how do we cope with multiple installs...by the time you solve these problems you have a huge amount of new untested code.

I'd like to get a more Windows and Mac interface compliant installer, and we'll get there eventually, but the work I'm doing now is aimed at the biggest real problems we face:

• Users not knowing where X-Plane is.
• Errors during the install in its default configuration.
• Problems installing and configuring scenery.
  

Okay, there it is, fire away!