Fun With Servers

Just an FYI: when it rains it pours. Normally betas increase load on our set of update servers. To compound this, one of them is suffering a midlife crisis^H^H^H^H^H^H^H^Hhard drive failure.* We're working on it now; hopefully it will be resolved in the next 24 hours.

EDIT: the update server is back up - our host not only swapped out the drive, but the whole box. We'll have to take it down one more time in the future, but for the most part I think we're out of the woods.

Another note on servers: Chris has restructured X-Plane for Android to separately download the art assets from our servers, rather than contain all art assets in the actual download. What he found after several painful weeks was that the Android store is not yet reliable for large apps. While the official app size limit is 50 MB, many phones have problems with their configuration that cause downloads to fail. When the user buys our app and the download fails, they get angry at us. (X-Plane may have been, until it was restructured, one of the largest Android game APKs. The other games with large amounts of 3-d content were already doing separate downloads.)

We originally wanted to build a monolithic app (everything in the APK) because we thought that this would provide the simplest, easiest configuration to maintain, and thus hassle-free installation for our users. You get the APK, you install it, you fly! Unfortunately, the Android Market isn't reliable for such a large download, so we had to re-evaluate.

The new system downloads only the core app from the Android Market and then pulls the art assets from one of our servers. So far this appears to be an improvement. If/when Google provides an integrated solution, we will probably switch back to it to simplify the process again (right now we have two points of failure: the Android Market and our server farm, which, per the above notes, sometimes does fail). But for now, we'll host the apps and try to give people the best download experience we can.

Finally, I will try to roll out at least a beta of new installers some time this week. The new installer simultaneously downloads from multiple servers, with a more efficient HTTP implementation; this should hopefully result in better download times and also lower server load per demo.

* Chris pointed out: most normal humans don't know what this ^H^H^H^H is about...it's nerd-speak for the delete key, e.g. to undo a text. ^H is control-H, which you may find works just like the delete key. Yes, I'm a huge nerd.

Updater Design Decisions

My blog postings have been a little thin - but not because nothing is going on. Rather it's a combination of working on next-gen stuff that is still in heavy development, travel, and (perhaps due to the travel, perhaps due to lack of sleep, or perhaps due to certain forms of Brandy brought on site by a certain British developer who won't be named, but whose handle on the org starts with a P and rhymes with "Cropsman", cough, cough) a minor fever that I've just about recovered from.

A user asked me about the design of the X-Plane updater - so this post is only going to be of interest to the few authors out there who are creating installers for their add-ons and need to update. We try to keep the file formats for X-Plane simple, e.g. "this one folder is your airplane", so that if you don't make an installer, the user won't be overwhelmed with a complex unpack-and-install routine. But if you do need to install or update, well, here's what we were thinking when we developed our installer.

We had a few needs for our installer:

• It had to be cross-platform to Mac, Windows and Linux.
• It had to update existing products as well as install new ones.
• The updates had to be minimal deltas, e.g. a demo can be 600 MB but an update should only be the 30 MB of files that really changed.
• We wanted the updater to do delta updates from any old version. (Some update systems require installing a number of patches in sequence. We have users who buy the DVD and then update a year later; we wanted to let them update in one shot.)
• Building the patches had to be really, really fast. We use our updater to publish our betas. So while X-Plane probably has 2-3 major patches in a single year and maybe 5 or 6 bug fix patches, we will (due to betas) cut perhaps 100 or more actual "updates".
• On that last point, cutting separate updates by platform was a deal breaker, as it would triple the number of updates we had to cut. Having a demo install and an update be the same on the server was a big win.

Some of those needs are pretty specific to Laminar Research, but some might apply to a third party. There are commercial installers that will let you generate patches. If you don't generate a lot of patches, you can probably use existing tools - basically you'll spend a lot less time up front (since the tool set exists) but you might spend more time in the long run cutting separate patches by OS.

One of the first things we decided was to not use server-side technology or special protocol. That is, we don't need any kind of smart servers to run the updater - the updates are just files hosted over HTTP on standard rented or owned apache servers. All of the work is done in the client. We did this for a few reasons:

• It lets us throw up the install anywhere - we don't have complex needs for what's on the server. Virtually any server will do. (As the company has grown, our server needs have grown too, so this is less of a concern now, but back then we had fewer servers in service, and were buying less overall server capacity.) If a server goes down, we can press another one into service about as quickly as we can move the file set to the server.
• We're not server programmers. Coding the installer/updater on the client side let us leverage existing company technology, etc.

Here's what we came up with. The basic idea of the installer is very simple. Note that since the installer uses HTTP files, you can "watch" the installer simply by downloading the files that it downloads.

1. First the installer goes to a master server and gets a "version list", which tells it what it's going to actually get and what (multiple) mirrors are available. This one master file is the only file that must be hosted at X-Plane.com, and it allows us to change a very small file to press mirrors into service.
2. Installs and updates are actually the same to us - it's a set of files that the user should have once the install or update finishes. This starts with a file directory that lists every file (by path) that needs to be downloaded, as well as its MD5 signature.
3. The directory also contains the MD5 of every old version of any file in the version, and lists files that have to be deleted to update the version (e.g. the file panel.png used to have MD5 signature 2934b..23abc2 but is now removed).
   
4. The client, once armed with the file directory, can now download all files in the directory (install) or compare the existing files on disk to the directory by MD5 and only download changes.
5. Individual files are on the server in zip form for download and decompression.

That's pretty much it - you could write an installer in a scripting language using a tool like CURL or WGET - it wouldn't be very complex because the installer is really just a clever, painted download+unzip tool.

Dealing With Modified Files

There is a bit of complexity in this design that you might not need for a third party installer: handling modified files. Note that the directory contains the MD5 not only of the current version of the file (to detect when no update is needed and save bandwidth) but also to see when the user has modified a file that is "managed" by X-Plane.

Before the installer overwrites/deletes a file that you own, it compares the file's MD5 to the entire known list of MD5s for the entire version. If this file doesn't match a known old version, it puts up the warning dialog box that you have modified a file that is about to be overwritten.

While we recommend that add-ons use scenery packs and other "safe" ways to customize the sim, this helps detect when a user has gone in and edited the cloud textures in Photoshop, and prevents us from overwriting them without warning.

1,2,3,4,5...6,7,8,9,10,11,12

Surely I'm not the only one with that song embedded in some deep part of my psyche, right? One thing we seem to have down with 940 is counting upward...945 is in testing now.

There are two things driving the perpetual 940 bug fix releases:

1. Third parties are using the sim heavily, so the bug fix releases have been vehicles for fixing a number of small bugs that are only reproducible with custom add-ons. (The fix for night lighting and orthophotos is an example of this.)
2. We've played a real game of whac-a-mole with the throttles. (The bug is that X-Plane will lock the two throttles together. I believe that the new 945 release candidate finally fixes this.) Having done a bit of a 3 stooges routine on this bug, we've tightened up our internal procedures a bit.

As always, if your third party add-on broke with 940, please file a bug report! The only case of this I know of is the throttles bug - 945 should fix the BK-117, the Mu, and any other plane suffering from throttle-glue.

Will there be a 946? There very well might be, and I sincerely hope it will not be because 945 missed its target. But if, two months from now, we've collected another half-dozen fixes for third party authoring, I don't see why we wouldn't release them.

Could The Name Beta Be Any More Confusing?

The version number gets bumped when we have a release and it still needs patching. Of course, some bugs get reported after we ship, and some address issues not even under consideration during release. Had we not had any fire drills, we would probably still be on 942 or 943 right now, because we've had third party reports with very specialized authoring cases after we finalized the build.

Here's what makes things even more confusing: that check box "Check for new betas as well as updates" in the installer. At any one time we may have two versions of X-Plane availalbe for download:

1. The official version (currently 944) for everyone.
2. Whatever the latest in-test build is (currently 945).

(You can even see which versions are current here.) What's confusing is that while this check box says "get betas", in truth it gets any build that is "in staging". At this instant that I blog, X-Plane 9.45 final is in staging. That is, we believe 9.45 fixes our two known bugs, we think it's done, it's a release candidate, we think it is the next American Idol^H^H^H^H^H^H^H^HX-Plane. But until we certify it, you still have to check "get betas" to download it.

What we probably need to do is re-label the check-box in some way; the "staged" approach where every build is available for test first is, I think, a valuable tool to let third party authors confirm that we fixed their bugs before the entire world gets the app.

Scalability and apt.dat

Beta users, see the bottom of this post for how scalability turns into possible bugs.

In computer science, a program or architecture is scalable if it doesn't totally vomit up a lung as its constituent parts become bigger. For a cleaner definition, see Wikipedia, source of all internet knowledge.

An ant is not scalable - if you made an ant 100 times larger in every dimension, its tiny legs would break under its new weight. (An ant is not scalable because its weight grows faster than its structural strength. Thus elephants are not built like ants.) Geeks: scalability is to computer science as marginal cost is to economics.

Before X-Plane 940, the apt.dat file was distinctly not scalable. The entire file was loaded into memory; as users created more and more taxiway lines and signs and details, we simply used more and more memory. This approach isn't very scalable because authors have the potential to grow the apt.dat file faster than our system requirements can increase.

X-Plane 940 fixes this by not loading the entire apt.dat file into memory. Instead, only essential airport information is loaded into memory, along with a note as to where in the file the airport lives. Whenever an airport actually has to be built into a 3-d mesh while you fly, we go back to the apt. file and load the rest of the data for the one airport we are building, use it, and throw it out. Since 3-d airport meshes are built on a second core, the cost of loading one airport off disk is pretty harmless.

The problem with this fix is that it introduced a new scalability problem. Consider:

• Meshes in 940 are built on as many CPU cores as you have - some users have 8!
• Each CPU core could be working on a different airport, depending on how many are nearby.
• Each airport has to load up the apt.dat file to get the extra airport data.

This means that at times on an 8 core machine we could easily have allocated 50 * 8 = 400 MB of memory just to temporarily hold 8 copies of the apt.dat file.*

This is of course completely silly - there's no reason to load the whole apt.dat file to get one airport, and the fix that is going into beta 8 lets the airport loader surgically grab just one airport. Thus we will be scalable again, because adding more cores won't cause memory usage to go up.

Beta Users: Please keep an eye out for X-Plane running out of memory - if it starts to do so in the next beta it means that some part of this code change munged memory management. We're running stress tests on the sim now, but touching the low level memory nd file handling code late in beta isn't something I like to do.

* While loading 8 copies of the apt.dat file is wasteful of memory, it is not slow; X-Plane uses memory mapped file I/O, so reading a small part of a large file is very fast - just not very virtual-memory efficient.

New Installers

The 2.05 installer is now released - this is a multi-language update that fixes a few bugs and makes the map interface more usable. If you have existing DVDs, you may want to use the new 2.05 DVD installer because it scans the scenery folder very quickly when you pick "add-remove scenery".

(The old installer would check the signatures of all scenery files - this one simply checks whether they exist.)

No URLs have changed - installers will always have the same file names and live at the same URLs; you can pick "Get Info" or "Properties" to tell their version (or run with --version on Linux).

I have a ton of emails I need to get through, so if you emailed me, I do apologize; I will try to clean out the pending tech support issues, etc. in the next week.

Releases, Bugs, and the iphone

Sometimes I have one of those weeks where all I do is look at crashes and weird behavior.  This is turning into one of those weeks.  So here's some status on the various bugs floating around.

I should say: you'll find a lot of developers blaming the technology providers for bugs (just look at how many OpenGL developers have blamed ATI for their apps crashing).  Sometimes it's the app, sometimes it's the driver.  More importantly:

• You don't know who's fault it is until you fully understand the bug.
• The fix for a bug might not be in the broken code.  That is, one piece of code can work around a bug in another.

So...you can't necessarily tell whose fault it was from new drivers coming out, us changing the sim, etc.  But...when it's my stupid code, I'll admit it openly - no one should think that a bunch of other smart programmers are screwing up on my behalf.  (This is also useful to other apps developers, who can know that my bug isn't the same as their bug, since my bug was in my code.)

X-Plane: we're happy with 921r2 finally...the final bug (crash on startup on the Mac) was due to an incompatibility between Apple's OpenAL implementation and special memory allocator (which is really just a wrapper around NEDMalloc).  I still don't know exactly what the rules should be (you try reading the C++ spec!) but for now we turned the allocator off on Mac.

(This brings up another issue about bugs - you can't tell whose bug it is by whose code crashes, since one piece of code can sabotage another.)

So the next X-Plane release will probably be 930, with new features.  We may have a few more language patches if needed.

iPhone: the 9.01 iPhone patch is out, and it improves framerate a bit.  We are still seeing crashes on startup for users who have just downloaded the app.  Rebooting the phone will fix this, but please see this post for more info!  We need your help to collect more data.

Radeon 9800 on Windows: for the longest time we've had users reporting "framebuffer incomplete" errors when using catalyst 8.x drivers, an R300 chipset, and Windows with X-Plane 9.x.  I have been trying to reproduce this problem "in the lab" off and on for months, but finally saw it this week.  From what I can tell, we're getting into some low-memory condition and the driver is freaking out in various ways.  The command line options people sometimes use to get past this are probably rearranging memory, not saving it.  I don't know why the Catalyst 7.x drivers don't have this problem.  But...at least I am making more progress than I was before.  Please see this post for more info.

Installers: I am working on the 2.05 installer.  I have seen a number of users report problems running a full install from DVDs, so I am just starting to investigate that.  I will post more when I have something to test.  Unfortunately the problems reported are not something we see here.

Bad Alloc, New Patch, New Installers

The 921 patch is now available. This fixes the bad-alloc problems relating to complex airports.

Please note: if you have a bad-alloc crash, it could be because you are out of memory. Make sure you have virtual memory turned on, your page file is large enough, your disk isn't full, etc. If you have a bad alloc error, try the sim without third party add-ons to see if you really are running out of memory. If you are running Vista or XP, use /3GB or the BCD - see this for more info. Basically when you are running out of memory, you either can crash on "bad alloc" if we need memory for the CPU or "we ran out of video card memory" if we can't map geometry into virtual memory.

If you get a crash with 921 on OS X, please let me know by email! I've seen one of these reports.

I also have cuts of the new installer/updater suite, version 2.05 - if you are going to update, tryo ne of these:

http://dev.x-plane.com/update/installers9/stage/

The main features of the new installer are:

• Support for French, Spanish, Italian, German, and Russian.
• Clearer colors on the world map.
• Scanning the global scenery folder to add/remove scenery is much faster.

I recommedn the new installers for that last point alone.

Why Is Installer Scanning Slow?

When you update X-Plane, it scans about 10,000 files before it runs the update. This scan checks every single byte of every file. As a result, it's a little bit slow - it takes a few minutes.

With the new installer, when you add and remove scenery, we do the same scan on the installed scenery. This scan is very slow - 78 GB is a lot of data!

When the next installer comes out (2.05, which will be cut after 920 goes final), the add-remove scenery function will do a quick scan to see what files exist. This is a lot faster - even with full scenery the scan takes only 10-20 seconds on a slower computer.

I can't pull the same trick for updates - we need to detect any possible defect in a file during update to ensure that the install is correct!

Installer Chaos

I spent part of the morning tracking down installer issues.  Here are some things I have learned:

1. If you have Windows Vista and User Access Control, there's a good chance that the installer may get caught on permissions problems.  I don't fully understand what's going on, but I don't think it's a coincidence that it is US GraphSim customers who see this.  

   Basically those DVDs, which went to press a little bit early, default to installing X-Plane on the C drive, which is not a generally accessible area.  UAC then makes some guesses and allows some but not all operations (or something equally weird) and the user gets stuck.

   I am still investigating, but I think the fix is to install to your home folder.

2. Some users have bad DVDs.  It happens, and I think it happens more now that we're dual layer.  There's also a variance in the sensitivity of drives - some users send us back damaged disks and they look bad but we can still read them.

   Tech support will tell you this, but if you can't install, one test is to simply copy the entire DVD to your hard drive; if the copy fails mid-way with "I/O error" or some other message about damaged media, call up tech support, they'll swap you a new disk.

   We try to find duplication facilities that do high quality work, but bad disks do happen.  Tech support should be able to make this right for you.

3. I've gotten a wide variety of weird reports on Linux - there are a lot of distros out there, so potentially a lot of versions of the OS code base.  For Linux users I strongly recommend the X-Plane Wiki, where we are trying to accumulate all of the fringe cases.

901 - Stealth Release

901 is out now...basically it's just 900 + French and German localizations. Also a few minor tweaks:

• We added a few datarefs and commands by request - check DataRefs.txt.
  
• The plugin font size is back to 6 pixels, like v8. It looks a bit ugly now, but we'll put a new font in place soon. The important thing is: 6-pixel fonts for v8 and v9, so one plugin can operate everywhere.
• A few plugin bug fixes - see the SDK known bugs page.
• Linux won't complain if your disk is mounted as iso9660. Since the v9 final disks are all mastered as ISOs and not UDF, asking you to mount them as UDF is asking the impossible.
• If you have a demo install, it will add the word "Demo" to its name in the startup screen. This is to help users with more than one install identify which one they are running.
  

If this all seems minor and tweaky, well, it is. The main point of 901 was to get the localizations in. In the future we'll have a feature patch that brings new airplane features in and other cool stuff.

901 is freely available using the X-Plane web updater...the download size is only about 14 MB.

Finally, I've posted all new DVD installer/updater/web demo installer apps - version 2.04 localizes to French and German too.

901RC1 Already? Try it!

Well, the paint is barely dry and we already have an X-Plane 9.01 RC1. Here's what's going on and how you can try it.

9.01 is the first of what may be a few small internationalization and localization patches. 9.01 adds German localization and updates the French strings. 9.01 also has a handful of very minor bugs; Austin will probably post the bug list shortly. 9.01 is already RC because the code changes are really minor.

After 9.01 we'll do another localization patch (probably in the next month) that will include unicode and true-type font support. That build (902? 905? Who can guess what number Austin will pick!) will probably have a formally named beta because the unicode changes are extensive.

900r3 is still the latest final build and it's what you get if you update your copy or run the web demo installer.

To get 901r1, go to the updater options and check "get betas"; update your sim and 901r1 will be downloaded. Please try it!

Release Candidates and Betas

I just added this article on version numbers and betas to the X-Plane Wiki.  A few notes:

• You can select to update to new betas using a check-box in the X-Plane Web Updater. Previously there was a special updater for betas - now it is a preference, to cut down on the number of applications we have floating around.
• Release candidates do not get re-released or renamed when they go final!  A release candidate means "we think this is totally done and can be released".  It is declared "final" when, after letting people use it for a while, we decide that there probably aren't any bugs left that we will fix in this release.  So 900r3 is the final version of X-Plane 900.  

It took us three tries (900r1, 900r2, 900r3) to get the final version of 900 right.  So we will not rebuild the sim just to remove "r3" from the app version...900r3 is the latest - if you have it, there is nothing to update.

And of course, my usual rant about betas: betas aren't free candy, they're an attempt to work the bugs out of buggy software!  If your goal is to enjoy your X-Plane experience, the safest bet is almost always to not download the beta.  (An exception would be if there is a work-around for a driver bug that affects your hardware, available in a new beta.)

If you are a third party developer/author, please do download the betas, early and often!  It is much easier for us to fix bugs early in beta.

Six Or Eight DVDs

I made the mistake of nuking this poor user's comment and I can't figure out how to get it back; it's a question that I think a number of people might have, so:

Ben, I am thinking of buying the dvds. I have heard the new (final)
version fills the whole dvd (6gb) whereas the beta took about 1,5 gb.
What is the difference? If I buy the final version, will I get better
geographic detail (more vectorpoints) or what is the real reason?

First, you will not get more scenery detail (geographic, vector points, or otherwise) in the final than beta. The scenery is the same in beta and final.

Generally speaking, every difference between beta and final is available via web download. In fact, the final DVDs are made by:

1. Installing beta 1.
2. Running the updater to get the final version.
3. Cutting a new DVD off that finished result.

So I am quite sure that the final disks are the same as a beta DVD plus update.

Now the question is then, why did we do eight DVDs before and six now? The answer is two-part:

1. Walmart.
2. Best Buy.

The retailers prefer a six-DVD pack that is commonly used for software; our distributor therefore wanted us to cut from eight to six DVDs. We did this both by using a more powerful compression algorithm on the DSF (7-zip instead of regular zip for DSFs) and by packing the DVDs much tighter. The old DVDs are one continent-per-DVD, plus the sim; the new set is simply a ton of files packed onto 6 DVDs until they're almost full. Since the new installer lets you pick areas to install, the fact that the files are packed is not a huge problem.

So why did Austin have a sim DVD with only 1.5 GB of stuff? Convenience during beta. Austin updated the beta DVDs at beta 11; by having the sim by itself he could make a new sim DVD and not recut all of the scenery DVDs. (Cutting DVDs takes forever; an hour to burn each one, a lot of testing, a few copies, start all over if you find a problem. It can take me a whole long day and into the night to create a master.)

As a final note, there is a difference between the early US retail six-DVD set and the Laminar one; the installers are different. But they are both six DVD packed sets with scenery on the sim disk (disk one). The retail disks had to be finished first; the new installer was not ready. We will be using the new installer from now on.

The operative point here is: it really doesn't matter what you buy; everything we've released can be web-updated to the current version!

No Need to Repurchase

I think Austin has made this clear, but...

IF you got x-plane 9.00 BETA, then you do NOT need buy version 9.00 final... if you just run the updater, then it will UPDATE your beta TO the final version.. there is no need to make another purchase.

BUT, if you WANT to have the FINAL version on DVD for backup purposes, or to get the latest DVD INSTALLER, then you should re- purchase x-plane 9.00 now... though i do not recommend or suggest that at all... you just CAN, if you want the FINAL, NOT BETA, on DVD

The key equation here is:

Beta DVD + Web Updates = Final Sim

Everything that has changed from the beta DVD to the final DVD is available for free using the updater. If you want to buy more DVDs, Austin's not going to stop you, it's a free country. But we made sure that everything that was big and couldn't be downloaded would be there on the beta 1 DVD so that users could buy the beta DVD and have a complete sim.

So if you have a beta DVD, just use the updater to get the latest version (and RC3 is final, so if you have RC3, you're already done) and you're all set.

(Am I beating this to death? Yes...but...people get very angry at the idea that their DVDs are "not the latest" - so we put a lot of effort into making sure that you can buy the DVD early on and still have the latest!)

Updating X-Plane (Take 2)

Now that X-Plane 9.00RC3 is final, we've released a complete set of "next-gen" updaters and installers, identifiable because they run at 800x600. With these new installers and updaters, we've changed the way updating the sim works.

In the past, we had one application (the "net installer") that would:

• Update any existing files it found and
• Get any files that were missing.

This was useful in that you could get a fresh copy of X-Plane or update your existing one.

But it also made life a living hell for our tech support folks; the single biggest tech support item we would get is users who would:

1. Go to update X-Plane using this tool.
2. Pick a new location, not their existing install.
3. Download 700 MB of demo instead of 30 MB of updates.
4. Run the demo and discover they now have no scenery.
5. Become confused and call or email us.

To try to combat this confusion, we've separated the concept of installing a demo from the concept of updating the sim.

The new "web demo installer" always makes a new copy of X-Plane. It will not install into an existing location; if you ask it to install to an existing folder, it will demand you pick a new name. It will always fetch the full 700-MB demo.

The new "updater" always picks from an existing copy of X-Plane (presented as a list of known X-Plane 9 installs, rather than a file picker - one of the big problems was people picking a folder inside the X-Plane folder). It always updates this existing folder, and will thus never fetch a new install or create a new install.

My hope is that users can identify the task they wish to execute (install a demo, or update what they have) and thus use an application that will guide them through a path without pitfalls. Top concern is that the updaters not install second copies.

Advanced users: you can still do anything with these tools that you could before - but the functionality is now split into two apps. The updater will never rename an existing folder name (as this breaks people's shortcuts), and the installer will let you customize both the install name and the install location. You don't have to install "X-Plane 900r3 Demo" to the desktop.

Finally, a note on beta: previously we had a separate set of tools for beta; searching for betas is now a check-box preference that puts nasty red writing up. This is mostly to keep me sane and the number of installer builds down to what I can count on my fingers. So when the next beta comes out, just take your updater and enable "search for betas".

Things That Are Not Jokes

Just to clarify, these things are not April Fools Day jokes.

• RC3 coming out.
• The installer installing to the desktop.

And as always, relax, you will be able to put the install anywhere you want via the "destination" button.

Instaling to the Desktop (Let's Start a Riot)

I am intentionally announcing a decision that might make experience X-Plane users unhappy - my goal here is to solicit arguments against it (if there are any) since it's a sort of a strange change.

The proposal is to make the default installation location for a new copy of X-Plane be...the desktop.

EDIT: to clarify some of the blog comments, this is a default installation location; the user will be able to customize both the folder name that is created to contain X-Plane (default will be "X-Plane 9") and the folder into which this new X-Plane 9 folder will be placed.

The desktop? What are you hacks thinking? Well, here's what we're thinking:

1. Our goal is to minimize tech support calls during installation. This means making an installer where the last computer-savvy users will not get stuck in the installation process. If you know what you're doing, you're not who we're aiming at. (I reiterate, we get a lot of calls about installation problems from users who have never used a computer before.)
   
2. We need a location that the user installing X-Plane has access to, guaranteed. That rules out places like the Applications folder on OS X - we expect the least sophisticated users to not customize the install location, so we need one that will work.
3. We need a location that the user can find. This rules out their home folder (the user may not use their home folder or know it exists) as well as the C drive and the Program Files folder on Windows (both can have their files hidden by default on Windows XP to keep users from breaking tings).

The desktop solves all of these problems. Every user has write-access to his or her own desktop, and every user knows where it is.

If, like me, you don't want X-Plane on your desktop, you can simply click the "destination" button in the installer to put it somewhere else.

As a final note, the strongest alternative to this on Windows was to put the app in program files and build a start menu short-cut. But this starts the sweater unraveling...if we have a shortcut, we need an uninstaller rather than trashing the folder...if we have an uninstaller, how do we cope with multiple installs...by the time you solve these problems you have a huge amount of new untested code.

I'd like to get a more Windows and Mac interface compliant installer, and we'll get there eventually, but the work I'm doing now is aimed at the biggest real problems we face:

• Users not knowing where X-Plane is.
• Errors during the install in its default configuration.
• Problems installing and configuring scenery.
  

Okay, there it is, fire away!

More Threads - the Installer

Nine women cannot have a baby in one month - that's the classic example that gets thrown around computer science for the difficulty of parallelization - that is, just because we have ten times as many resources doesn't mean we're going to go ten times as fast.

Problems of scalability via parallelization have become very important for graphics engines as everybody and their mother now has at least two cores, and users with more serious hardware are going to have four.

I get asked a lot: "will X-Plane utilize X cores..." where X is a number larger than two. My general answer is: sometimes, maybe, and probably more in the future. I can't make strong predictions for what we'll ship in the future, but the general trend for the last 18 months has been us using more cores every time we go into a piece of code to do major architectural changes.

I've been doing a lot of work on the installer this week - the first major overhaul of the installer since we originally coded it all the way back at X-Plane 8.15. And the new installers and updaters will try to take advantage of multiple CPUs where possible. A few cases:

• The X-Plane updater runs an MD5 checksum over the entire X-Plane folder to determine which version of the various file components you have and whether they need to be updated. This s not a fast process. I am working on threading this so that more CPUs can work on the problem at once. It looks like there will be only modest benefits from this because the process is also highly bottlenecked on the disk drive.
• The installation engine from the DVD will use more than one CPU to decompress files. For zip compression this wasn't very important, but the scenery will be compressed via 7-zip compression to get us down to disk DVDs. 7-Zip compresses DSFs about 10% smaller per file than zip, but it's horribly slow to decompress, so being able to throw twice the CPU at it is a big win.

Now on one hand, our top performance goals are for the sim, not the installer. On the other hand, faster installations are good. But my main point here is: when we wrote new code four years ago, we assumed one CPU and a nice graphics card. We now assume at least two cores and possibly more, and that informs the design of every new feature, not just the rendering engine. If we don't create a multi-core-friendly design, we're effectively leaving at least 50% of the CPU on the table.

When Will X-Plane Stop Saying It Is Beta?

If you look at the about box for X-Plane 9 RC1, it still lists itself as a beta.

Here's the thing: that label is dynamic. Right now when X-Plane calls up the server to see if there is a new patch, it notices that its current version (900Rc1) is newer than the latest "final version". (This is because we haven't declared any version of 9.xx final yet.)

So it thinks for a second and goes "oh - I'm newer than the latest final version, I must be a beta." And that yellow beta label appears.

When we finally declare a version final (which involves tweaking the versions listed on the servers) the existing code will then look at the server and go "oh, I'm the latest version" and that beta label will disappear.

Installer Theory

Let me go into the installers in a little more detail before I start a mini-riot. Basically there are three "installation" operations that we (Laminar Research) support:

1. Installing a new full sim from a DVD.
2. Installing a new demo from the internet.
3. Updating any installation from the internet.

The problem with the current installer model is that tasks two and three (get a new demo, update an existing installation) are handled by a single application. This means that the web-demo-installer/updater cannot know which choice (2) or (3) the user is trying to do (since both are legitimate) and therefore user error can result in the wrong operation.

The most common problem we have is users installing new demos when they meant to update a DVD install. The result is an old copy of X-Plane with scenery, a new copy without scenery, and a very confused user who has to contact us for help.

The solution I am working on is simple: provide separate applications for all three tasks.

1. Like before, the DVD installer comes on your DVD. It installs the DVD, nothing else.
2. A demo installer does nothing but install the demo. You get the installer from the web, probably from the "demo" page (which would have to no longer be an "update" page too). The demo installer always does a full install, and will stop you from trying to install on top of existing installations.
3. The updater is always fetched by the app and updates the app. Thus the paradigm is that the app is "self-updating" (even though most of the work is done by a helper application). That updater (fetched by the sim) can only update the copy of the sim that fetched it.

Dealing With Problems

The above work-flow is meant to address most users doing the usual thing without technical problems. We must also consider how we will deal with tech support problems, and finally what the impact is on advanced users. There are two cases where we sometimes have to help users out:

• If the DVD installer for some reason will not work for a user's computer, we usually provide a downloadable DVD installer. You would insert the DVD, run this "DVD installer" and thus install the DVD. This is not the normal case, but we will probably continue to provide DVD installers specifically for users with tech support problems (based on the tech support incident), just like we have been in the past.
• We will provide the updater directly for users who need to update the sim but cannot launch the sim. This is the exception case, so a direct link to the updater would not be globally advertised on the main page of the sim. Rather it would be a support link, again only provided to users who really need it. 98% of the user base will be able to update from the sim.

When you run the updater from the web (the support case) it will prompt you to locate your X-Plane folder if needed.

Advanced Users

Will you be able to keep multiple copies of X-Plane around? Absolutely. But you'll have to manage your operations in terms of the "three operations" (make a new install from the DVD, make a new demo install from the web, update any one version you have laying around).

If you want to get a new web demo, you'll have to use a separate application than you would for an update. I don't think this is a huge problem; generally your best bet for keeping an old copy of X-Plane (when running a new beta) is to simply copy the entire X-Plane folder, rather than downloading the contents from the net.