Fun With Numbers

"Scenery load is now twice as fast!" Where do Austin and I come up with these claims? Here's a little bit of info on on our performance tuning process.

When we work on performance, we always measure a baseline metric and then look at this metric while we work on the code. The test situation must be the same every time, so usually we use a standard plane position (like on a known runway) for framerate, or a known scenery box (for scenery loading).

Here's the statistics for X-plane scenery load before and after tuning the DSF loader:

Before optimize
DSF load time: 2297036 for file +33-119.dsf
DSF load time: 4614647 for file +33-118.dsf
DSF load time: 4251205 for file +33-117.dsf
DSF load time: 5137940 for file 34-119.dsf
DSF load time: 5038098 for file +34-118.dsf
DSF load time: 3893140 for file +34-117.dsf
After Optimize
DSF load time: 1634270 for file +33-119.dsf
DSF load time: 2156187 for file +33-118.dsf
DSF load time: 1882647 for file +33-117.dsf
DSF load time: 2418666 for file +34-119.dsf
DSF load time: 2318480 for file +34-118.dsf
DSF load time: 1819997 for file +34-117.dsf

These numbers are not normaly visible; a special sim option available to us causes it to print the load time per DSF. The nubmers are in microseconds and shown per DSF.

So on immediate inspection you can see improvements of between 140 and 205%. But to really understand what this means you have to consider all of the fine print:

These stats are with objects and roads turned off - this is with scenery on minimal settings in both cases. With more 3-d stuff turned on, the overall loadtime might be better or worse; these statistics don't tell us anything about that.

These tests were done on the same 6 DSFs in both cases, but other DSFs might have different improvements. You can see that even in the six DSFs here the amount of improvement varies a lot!

These tests were done on my dual 1.8 ghz G5. This is a fast machine for a Mac (but probably only average compared to modern desktop PCs). The speed-up could vary on other machines that don't have similar performance characteristics, since scenery loading depends on the speed of the CPU, memory bandwidth, I/O performance, etc.

This is all a long-winded way of saying "your mileage may vary". When Austin and I post performance numbers, they are calculated in strict test conditions with all variables controlled. But since these statistics represent one datapoint, they are not universal changes, just indicators that our code changes are going in the right direction.

(Could it be that an optimization makes a difference on our machines but not yours? Yes - this is always a risk! We try to run on different machines and we listen to user feedback on performance to catch this case.)