Here's a short list of some of the features of OBJ8 you should never use:
Don't use ATTR_no_depth. This attribute should probably never have been invented and will be removed someday. There is no situation I can think of in the sim where this attribute is a good idea. (If you do need it, please let me know - tell me why!) Use ATTR_poly_os to put your object on top of a runway or the ground.
Don't use ATTR_ambient_rgb or ATTR_specular_rgb. These attributes are no-ops and have never worked.
Don't use ATTR_diffuse to change the color of your object. Use a small solid-colored patch of your texture - it's much faster. The only reason for ATTR_diffuse is in conjunction with ATTR_emission. I will explain this special usage in another post.
Don't use ATTR_shade_flat. You can represent any shading, smooth or flat, with ATTR_shade_smooth. In AC3D this means clicking the 'smooth' button and using the crease-angle to create the appropriate look for your object.
Don't use ATTR_poly_os with huge offsets. Even if you stack several layers of offset polygons on top of each other, you do not need to make each offset larger and larger; the offset only needs to be large enough to put the polygon over the base terrain. An offset of 1 or 2 is almost always adequate.
A few other warnings: Don't rely on your objects being drawn in a certain order in the sim - this will change! Don't put lights in the middle of your objects - when textured they will only look right at the end of the object.
Tuesday, January 31, 2006
One Texture Per Object 4-Evuh
I sometimes get requests for two features:
The issue with the first request is simply framerate - see my previous post on the "crayon rule". X-Plane has to do a certain amount of setup work for each OBJ it draws. Therefore we have two ways to implement multiple textures per object.*
If we want to obey the crayon rule, we would draw the parts of the objects that use the first texture first for all objects, then change textures once and draw the second texture. But this isn't so good - if you are using polygon offset or translucency or any other feature that requires the object to be drawn in order, those features won't work. Also we do a certain amount of setup work per object - we now have to do that work twice! This is about the same as doubling the number of objects. Ouch!
Or we can disregard the crayon rule and change textures once per object. That would be performance death . No optimization has more impact on framerate than minimizing texture changes! This is why X-Plane keeps moving from multiple textures to one texture - in both airplanes and objects! We do it for speed!
So that's why we require one texture. But why not zero textures? Well, there are a few issues with allowing untextured objects:
We do allow you to tint your object with ATTR_diffuse, but do not do this! The performance of the diffuse attribute is way worse than using a bit of your texture for color. ATTR_diffuse only has one legitimate use - paired with ATTR_emission to allow for the creation of objects that are self-lighting in low-light daytime conditions. I will post about this in a future entry.
* For OpenGL geeks: technically with pixel shaders you could bind multiple textures simultaneously and write a shader to select one, perhaps by mapping ST coordinates from a virtual texture space. So perhaps someday when we require pixel shaders for X-Plane and they are ubiquitous we can re-examine this.
- The ability to use more than one texture per OBJ.
- The ability to not use a texture per OBJ.
The issue with the first request is simply framerate - see my previous post on the "crayon rule". X-Plane has to do a certain amount of setup work for each OBJ it draws. Therefore we have two ways to implement multiple textures per object.*
If we want to obey the crayon rule, we would draw the parts of the objects that use the first texture first for all objects, then change textures once and draw the second texture. But this isn't so good - if you are using polygon offset or translucency or any other feature that requires the object to be drawn in order, those features won't work. Also we do a certain amount of setup work per object - we now have to do that work twice! This is about the same as doubling the number of objects. Ouch!
Or we can disregard the crayon rule and change textures once per object. That would be performance death . No optimization has more impact on framerate than minimizing texture changes! This is why X-Plane keeps moving from multiple textures to one texture - in both airplanes and objects! We do it for speed!
So that's why we require one texture. But why not zero textures? Well, there are a few issues with allowing untextured objects:
- The request for an untextured object almost always includes a request to use colors on the object instead. This would actually possibly be slower, not faster. Color information is twelve bytes per vertex, while texture is only eight. Each vertex per object is now 32 bytes, a wonderful power of 2 that happens to be very efficient on certain modern hardware. With color we hit 36 bytes. Ouch! Remove the color (not useful anyway) and we have 24 bytes. (Why not use 4-bytes instead of 12 per color? The hardware works better with color as floating points, not bytes - surprising but validated by performance testing.)
- X-Plane would have to take time off from drawing to turn texturing off and then turn it on again. So unless we're going to stop texturing for MOST of the sim, this "state change" hurts us.
- The actual process of drawing with a texture is basically free on modern hardware. Either the chip has dedicated hardware to do the texturing that would be idle otherwise or, if it's a pixel-shader chip, the ability of the card to texture is so far ahead of what X-Plane needs that the issue is moot.
We do allow you to tint your object with ATTR_diffuse, but do not do this! The performance of the diffuse attribute is way worse than using a bit of your texture for color. ATTR_diffuse only has one legitimate use - paired with ATTR_emission to allow for the creation of objects that are self-lighting in low-light daytime conditions. I will post about this in a future entry.
* For OpenGL geeks: technically with pixel shaders you could bind multiple textures simultaneously and write a shader to select one, perhaps by mapping ST coordinates from a virtual texture space. So perhaps someday when we require pixel shaders for X-Plane and they are ubiquitous we can re-examine this.
Wednesday, January 25, 2006
Fast Framerates on Intel Macs? YES!
I'm sitting in a lab at Apple with Austin; that screenshot is X-Plane 835 running on a 2 ghz DuoCore Intel Macintosh in a 20-inch iMac, 512 MB of RAM. That's 25 miles of visibility, insane roads, mega-tons of objects, 30 fps.
One reason why X-Plane runs so fast on the new iMacs is that the graphics card is an ATI X1600 - that's a big step up for the iMac.
When will the Universal X-Plane be available? I can't say yet but we are making progress!
The war on .ter files
I've been getting a lot of bug reports where X-Plane quits with a message about a missing .ter file. The problem is almost always installation related:
The only X-System folders that can support global scenery DSFs are the ones that started from the X-Plane global scenery DVDs.
Therefore if you want to use X-Plane 832 with global scenery, you must first install X-Plane 821, then run the updater to upgrade to 832.
If you simply start with 832 and try to copy files then you may miss some of the artwork and the missing .ter file message will appear. I will try to improve the error message with some clear instructions in the future, but only a proper install can remedy the situation.
The only X-System folders that can support global scenery DSFs are the ones that started from the X-Plane global scenery DVDs.
Therefore if you want to use X-Plane 832 with global scenery, you must first install X-Plane 821, then run the updater to upgrade to 832.
If you simply start with 832 and try to copy files then you may miss some of the artwork and the missing .ter file message will appear. I will try to improve the error message with some clear instructions in the future, but only a proper install can remedy the situation.
Tuesday, January 24, 2006
Dispelling Rumors
There may be new apt.dat features in a future version of X-Plane. I really can't comment on what will be in future releases. But I can ssure you this much:
You will not have to change existing scenery to work with the new version of X-Plane! No upgrade will be required.
When we introduced DSFs we continued to support ENVs; when we introduced OBJ8 we continued to support OBj7 (and OBJ2!)...whatever we do to the apt.dat file, whether it be a new version or just new features, X-Plane will continue to work with existing apt.dat files as it always has.
Austin announced that X-Plane runs on Intel-based Macintoshes, and Steve Jobs announced that the Intel-based iMacs are 2-to-3 times as iMac G5s. But here's the catch:
X-Plane 832 is not a universal binary - it has no native Intel code in it. So it runs under emulation on the Intel-based Macintoshes, which really hurts performance. So do not expect god framerates on Intel Macintoshes in 832 - you'll have to wait for a universal binary version of X-plane for that.
Hopefully I've dispelled some rumors. This won't dispell anything, but..I am writing this post from Cupertino.
You will not have to change existing scenery to work with the new version of X-Plane! No upgrade will be required.
When we introduced DSFs we continued to support ENVs; when we introduced OBJ8 we continued to support OBj7 (and OBJ2!)...whatever we do to the apt.dat file, whether it be a new version or just new features, X-Plane will continue to work with existing apt.dat files as it always has.
Austin announced that X-Plane runs on Intel-based Macintoshes, and Steve Jobs announced that the Intel-based iMacs are 2-to-3 times as iMac G5s. But here's the catch:
X-Plane 832 is not a universal binary - it has no native Intel code in it. So it runs under emulation on the Intel-based Macintoshes, which really hurts performance. So do not expect god framerates on Intel Macintoshes in 832 - you'll have to wait for a universal binary version of X-plane for that.
Hopefully I've dispelled some rumors. This won't dispell anything, but..I am writing this post from Cupertino.
Thursday, January 19, 2006
The Crayon Rule
Quick note: the region system will go in a later version of X-Plane, not 8.30. It's coming soon though! I will try to post a spec soon...I am way behind on my documentation.
Anyway, the "Crayon Rule": the worst thing you can do for frame-rate in X-Plane is change textures.
The name comes from a conversation almost three years ago when Austin explained to Sergio the cost of changing textures like this: drawing in OpenGL is like drawing with a crayon. Changing textures is like going back and getting a different crayon. It's a lot faster to do all the red in your picture and not have to change between red and blue crayons all the time.
The crayon rule continues to hold true and it's understandable why:
So the big things you can do as an author to obey the crayon rule are:
Anyway, the "Crayon Rule": the worst thing you can do for frame-rate in X-Plane is change textures.
The name comes from a conversation almost three years ago when Austin explained to Sergio the cost of changing textures like this: drawing in OpenGL is like drawing with a crayon. Changing textures is like going back and getting a different crayon. It's a lot faster to do all the red in your picture and not have to change between red and blue crayons all the time.
The crayon rule continues to hold true and it's understandable why:
- The texture you are using to draw is usually in VRAM. But the texture you are changing to could be in physical RAM (and must be transferred to the card), or worse it could have been paged out and must come from disk.
- The drawing pipeline must be stopped and reconfigured with the texture change.
- These things involve the host machine and CPU. Graphics cards are getting faster but CPUs are not - at least not much faster. So while the new graphics cards will be able to draw twice as many triangles, they will not be able to change textures twice as fast. In other words, we're stuck with the crayon rule for a long time.
So the big things you can do as an author to obey the crayon rule are:
- Use fewer large textures instead of many small ones.
- Group your polygons that use the panel texture together - switching to the panel texture is a crayon change.
Tuesday, January 17, 2006
Irresponsible Programming
I like to pretend I am a responsible software engineer, but...I am not. X-Plane is RC2 right now but I have just hacked in the "region system".
Put simply, the region system allows you to make collections libaries of objects (in scenery packages) to customize the look of the scenery, that only apply to a region of the world, rather than globally. This lets you make sure your custom Igloos are not used in the Bahamas.
The reason for this impulse is that we are setting the stage for non-US global scenery objects. I will try to post some docs soon and a tutorial showing how to customize X-Plane's objects.
Put simply, the region system allows you to make collections libaries of objects (in scenery packages) to customize the look of the scenery, that only apply to a region of the world, rather than globally. This lets you make sure your custom Igloos are not used in the Bahamas.
The reason for this impulse is that we are setting the stage for non-US global scenery objects. I will try to post some docs soon and a tutorial showing how to customize X-Plane's objects.
Friday, January 13, 2006
What's an Async Task? (Is it good with jam?)
You may have seen a crpytic message like "async task jammed" or "thread timed out" with X-plane 8.20 or 8.30, after which the sim quits. Here's what's going on and what we're trying to do to fix this.
X-Plane offloads part of the scenery load process to a second processor. (If you don't have a second CPU, the first one does the work, but the operating system switches between the two tasks automatically.) For example, X-Plane "outsources" the work of turning a road (which is just a line segment in a scenery file) into real 3-d roads using the second CPU.
X-plane puts a request for the roads out to the second CPU, and then every frame it checks to see if it's done. If the roads are processed, it can then send them to the video card and go on to the next thing to be done. If the roads are not finished - no biggie - it just checks again on the next frame.
Now here's where the trouble begins: sometimes X-Plane needs the outsourced work to be done immediately. For example, when we load new scenery we have to recycle the memory from the old scenery. This requires the work of building the roads on the old scenery to be done! When this happens, X-Plane waits (up to 2 minutes) for the background work to finish.
If after 2 minutes the work is not done, X-Plane gives up and puts up a cryptic message like "async task jammed" or "thread timed out". (In 830 beta 11 the message should be less cryptic.) I must admit: I do not understand how this condition happens; it has never happened on my machine or Austin's.
Here's what we're doing to alleviate the problem: in X-Plane 830 you can turn off the "load scenery in the background" option. When this happens, rather than outsource jobs to the second CPU, X-Plane will do the work immediately. This will have two effects:
I hope to get the background code to be more reliable in 840, but in the meantime you should be able to fly (etiher with or without background loading) in 830 with reasonable results if you set your rendering settings appropriately.
X-Plane offloads part of the scenery load process to a second processor. (If you don't have a second CPU, the first one does the work, but the operating system switches between the two tasks automatically.) For example, X-Plane "outsources" the work of turning a road (which is just a line segment in a scenery file) into real 3-d roads using the second CPU.X-plane puts a request for the roads out to the second CPU, and then every frame it checks to see if it's done. If the roads are processed, it can then send them to the video card and go on to the next thing to be done. If the roads are not finished - no biggie - it just checks again on the next frame.
Now here's where the trouble begins: sometimes X-Plane needs the outsourced work to be done immediately. For example, when we load new scenery we have to recycle the memory from the old scenery. This requires the work of building the roads on the old scenery to be done! When this happens, X-Plane waits (up to 2 minutes) for the background work to finish.
If after 2 minutes the work is not done, X-Plane gives up and puts up a cryptic message like "async task jammed" or "thread timed out". (In 830 beta 11 the message should be less cryptic.) I must admit: I do not understand how this condition happens; it has never happened on my machine or Austin's.
Here's what we're doing to alleviate the problem: in X-Plane 830 you can turn off the "load scenery in the background" option. When this happens, rather than outsource jobs to the second CPU, X-Plane will do the work immediately. This will have two effects:
- There will be slight pauses as you fly while x-plane stops and builds roads, etc.
- There is no way that we can get stuck waiting for the second CPU, because it won't be used.
I hope to get the background code to be more reliable in 840, but in the meantime you should be able to fly (etiher with or without background loading) in 830 with reasonable results if you set your rendering settings appropriately.
Fun With Numbers
"Scenery load is now twice as fast!" Where do Austin and I come up with these claims? Here's a little bit of info on on our performance tuning process.
When we work on performance, we always measure a baseline metric and then look at this metric while we work on the code. The test situation must be the same every time, so usually we use a standard plane position (like on a known runway) for framerate, or a known scenery box (for scenery loading).
Here's the statistics for X-plane scenery load before and after tuning the DSF loader:
These numbers are not normaly visible; a special sim option available to us causes it to print the load time per DSF. The nubmers are in microseconds and shown per DSF.
So on immediate inspection you can see improvements of between 140 and 205%. But to really understand what this means you have to consider all of the fine print:
These stats are with objects and roads turned off - this is with scenery on minimal settings in both cases. With more 3-d stuff turned on, the overall loadtime might be better or worse; these statistics don't tell us anything about that.
These tests were done on the same 6 DSFs in both cases, but other DSFs might have different improvements. You can see that even in the six DSFs here the amount of improvement varies a lot!
These tests were done on my dual 1.8 ghz G5. This is a fast machine for a Mac (but probably only average compared to modern desktop PCs). The speed-up could vary on other machines that don't have similar performance characteristics, since scenery loading depends on the speed of the CPU, memory bandwidth, I/O performance, etc.
This is all a long-winded way of saying "your mileage may vary". When Austin and I post performance numbers, they are calculated in strict test conditions with all variables controlled. But since these statistics represent one datapoint, they are not universal changes, just indicators that our code changes are going in the right direction.
(Could it be that an optimization makes a difference on our machines but not yours? Yes - this is always a risk! We try to run on different machines and we listen to user feedback on performance to catch this case.)
When we work on performance, we always measure a baseline metric and then look at this metric while we work on the code. The test situation must be the same every time, so usually we use a standard plane position (like on a known runway) for framerate, or a known scenery box (for scenery loading).
Here's the statistics for X-plane scenery load before and after tuning the DSF loader:
Before optimize
DSF load time: 2297036 for file +33-119.dsf
DSF load time: 4614647 for file +33-118.dsf
DSF load time: 4251205 for file +33-117.dsf
DSF load time: 5137940 for file 34-119.dsf
DSF load time: 5038098 for file +34-118.dsf
DSF load time: 3893140 for file +34-117.dsf
After Optimize
DSF load time: 1634270 for file +33-119.dsf
DSF load time: 2156187 for file +33-118.dsf
DSF load time: 1882647 for file +33-117.dsf
DSF load time: 2418666 for file +34-119.dsf
DSF load time: 2318480 for file +34-118.dsf
DSF load time: 1819997 for file +34-117.dsf
These numbers are not normaly visible; a special sim option available to us causes it to print the load time per DSF. The nubmers are in microseconds and shown per DSF.
So on immediate inspection you can see improvements of between 140 and 205%. But to really understand what this means you have to consider all of the fine print:
These stats are with objects and roads turned off - this is with scenery on minimal settings in both cases. With more 3-d stuff turned on, the overall loadtime might be better or worse; these statistics don't tell us anything about that.
These tests were done on the same 6 DSFs in both cases, but other DSFs might have different improvements. You can see that even in the six DSFs here the amount of improvement varies a lot!
These tests were done on my dual 1.8 ghz G5. This is a fast machine for a Mac (but probably only average compared to modern desktop PCs). The speed-up could vary on other machines that don't have similar performance characteristics, since scenery loading depends on the speed of the CPU, memory bandwidth, I/O performance, etc.
This is all a long-winded way of saying "your mileage may vary". When Austin and I post performance numbers, they are calculated in strict test conditions with all variables controlled. But since these statistics represent one datapoint, they are not universal changes, just indicators that our code changes are going in the right direction.
(Could it be that an optimization makes a difference on our machines but not yours? Yes - this is always a risk! We try to run on different machines and we listen to user feedback on performance to catch this case.)
Wednesday, January 11, 2006
Crash bang crunch
Thank you to everyone who has reported the crash bug in beta-8. I found the crash this morning thanks to some helpful bug reports (technical explanation: boneheaded code on my part) and the fix should be in the next beta.
Also it looks like QuickTime 7.0.4 isn't compatible with the AC3D x-plane export plugin. This makes no sense to me...I spent over an hour remotely debugging on a user's machine (thanks Peter!) and came to the conclusion that something is borked deep down inside a library. Somewhere. Maebe. I'll try to get this resolved soon but since I do not have an OS X 10.4 machine, it may take a little bit. In the meantime it is possible to go back to QuickTime 7.0.1.
Also it looks like QuickTime 7.0.4 isn't compatible with the AC3D x-plane export plugin. This makes no sense to me...I spent over an hour remotely debugging on a user's machine (thanks Peter!) and came to the conclusion that something is borked deep down inside a library. Somewhere. Maebe. I'll try to get this resolved soon but since I do not have an OS X 10.4 machine, it may take a little bit. In the meantime it is possible to go back to QuickTime 7.0.1.
Tuesday, January 10, 2006
Fun with Animation
Here are a few notes on animation:
The animation commands for OBJ8 take two changes (a start and end rotation or translation) and two dataref values and map them. For example,
The answer is: the result is extrapolated. If this dataref goe sto 2.0, the rotation will go to 180 degrees. (How I came up with that is left as an exercise for the reader.)
You can take advantage of this to animate rotating things like radar dishes and beacons using the sim/time/total_running_time_sec dataref. This dataref is the number of seconds x-plane has been running. For examlpe:
When the time is 0 the rotation is 0. 5 seconds later the rotation has made it up to 360, one full turn. Then what? At 6 seconds the rotation will now be 420 degrees. But 420 degrees is the same as 60 degrees; you can't tell whether the object has made one, two or a hundred full revolutions before turning the extra sixty degrees. So the rotation is actually becoming a huge number but the wrap-around cannot be seen, making the appearance of continuous rotation.
Rotations always occur around the origin. But what if we want to rotate around another point in our object? We must use the ANIM_translate command to move the rotation to where we want. For example:
Note that the animation-translate commands have the same coordinates for both translations and no dataref. This is a static translation - it is alawys the same and its sole purpose is to move the rotation to be around a useful point. It is always followed by an opposite translation.
(It is possible to omit the second translation and simply build your animated sub-part with 0,0,0 = rotation point instead of 0,0,0 = the object's location in x-plane. But this is an advanced optimization.)
An example:
A rotating radar dish
And from the end of the OBJ:
The animation commands for OBJ8 take two changes (a start and end rotation or translation) and two dataref values and map them. For example,
ANIM_rotate 0 1 0 -40 70 0 1 sim/flightmodel/whateverwould rotate the following geometry around the Y axis from -40 to 70 degrees as the dataref goes from 0 to 1. Intermediate values between 0 and 1 are interpolated. (For example, 0.5 maps to 15 degrees.) But what happens if the dataref exceeds the bounds specified?
The answer is: the result is extrapolated. If this dataref goe sto 2.0, the rotation will go to 180 degrees. (How I came up with that is left as an exercise for the reader.)
You can take advantage of this to animate rotating things like radar dishes and beacons using the sim/time/total_running_time_sec dataref. This dataref is the number of seconds x-plane has been running. For examlpe:
ANIM_rotate 0 1 0 0 360 0 5 sim/time/total_running_time_secThis will cause the following geometry to continuously rotate, making a full rotation every 5 seconds. But why does the animation loop? Well, let's take a look in detail:
When the time is 0 the rotation is 0. 5 seconds later the rotation has made it up to 360, one full turn. Then what? At 6 seconds the rotation will now be 420 degrees. But 420 degrees is the same as 60 degrees; you can't tell whether the object has made one, two or a hundred full revolutions before turning the extra sixty degrees. So the rotation is actually becoming a huge number but the wrap-around cannot be seen, making the appearance of continuous rotation.
Rotations always occur around the origin. But what if we want to rotate around another point in our object? We must use the ANIM_translate command to move the rotation to where we want. For example:
ANIM_trans 3 4 5 3 4 5 0 0 no_refThis will rotate the geometry around the Y axis from -90 to 90 degrees as the circle goes from -1 to 1. But the rotation happens around the point 3,4,5.
ANIM_rotate 0 -1 0 -90 90 -1 1 jetway/circle1
ANIM_trans -3 -4 -5 -3 -4 -5 0 0 no_ref
Note that the animation-translate commands have the same coordinates for both translations and no dataref. This is a static translation - it is alawys the same and its sole purpose is to move the rotation to be around a useful point. It is always followed by an opposite translation.
(It is possible to omit the second translation and simply build your animated sub-part with 0,0,0 = rotation point instead of 0,0,0 = the object's location in x-plane. But this is an advanced optimization.)
An example:
A rotating radar dish
And from the end of the OBJ:
LIGHTS 0 2
TRIS 0 1536
ANIM_begin
ANIM_trans 20.226 22.468 -0.039 20.226 22.468 -0.039 0.0 1.0 no_ref
ANIM_rotate 0 -1 0 0 360 0 6 sim/time/total_running_time_sec
TRIS 1536 1116
ANIM_end
My Supervisor (no it's not Austin)
One of the secrets of X-Plane development revealed...Nala supervises my work very carefully to be sure I not introducing bugs into the rendering engine. In the event you get threading errors in the next 830 beta, you now know who to blame!
Subscribe to:
Posts (Atom)