Scenery loading has changed a lot in 8.30. Before 8.30 we loaded 6 scenery tiles (DSFs or ENVs) and completely built them up into memory before we let you fly. We then loaded any new texures while flying. The result is a possible hit on framerate right after a scenery load for textures, and a very long pause while the scenery area gets built.
In 8.30 we now read the full DSF and build the terrain, then start the sim running. We then build the local 3-d "clutter" (objects and roads) while you fly, but we only build up the ones near your plane. As you fly, the ones behind you are destroyed and new ones in front of you are built.
This means a few things:
- Shorter scenery pauses, because we have to do less before we can start the sim.
- Lower framerates right after a load because the sim is doing extra work while you fly to finish the load.
- Lower framerates occaisionally while you fly as a few more roads are built and objects destroyed. This is pretty minimal though because these items come into range gradually.
- Lower memory usage, since only the roads nearby are fully built, not all of them.
This strategy works out particularly well for people with dual-processor or dual-core machines; some of the work to do this "clutter-building" is done on the second CPU while you fly, so the framerate hit is a lot less noticable. Basically we are taking work that was done by the main CPU while you waited and doing it on the second CPU while you fly.
One problem still in 830: when you place the plane, we do not pre-build the roads (but we DO pre-load the textures). If we did not preload the textures, you would be placed in a gray area and see the textures come in. So hopefully in a later beta I will preload the roads if you place the plane using the airport-picking dialog box. This should help provide good framerates as soon as your plane is placed.
Friday, December 30, 2005
Wednesday, December 28, 2005
The emperor is not as forgiving as I am....
I have a new program called "Env2Overlay" that converts the custom obojects in an ENV file to overlay DSFs. This makes it very easy to convert custom airports to version 8.
However one thing I've found is that a lot of ENV scenery contains references to missing objects. The ENV loader historically has worked even with objects missing. The DSF loader will refuse to load scenery if objects are missing.
For now I am going to leave the DSF loader in "unforgiving" mode so that missing objects are an error. I think this is more useful to authors: better to clean up the scenery package while converting it rather than convert it to DSF and continue to have missing objects; I think this indicates a packaging problem or other lost content!
However one thing I've found is that a lot of ENV scenery contains references to missing objects. The ENV loader historically has worked even with objects missing. The DSF loader will refuse to load scenery if objects are missing.
For now I am going to leave the DSF loader in "unforgiving" mode so that missing objects are an error. I think this is more useful to authors: better to clean up the scenery package while converting it rather than convert it to DSF and continue to have missing objects; I think this indicates a packaging problem or other lost content!
Tuesday, December 27, 2005
Overlays work!
I am pleased to say that I have overlays working in X-Plane 8.30. Overlays are DSFs that contain only objects and are loaded on top of the existing terrain. Overlays allow you to add objects to the global scenery to customize airports, etc.
The overlay system will become more powerful over time, but it should be available shortly when X-Plane 830 goes beta.
DISCLAIMER: the overlay DSF listed below may change in format during beta! This is only illustrated to show the relative brevity of overlays compared to full DSFs. (Users who have turned full DSFs to text know how big they can be.)
To give you an idea of what an overlay DSF looks like:
Tools to make overlays coming soon!
The overlay system will become more powerful over time, but it should be available shortly when X-Plane 830 goes beta.
DISCLAIMER: the overlay DSF listed below may change in format during beta! This is only illustrated to show the relative brevity of overlays compared to full DSFs. (Users who have turned full DSFs to text know how big they can be.)
To give you an idea of what an overlay DSF looks like:
A
800
DSF2TEXT
PROPERTY sim/require_object 1/0
PROPERTY sim/overlay 1
PROPERTY sim/west -71
PROPERTY sim/east -70
PROPERTY sim/north 43
PROPERTY sim/south 42
PROPERTY sim/planet earth
OBJECT_DEF KBOS_TAXI:15R33L.obj
OBJECT_DEF KBOS_TAXI:279.obj
OBJECT_DEF KBOS_TAXI:C.obj
OBJECT_DEF KBOS_TAXI:D.obj
OBJECT 0 -70.999146 42.359913 76.000000
OBJECT 0 -70.997780 42.359650 76.000000
OBJECT 0 -70.997566 42.358742 75.000000
OBJECT 0 -70.996155 42.358467 75.000000
OBJECT 0 -70.993088 42.355259 75.000000
OBJECT 1 -70.999634 42.358353 136.000000
OBJECT 1 -70.998306 42.358078 0.000000
OBJECT 1 -70.996956 42.358261 135.000000
OBJECT 1 -70.996773 42.358913 135.000000
OBJECT 2 -70.999100 42.358513 254.000000
OBJECT 2 -70.994141 42.355080 136.000000
OBJECT 3 -70.992317 42.361107 76.000000
OBJECT 3 -70.998398 42.359409 314.000000
Tools to make overlays coming soon!
Monday, December 26, 2005
Thank You
I just wanted to say thank you to everyone who has emailed in the last few days - it's been very uplifting! I'm sorry I haven't been able to answer everyone individually - but I do appreciate the kind words!
Friday, December 23, 2005
Instant Gratification
I've spent the last week performance tuning X-Plane 830. Performance tuning is a rewarding change of pace from regular global scenery development, because the gratification is instant. Run the sim, get a framerate. Twaek code, run the sim again, get a higher framerate. The gains are immediate and quantifiable.
By comparison, work on global scenery is an exercise in deferred gratification and unpleasant surprises. I write a lot of the code before I have artwork; the results don't look good until later when the real images are dropped in.
This is a picture of X-Plane 8 beaches with testing textures; Sergio made a beach texture with a descriptive word on each segment and colors indicating their uses. In the simulator, I can verify that the texture placement is correct, the textures line up correctly, they're not cut off, etc. When people post screenshots of the new global scenery I am usually surprised by what it looks like, because I spent most of the development time looking at images like the one on the left, not the final product.
The unpleasant surprises come from the size of the world: given how big and varied the world is (and how unreliable our global data can be at times), there will always be a location where the particular local data causes our algorithm to make bogus results. A few users have reported vertical walls of water; a classic global data varation problem. Given all of the combinations of shapes of water bodies overlayed on elevation data, our flattening algorithm doesn't always get it right. Since the water body and elevation data may come from different sources, it's possible that the water body data is saying there is a lake where the elevation data is saying there is a mountain!
Anyway, enough babbling; Happy Holidays to everyone! Soon we will have a New Year and a new beta of X-Plane - in the meantime, enjoy the time with your families and fly safe!
By comparison, work on global scenery is an exercise in deferred gratification and unpleasant surprises. I write a lot of the code before I have artwork; the results don't look good until later when the real images are dropped in.
This is a picture of X-Plane 8 beaches with testing textures; Sergio made a beach texture with a descriptive word on each segment and colors indicating their uses. In the simulator, I can verify that the texture placement is correct, the textures line up correctly, they're not cut off, etc. When people post screenshots of the new global scenery I am usually surprised by what it looks like, because I spent most of the development time looking at images like the one on the left, not the final product.The unpleasant surprises come from the size of the world: given how big and varied the world is (and how unreliable our global data can be at times), there will always be a location where the particular local data causes our algorithm to make bogus results. A few users have reported vertical walls of water; a classic global data varation problem. Given all of the combinations of shapes of water bodies overlayed on elevation data, our flattening algorithm doesn't always get it right. Since the water body and elevation data may come from different sources, it's possible that the water body data is saying there is a lake where the elevation data is saying there is a mountain!
Anyway, enough babbling; Happy Holidays to everyone! Soon we will have a New Year and a new beta of X-Plane - in the meantime, enjoy the time with your families and fly safe!
Wednesday, December 21, 2005
Improving Framerate
I've received a few emails requesting recommendations for graphics cards...unfortunately I must admit: I do not know which graphics cards will improve framerate the most. The benefit you get from a new graphics card depends on the other components of your system. Also I haven't tried every graphics card so I don't have good hard data.
I will suggest this generic advice though: for nVidia cards try the 6600 or 6800 rather than one of the FX series cards. For any card, make sure it supports DX9 pixel shaders (for nVidia this means a 5000 or 6000 series; for ATI this means 9500 or greater, or an x600 or greater). I would not recommend a card with HyperMemory or TurboCache; these schemes use your system memory as VRAM. The cards are cheaper but the performance won't be as good as real RAM.
I've also received questions about settings for the best framerate. If X-Plane 821 is fogging up with the new scenery, 830 may provide some amount of relief; we're working on performance optimization. I can't say how much improvement we'll get, but I would recommend waiting for 830 before evaluating your hardware for an upgrade.
There is one setting that can really help mid-level systems: the "world detail distance" rendering setting. If you can run with a few objects, try changing it from 'default' to 'low'. This will cause objects to disappear sooner as you fly away from them; because the sim will not draw as many objects in the distance, you may be able to turn up the object density. In otherwords, you can trade off the density of near objects vs the density of far away objects.
(Also beware of textured lights; they really impact framerate heavily.)
I will suggest this generic advice though: for nVidia cards try the 6600 or 6800 rather than one of the FX series cards. For any card, make sure it supports DX9 pixel shaders (for nVidia this means a 5000 or 6000 series; for ATI this means 9500 or greater, or an x600 or greater). I would not recommend a card with HyperMemory or TurboCache; these schemes use your system memory as VRAM. The cards are cheaper but the performance won't be as good as real RAM.
I've also received questions about settings for the best framerate. If X-Plane 821 is fogging up with the new scenery, 830 may provide some amount of relief; we're working on performance optimization. I can't say how much improvement we'll get, but I would recommend waiting for 830 before evaluating your hardware for an upgrade.
There is one setting that can really help mid-level systems: the "world detail distance" rendering setting. If you can run with a few objects, try changing it from 'default' to 'low'. This will cause objects to disappear sooner as you fly away from them; because the sim will not draw as many objects in the distance, you may be able to turn up the object density. In otherwords, you can trade off the density of near objects vs the density of far away objects.
(Also beware of textured lights; they really impact framerate heavily.)
Tuesday, December 20, 2005
DSF2Text - Just a Link in the Chain
"Five long years I thought you were my manA new version of DSF2Text is out, one that works with the global scenery, and I have already seen authors adding custom objects to the global scenery. Great! I have also received some emails from users having trouble with DSF2Text. Please keep this in mind before you despair:
But I found out I'm just a link in your chain"
-- Aretha Franklin, Chain of Fools
DSF2Text is designed to be a link in a chain of scenery editing tools; its primary audience is programmers who can feed their program's text files into DSF2Text to generate final DSF files. I never intended to have people editing 45 MB text files to make scenery; the tutorial on how to add objects to DSFs is really a hack.
Please bear with us; in the next few weeks I hope to unveil some new technologies and tools that will be much easier and faster to use. In the meantime if you can get DSF2Text to work great, but if not, please do not worry; better things are on the way.
(As a side note, if you are working on payware X-Plane scenery and are modifying our DSFs, please contact me by email; the X-Plane EULA allows you to make derived freeware from our artwork, but not payware. The new overlay DSFs will allow you to create DFSs that are totally your own.)
Thursday, December 15, 2005
The Darkest Night
Sergio is on the blog now! Sergio is the Godfather of global scenery; his immense and tireless work very much make the global scenery what it is. Without him all we have is a huge pile of triangles.
Massimo and Cristianno also work on the artwork for the globabl scenery and have put in long hours and really gone the extra mile working on the artwork. To appreciate what the three amici have done, try viewing the raw PNG files from the scenery and witness the care, detail artistry and precision of their work.
[EDIT -- this bug is fixed in X-Plane 8.30. so the following only applies to older versions of the sim.]
On to the night...this is a bug that gets reported at least once a day: as it gets darker in X-Plane, lights appear on the cities - a few at a time, then more - then poof! They all disappear and everything goes black!
You may notice that if you turn off both the landing and taxi lights then the city lights come back on again. What's going on?
Well, this bug has been in X-Plane since the landing light was first introduced, and it's actually a limitation: X-Plane cannot draw the night lighting overlays and the landing light at the same time because the landing light code was written years ago for older, less powerful graphics hardware. Before we had global scenery with lit cities and night, this was annoying but relatively unnoticable. Now that the cities are bright at night, this limitation is very clear.
Perhaps it can be fixed -- I don't know for sure; I am working with this code now. I can tell you this though: if you have older hardware (GeForce 2, GeForce 4 MX, Rage 128, ATI Radeon 7000, maybe a few others) then it cannot be fixed for you - your card doesn't have power to draw the effect correctly. If you have a newer card (GeForce FX, 6000 or 7000 series, ATI Radeon 9000 series, or Radeon "x" series) you should be able to use a fix if we can develop one. In the middle are cards whose capabilities I don't know off hand.
Massimo and Cristianno also work on the artwork for the globabl scenery and have put in long hours and really gone the extra mile working on the artwork. To appreciate what the three amici have done, try viewing the raw PNG files from the scenery and witness the care, detail artistry and precision of their work.
[EDIT -- this bug is fixed in X-Plane 8.30. so the following only applies to older versions of the sim.]
On to the night...this is a bug that gets reported at least once a day: as it gets darker in X-Plane, lights appear on the cities - a few at a time, then more - then poof! They all disappear and everything goes black!
You may notice that if you turn off both the landing and taxi lights then the city lights come back on again. What's going on?
Well, this bug has been in X-Plane since the landing light was first introduced, and it's actually a limitation: X-Plane cannot draw the night lighting overlays and the landing light at the same time because the landing light code was written years ago for older, less powerful graphics hardware. Before we had global scenery with lit cities and night, this was annoying but relatively unnoticable. Now that the cities are bright at night, this limitation is very clear.
Perhaps it can be fixed -- I don't know for sure; I am working with this code now. I can tell you this though: if you have older hardware (GeForce 2, GeForce 4 MX, Rage 128, ATI Radeon 7000, maybe a few others) then it cannot be fixed for you - your card doesn't have power to draw the effect correctly. If you have a newer card (GeForce FX, 6000 or 7000 series, ATI Radeon 9000 series, or Radeon "x" series) you should be able to use a fix if we can develop one. In the middle are cards whose capabilities I don't know off hand.
Wednesday, December 14, 2005
The Overlay System
I have received a number of emails from people trying to use DSF2Text (a fixed version is coming real soon). From this I can only guess that people are trying to customize the global scenery!
We are working on the "overlay" system - when this is a little more hammered out (a week or two) I'll try to describe it in detail. But the basic idea is to allow additional DSFs that contain only objects and other 3-d "clutter" to be superimposed on the basic terrain, adding to or replacing the default objects.
This would have a few benefits:
- Small distribution size for custom airports and other such object-based scenery (because you don't have to redistribute our mesh).
- No legal limitations (since you don't have to copy our scenery).
- Since many overlays can go on one base, a user will be able to install a number of custom airports.
- Probably the small overlay DSFs will be easier to work with - a 25 MB DSF makes an insanely huge text file in DSF2Text.
Anyway, this is all very experimental; I will have more info soon, but all of this is subject to change. For now I can only say with certainty that we are working on this problem!
We are working on the "overlay" system - when this is a little more hammered out (a week or two) I'll try to describe it in detail. But the basic idea is to allow additional DSFs that contain only objects and other 3-d "clutter" to be superimposed on the basic terrain, adding to or replacing the default objects.
This would have a few benefits:
- Small distribution size for custom airports and other such object-based scenery (because you don't have to redistribute our mesh).
- No legal limitations (since you don't have to copy our scenery).
- Since many overlays can go on one base, a user will be able to install a number of custom airports.
- Probably the small overlay DSFs will be easier to work with - a 25 MB DSF makes an insanely huge text file in DSF2Text.
Anyway, this is all very experimental; I will have more info soon, but all of this is subject to change. For now I can only say with certainty that we are working on this problem!
Tuesday, December 13, 2005
One Frame Per Second
Let me commit this to writing: I WILL get new tools (DSF2Text, AC3D Plugin, etc.) out this week!!
The rendering engine has been taking up all my time this week. To give you an idea, I spent part of the weekend and Monday rewriting the way we handle physics interactions between the simulator and the graphics rendering engine...only to find the new code improved framerates by...one frame per second!
When I complained to Austin about this he said, "good, I'll take every one fp I can get", and I realized...this is how we make progress forward. If after a week we've got a 5 fp boost, that's definitely a big help to a user who is getting fog now.
(Of course, not all fps are created equal; a frame per second is the reciprical of the time the sim is taking, so a 1 fp improvement when the sim running at 20 fps is a huge victory; a 1 fp boost when the sim is running at 90 fps is negligable.)
The rendering engine has been taking up all my time this week. To give you an idea, I spent part of the weekend and Monday rewriting the way we handle physics interactions between the simulator and the graphics rendering engine...only to find the new code improved framerates by...one frame per second!
When I complained to Austin about this he said, "good, I'll take every one fp I can get", and I realized...this is how we make progress forward. If after a week we've got a 5 fp boost, that's definitely a big help to a user who is getting fog now.
(Of course, not all fps are created equal; a frame per second is the reciprical of the time the sim is taking, so a 1 fp improvement when the sim running at 20 fps is a huge victory; a 1 fp boost when the sim is running at 90 fps is negligable.)
Monday, December 12, 2005
New Scenery Framerate and Bugs
Wow...I went up to Boston for a weekend and came back to find a flood of email regarding the new scenery, a few hundred posts on the tech list, and blog comments and forum posts. If you didn't hear from me, please try a direct email; if that didn't work, please bear with me...I am on the verge of email bankruptcy.
Most of the posts have been on the subject of bugs and anomolies in the scenery and framerate problems. A little bit on both.
Bugs: bugs in the global scenery fall into the catagory of artwork problems and DSF problems. For example, the most common report I've seen is of the big green and brown patches in European urban areas. This is due to a bad text file (a typo - my fault!!) for the terrain artwork. This particular bug should be addressed in X-Plane 830 via an automatic update. Generally where the artwork is bad, we can provide a new artwork file in the update to fix things.
Where the DSF itself is bad there's not much we can do. Usually bad DSFs result from bad source data. This is a place where custom scenery will have to fill in the gap and provide local detail and customization that we can't provide with the very broad brush of global scenery.
We also have some algorithm limitations and data limitations. Non-CONUS road data is quite inaccurate, and CONUS road data does not have overpass information; both are a source of strange roadways. If you see a bug, please file a bug report! I can't keep up with forum posts to find potential problems.
Performance: X-Plane WILL run slower with the new scenery - that's because there's more data packed in there. Simply put I do not think we could make it look as good if we cut down the polygon counts to US DSF or ENV levels. I am working on engine improvements in X-Plane, so 830 may provide some relief, but please understand: X-Plane 821 is not slower than 820 - it's just doing a lot more work to show you each frame!
Most of the posts have been on the subject of bugs and anomolies in the scenery and framerate problems. A little bit on both.
Bugs: bugs in the global scenery fall into the catagory of artwork problems and DSF problems. For example, the most common report I've seen is of the big green and brown patches in European urban areas. This is due to a bad text file (a typo - my fault!!) for the terrain artwork. This particular bug should be addressed in X-Plane 830 via an automatic update. Generally where the artwork is bad, we can provide a new artwork file in the update to fix things.
Where the DSF itself is bad there's not much we can do. Usually bad DSFs result from bad source data. This is a place where custom scenery will have to fill in the gap and provide local detail and customization that we can't provide with the very broad brush of global scenery.
We also have some algorithm limitations and data limitations. Non-CONUS road data is quite inaccurate, and CONUS road data does not have overpass information; both are a source of strange roadways. If you see a bug, please file a bug report! I can't keep up with forum posts to find potential problems.
Performance: X-Plane WILL run slower with the new scenery - that's because there's more data packed in there. Simply put I do not think we could make it look as good if we cut down the polygon counts to US DSF or ENV levels. I am working on engine improvements in X-Plane, so 830 may provide some relief, but please understand: X-Plane 821 is not slower than 820 - it's just doing a lot more work to show you each frame!
Friday, December 09, 2005
A "real" Scenery Blog
I am converting the "scenery blog" to a real blog, meaning one that uses real blog software for RSS feeds, trackbacks, comment posts, all that good stuff.
A brief intro: I work for Austin Meyer on X-Plane, a cross-platform flight simulator based on real physics, developing the scenery system, scenery engine, 3rd party development kits, and the algorithms used to create the default scenery.
We just shipped the new "global scenery", a rendering of the entire world from 54S to 60N using SRTM2 data. Sergio (our lead artist) has posted some screenshots here. I'll try to post on topics of interest to scenery authors and X-Plane users.
A brief intro: I work for Austin Meyer on X-Plane, a cross-platform flight simulator based on real physics, developing the scenery system, scenery engine, 3rd party development kits, and the algorithms used to create the default scenery.
We just shipped the new "global scenery", a rendering of the entire world from 54S to 60N using SRTM2 data. Sergio (our lead artist) has posted some screenshots here. I'll try to post on topics of interest to scenery authors and X-Plane users.
Subscribe to:
Posts (Atom)