Friday, May 21, 2010

DSF Will Be Extended, Not Replaced

Austin attended the French X-Plane Congress last weekend; in response to some questions I have received, I want to clarify what is planned for DSF and X-Plane 10.

X-Plane 10 will extend DSF scenery capabilities by providing a number of new art asset types, as well as extensions to existing art asset file formats. We will not be changing the DSF file format or breaking compatibility with existing DSF scenery. If your DSF scenery works with version 9, I expect that it will work with version 10 as is.

(In fact, it looks like we do not even need to add new DSF extensions; DSF was designed to be a generic container for geometry data and properties, so we can usually extend X-Plane's scenery system by simply defining new art asset classes and properties.)

An example will illustrate what I mean by extending the art assets, not DSF.

In X-Plane 9, you create a forest (whether in a base mesh or overlay DSF) by creating a DSF polygon referencing a .for (forest) art asset. X-Plane will render this by filling the area inside the polygon with lots of trees.

In X-Plane 10, you will be able to optionally tell X-Plane to put the trees only on the edge of the polygon, rather than filling the entire inside. (This feature will be controlled by using values on the polygon parameter that are larger than 255, at least I think.) This will mean that you can use .for files and DSF polygon not only to create forest areas, but also to create rows of trees along the edges of roads or fields. A row of trees made by a .for file and DSF polygon will render much faster than a large number of individual OBJ-based trees.

This kind of art asset extension is similar to what we have already seen; X-Plane 850 introduced three new art asset types (.str object strings, .lin line paint, and .pol draped polygons) that all produced new rendering tricks using DSF polygons. These art assets were added to provide high performance rendering of airport surface areas. The new art assets didn't require any change to DSF because the representation of the position of these art assets is something DSF has always done: simple polygons.

DSF won't last forever, but at least for X-Plane 10 it looks like we can do a third full cycle of the sim using DSF as our base container format for scenery position data.

Saturday, May 15, 2010

Where Has All My VRAM Gone?

In the past I have suggested that there is a limit to the value of additional VRAM when buying a graphics card. When I posted this, Nicholas from the org emailed me to point out that with third party add-ons VRAM was a lot more important.

At the time I wasn't really convinced, but I've finally come around; it takes a while for the trend to get back to me. (I just don't have time to look at everyone's add-ons the way I used to 4 or 5 years ago.) It seems clear that airplane authors (and to some extent scenery authors) are using VRAM pretty aggressively. If you want to use third party add-ons and you care about texture res and texture sharpness, spring for some VRAM. It doesn't cost as much as it used to, and authors are starting to use it.

What Lives in VRAM

Bear in mind that in any discussion of how your video card operates, anything I post is informed speculation. The driver provides an abstraction (OpenGL) of what the hardware does, and a lot of the bookkeeping isn't visible to X-Plane at all. So what I am describing is typical of past video drivers that we have had insight to in the past, but it's not universal, and it's not at all guaranteed. (X-plane can't demand any of this behavior of the video driver.)

In order of how "stuck" in VRAM things are we have:
  • Video memory used for on-screen rendering. Depending on rendering settings you can lose anywhere from 12 to 24 MB of VRAM per million pixels on screen. So if you're running at 1920 x 1200, you might be using 50 MB of VRAM just for the screen. If you use FSAA, you're going to chew up VRAM even more aggressively. (Costs vary depending on the scheme; you might lose 4-16 MB x the FSAA level per million pixels on screen, depending on your GPU and driver.)
  • Off-screen rendering for things like the water reflections, the panels, the cloud shadows, the airplane shadow, etc. These don't have to be in VRAM all of the time, but they have to be in VRAM almost all of the time. Because they are created by the GPU, the driver tries hard not to move these out of VRAM. You might lose 6 to 16 MB of VRAM for these, depending on the airplane you use and settings. (Given 4 1024x1024 panel regions, the panel will chew up 32 MB!)
  • Textures end up in VRAM, but only when they are used. The key here is "working set". Only texures that are drawn need to be in VRAM, so over time stuff that isn't on screen will be removed from VRAM. This is why when you see "600 MB of texture memory" in the rendering settings, there is no need to panic. The working set is usually much smaller.
  • OBJ geometry actually lives in VRAM too, sometimes. Again this is a working set issue; objects that aren't drawn don't get cached there.
  • Textures from airplanes and scenery packages that are not loaded don't ever end up VRAM or even system memory; we only load what we need. Paged orthophotos have their resolution reduced while you fly, which makes their VRAM footprint quite small, even when drawn.
There are a few things authors and users are doing now that use up a lot of VRAM.
  • Monitors have gotten bigger; the VRAM used for the screen itself can never be swapped out, so the advent of the 1920x1200 LCD has taken its toll.

  • Since the panel texture is drawn off-screen, the panel texture is in an expensive category of VRAM use. Authors can limit the cost of this by using a single 1024x1024 panel region texture, if possible.

  • There is a hidden cost here: we pack together instrument textures into "atlases" to help with performance. The problem is that we pack for fit. Some of your instruments may be hidden but loaded into VRAM anyway because they sit in the same atlas texture as other instruments that are drawn. Thus you may be paying for the VRAM used by your entire panel even if a lot of it is hidden.

  • Because a lot of this VRAM is going to airplanes, reducing texture resolution doesn't have as much impact as it used to; X-Plane tries to keep the user's plane's resolution as high as possible since it is viewed up close. The panel cannot have its resolution reduced at all.

But It Works Sometimes

I think what drives users crazy about VRAM exhaustion is that X-Plane will sometimes run smoothly, and then fail later. And sometimes really strange things, like moving X-Plane to the background, then the foreground, or changing liveries or rendering settings in a trivial way, will change performance.

I have discussed this a bit in past posts. But the key here is "working set":
  • In any one frame, we can access everything that is permanently in VRAM, plus as much data as we can put through the PCIe bus from the CPU to the GPU.
  • We only need to access data in the working set (what is on screen).
  • Some of VRAM is permanently used (e.g. memory for the screen itself).
  • We lose PCIe bandwidth to both drawing from main memory (terrain is in your system memory and must go over the PCIe bus per frame) and from bandwidth spent juggling textures.
So the actual maximum throughput will have a lot to do with whether the video driver makes good decision about what lives in VRAM and what does not.

But how does the video card know what should be in VRAM? The answer is that it has to guess. It looks at frames going by and tries to use heuristics (that is computer-science geek speak for "carefully formulated wild guesses") to decide what goes in VRAM and what does not. When the heuristics happen to make good decision, your video card kicks ass. And when it does not, your framerate tanks.

The only way to guarantee good framerate is to use so little VRAM that everything that needs to be in VRAM can be in VRAM, without depending on the video driver to make lucky guesses with its juggling.

And this helps us understand why strange things like livery reloads and backgrounding the sim can affect framerate (for better or worse). These operations seriously reshuffle VRAM - either by deleting textures and loading new ones, or by forcing everything out of VRAM so the video driver must try to repack video RAM all over again.

Unfortunately as a user this means that there's not much you can do about this as a user. The main things would be: reduce screen size or FSAA or texture resolution, use fewer add-ons, or get more VRAM. Those peak bursts of framerate you see, they're not going to be sustainable .

Friday, May 14, 2010

REXPlane

REX is here for X-Plane. That's really exciting to me because REX is a very successful add-on to FS X, and it's been understandably difficult for us to convince companies that have a functioning and profitable business model on FS X to jump over to X-Plane, which requires building new capabilities in-house, starting at the bottom of the learning curve, and developing new processes. I hope these guys have a ton of success. Heck, I'd buy the product just because REXPlane is such an awesome name!

What surprises me is that we haven't seen more ports of airport scenery. These packages can be partly converted mechanically using Jonathan Harris' FS2XPlane tools, and in the case of an airport, the potential for recycling is huge - that is, the largest part of the development process is modeling, creating textures, and connecting the two. Since X-Plane can use these art assets with minimal mods, it seems like a very viable type of port.

Wednesday, May 12, 2010

Water Reflections Use the CPU

A slightly surprising note about X-Plane's rendering settings: increased detail in the water reflections mostly increases load on the CPU! Here's what is going on.

X-Plane's water reflections are a two-step process:
  1. A (simplified) version of the world is drawn into a texture .
  2. That texture is used (and perturbed) to draw the surface of the water.
The water reflection detail setting affects a number of this: the size of the texture used to draw the world, the level of simplification of the world drawn into the texture, and I think some details about the shader used to draw the water surface itself.

Of these factors, the biggest impact on frame-rate is almost always simplification of the world as drawn into the water texture. At its most detailed ("complete" reflections) that rendering is almost identical to the main sim rendering, and thus reflective water should cut your framerate in half!

In particular, the big hit of drawing the world a second time is that it means drawing objects a second time. Thus the higher the reflection setting, the more objects are being drawn. In a scenario where objects are already the main drag on frame-rate, using complete water reflections will make this even worse.

There are two things to conclude from this:
  1. A bigger, faster CPU won't let you turn up the water reflection detail. Most decent GPUs handle the shader-side just fine. It's the objects that are the problem.
  2. You shouldn't turn up the water reflection all the way unless you're taking screenshots. It just doesn't add much visual improvement for the fps hit.*
* This is one of those cases where I ask myself: why did I let the setting go up this high?

Draping

I'm trying to post weird in-development screenshots as I get them because, well, sometimes they are entertaining, in a nerdy sort of way.

This is experimental test code for road draping.

The idea will be to let X-Plane "drape" the road over the base mesh, so that an overlay can contain customized roads that work with any base mesh.

The Wonderfully Filthy

From one of the podcasts I listen to I just discovered Edward Burtynsky. He takes these amazing photographs of industrial landscapes - really scary post-apocalyptic images of oil refineries, chopped up cargo ships, etc. Take a few minutes to look through some of the images.

One of the side effects of working on X-Plane scenery for the last few years is that it has made me look a lot more closely at the world. Once you try to recreate the world on a computer (and watch your digital creation fall way, way short) you realize how much intricacy and detail every-day phenomena have.

So when I saw Burtynsky's photos I immediately thought "he sees the complexity and beauty* and sadness of industrial landscapes the same way we do!"

* Beauty? I am not suggesting that the SOCAR oil fields are beautiful, a particularly good idea, or something I want more of. But I think that there can be a poetry in the image - perhaps a poetry of despair.

Tuesday, May 11, 2010

You Only Have to buy Apollo Once

With the latest updates to X-Plane for iPad, you can now purchase the Apollo lunar lander game inside X-Plane.

A quick note: if you lose your preferences (and thus X-Plane thinks you have not already bought Apollo) you will not be charged a second time if you click "buy it" again. When you go to repurchase Apollo, X-Plane will notice that you already have bought the upgrade and will simply re-enable the update.

Hopefully in our next patch we will make it clear in the UI that there is no double-purchase. The upgrade to Apollo is a "persistent" in-app purchase - once you buy it, you own it forever, as you would expect. (And because the iTunes store has a copy of your receipt, you don't lose it even if your iPad or computer's hard drive gets wiped out.)

Also, the latest update should fix crashes on iPads that were running for a while. X-Plane was very close to the memory limit so iPads that had been running for a while wouldn't have enough free RAM for the sim. The new patch is a little bit leaner to work around this problem.

Monday, May 10, 2010

Why Is My Airplane Slow?

Sometimes I get reports of a slow airplane, and I do a quick audit for performance problems. The trick to spotting performance problems is to divide and conquer: turn off various aspects of the airplane to see which aspect is really causing performance problems, then optimize that aspect.

Here are some of the specific tricks I do:
  • Change views; the panel will be drawn differently in the 2-d view, 3-d cockpit view, external view (when close or far - zoom out and the panel won't be updated) vs. 2-d no HUD.

    If the 2-d view is slow but forward-no-HUD is not, your panel is expensive. If the 3-d view is slow but 2-d is not, one of your panels may be more expensive than the other (copy them in Plane-Maker from one to the other to see) or it could be that the preparation of the cockpit texture is slow.

  • Remove 3-d objects from your plane to test the cost of OBJs. Turn down X-Plane's texture res or shrink your textures to see if texture memory is at issue. (Some airplane textures are not affected by the texture res settings, so you may have to manually shrink them.)

  • Be sure to play with X-Plane's rendering settings; the GPU-specific options don't always cost "the same". For example, per pixel lighting is more expensive when there is more translucency on screen. If your airplane has a lot of overlapping surfaces or translucency this otherwise manageable setting might become too slow.

  • If you use panel regions, try switching to regular ATTR_cockpit. Panel regions provide superior lighting effects but can take more CPU time when you have a lot of instruments.

The key is to divide the many possible causes of performance problems to isolate one thing that can be optimized.

Sunday, May 09, 2010

Zen and the Art of OBJ 2: Performance

In my previous post I tried to break an OBJ down into a few basic sections:
  • Global properties of the OBJ.
  • Raw Mesh Data
  • Commands, which in turn set per-batch state and then draw the batches.
The performance cost of an OBJ feature often has a lot to do with where in the OBJ the command shows up, e.g. is it global or per batch.

Global properties tend to affect OBJ performance on a one-time basis. For example, if you use cockpit regions, you pay a fairly large penalty for having the panel texture be set up even if you only apply that panel texture to a single texture. Sure enough, COCKPIT_REGION is in the global properties section of an OBJ.

Per-batch properties affect the OBJ in two ways:
  • Every command you see in the commands section is going to involve some CPU intervention. A very long commands section is more work for an OBJ.
  • Every time there are attributes between TRIS commands, it defines a new "batch" - that is, a separate instruction to the graphics card to draw a new and distinct setup. Think of this as shutting down the factory to reconfigure the assembly line.
Generally batch count is more important than total commands. In other words, in evaluating this:
TRIS 0 300
ATTR_light_level 0 1 some_dataref
ATTR_no_blend 0.5
TRIS 300 12
the fact that there are two attributes is less interesting than the fact that there are two batches (the two TRIS commands run with different state). Even if you got rid of the no-blend attribute, you'd still have two batches because of the light-level change.

The most powerful aspect of the OBJ format is bulk data handling - that is, you have to add a huge number of triangles before the number of triangles becomes a performance problem.

For this reason, you should never use an attribute to reduce geometry count. A few examples:
  • Don't use ATTR_no_cull to reduce triangle count - simply issue the indices of the triangle twice.
  • Don't use ATTR_flat_shade to reduce vertex count - simply use more vertices with correct per-vertex normals to simulate flat shading.
  • Prefer texturing to materials whenever possible.
Finally a note on weighting: for airplanes, where the total number of objects is low (a few dozen) global object properties often matter most. For example, on an airplane, choosing to use huge panel regions, or huge textures can make a big difference in performance. By comparison, batches aren't that expensive unless you do something really crazy.

By comparison, for scenery, batches matter more; X-Plane will share the global properties of objects across hundreds or thousands of objects, but each batch hurts framerate. So when making autogen-style scenery, batches are most important.

Saturday, May 08, 2010

Zen and the Art of OBJ 1: The Anatomy of an OBJ

A number of people are working on an update to Jonathan's Blender X-Plane export scripts; this post is aimed at shedding some light on some of the recent changes to the OBJ format. X-Plane 9 introduced a number of new OBJ features (manipulators, invisible geometry and camera collisions, dataref-driven control of emissive texturing, normal maps, and a number of new light billboard options). If you simply read the new OBJ commands in the order they were added to the format, it's just a soup of funny names. But there is some logic to how the OBJ format is extended.

The World's Simplest OBJ

Here is a very simple OBJ file, broken up by my annotations. First we have the header and global section:
A
800
OBJ

TEXTURE great_image.dds
POINT_COUNTS 24 0 0 36
the global section describes properties universal to the entire OBJ. For example, what textures will be used to draw the object?

We picked up a few new global properties in the version 9 run:
  • Normal maps are declared globally for the entire OBJ.
  • The metrics of any panel regions to be used are declared globally.
We may pick up new global attributes in the future; if we do, they will be properties that apply to the entire obj.

Next comes the data section:
VT 0.449997 0.300003 0.860001 1.000000 0.000000 0.000000 0.000000 0.000000
VT 0.449997 0.300003 0.000000 1.000000 0.000000 0.000000 0.000000 1.000000
VT 0.449997 -0.509995 0.860001 1.000000 0.000000 0.000000 1.000000 0.000000
...
IDX10 0 1 2 3 2 1 4 5 6 7
...
IDX 21
I have removed a lot of the data section, because there's not much to be said about it. The data section contains the raw data for the meshes in your OBJ, and it hasn't changed since the OBJ 8 format was introduced.

The third and final section is the most interesting one: the commands section.
ATTR_LOD 0 3000
ATTR_hard asphalt
TRIS 0 36
The commands section describes how the data is used in the form of serial instructions to X-Plane. Most changes to the OBJ format have come in the form of new commands. We can categorize our commands into a few buckets:
  • Drawing commands create "stuff". There aren't very many drawing commands, and new ones don't appear very frequently. TRIS and LINES are the main commands, but the smoke commands also fall into this category, as do the light billboard commands. The new light billboard command LIGHT_PARAM is the only new drawing command for version 9, and it probably warrants its own blog post.
  • Attribute commands change how stuff is drawn - they effectively define properties for drawing on all triangles that can be modified. We picked up a number of new attributes: manipulators (controlling how the mouse works), light level control, solid camera, draw disabling, deck style hard surfaces, and panel regions. (While you must declare the panel region locations globally, a panel region is enabled for a specific batch.)
  • ATTR_LOD is sort of an exception, because it defines the structure of the model (e.g. a model with LOD really contains several separate command lists, of which only one is used).
Most new extensions to the OBJ format come in the form of new attributes. Attributes generally apply to a specific mesh within your model, not to the entire model.

Note that attributes can be thought of as "per mesh" or "per batch" properties, because they affect only the batches of mesh (TRIS commands) between the attribute being turned on and turned back off again.

Where Will New Features Appear?

I try to post some of my crazier ideas regarding OBJ on the scenery system RFCs page. Looking at the extensions, we can see how these extensions will all be either global, drawing primitive, attribute, or OBJ structure extensions. (I am not promising that any of these RFCs will be implemented, just showing how the OBJ format grows.)
  • Additive LOD. This is a change to the structure of an OBJ, but it doesn't actually change the format, just the legal LOD values.
  • Explicit OBJ Height. This is a global property on the OBJ.
  • Global Texture Variants would be a global property on the OBJ's textures.
  • Global Object Attributes are new global properties - they move some per-batch attributes to be object-wide.
  • Draped Object Geometry would be per batch.
In summary, the vast majority of proposed extensions are new per batch or per object properties.

Next: what are the implications on performance of the various sections of an OBJ?

Friday, April 30, 2010

Baking and Overlays

In past posts I have tried to describe the implications of DSF base meshes, which are "fully baked". The basic idea is: the base mesh is fully formed ahead of time as a single unit. This is a trade-off:
  • The advantage is performance. The sim has no work to do except draw that base mesh as fast as possible.
  • The disadvantage is flexibility. The sim has no easy way to modify that base mesh.
By comparison, DSF overlays are not fully baked - you can add 8 overlays to an area and they will all run on top of each other. There is a real performance cost to this. Compare the performance of a huge number of draped orthophotos (via .pol files, an overlay technique) with a real orthophoto base mesh cut with MeshTool. You'll easily get 100 fps on the DSF base mesh, but you won't come close with the overlay.

If you want to compare X-Plane to a first person shooter, consider the "cost" of overlays as one of the reasons why FPS games appear to be higher performance than general purpose flight simulators like X-Plane and MSFS. In a FPS, each level is likely to be fully baked, with only one level loaded at a time. This is equivalent to X-Plane's DSF base mesh. The FPS game doesn't need to manage overlays that are put together at runtime in unpredictable combinations, and this lets the FPS engine optimize for performance.

(In fact, the FPS engine might be able to optimize a second way, if third party level packs are not available. Not only can a level be 'fully baked', but it can be fully baked specifically for that particular rendering engine. By comparison, a DSF base mesh will run with X-Plane 8 or 9 - clearly it isn't specifically optimized for just one version of X-Plane.)

If you look at the scenery system "overview" I wrote around the time of X-Plane 8's release (this overview is now pretty out of date; I really need to update it) you'll see this:
There are now two scenery formats - one for editing and one for distributing scenery. Both are new.
DSF stands for "distribution scenery file" - the idea is that DSF was meant to be a container for fully baked finished scenery, optimized for small size on DSF and fast loading, but not editing. Our internal tools use another file formatm ".xes" to contain imported global scenery data before it is baked. Originally I thought that we would provide an editor to .xes files, but that has not happened. With MeshTool, you provide input data in more common public formats like SRTM HGT or GeoTiff, and .shp (shapefiles). You can think of .shp and .tif as the editing formats for MeshTool and base DSFs.

So how do we make it easy for users to edit scenery? I believe OpenStreetMap is the answer. The common request we get from users is for a way to edit the vector source data for global scenery (or sometimes, the request is to edit the features created by vector data). In other words: how does a user edit the coastlines, water bodies, and roads? With OpenStreetMap, OSM itself becomes the "editing" format for X-Plane scenery with DSF as the final result of baking.

Wednesday, April 28, 2010

What's Next for Scenery Tools

Last week I finally was able to post some scenery tools builds. This begins the beta period for WED 1.1 (WED with DSF overlay editing) and a release candidate period for the AC3D plugin and MeshTool. I expect the WED beta to take a while; as long as the program is reasonably functional in beta form (e.g. not losing data) there isn't a huge rush to package it up relative to other priorities.

WED will continue to evolve as the primary visual editor for scenery. This will include editing air traffic control data for the new ATC system and editing roads once X-Plane is ready for road overlays. (See this post for why road overlays aren't quite useful in the sim yet.)

I'm not sure what the next features for MeshTool will be - this may depend on user feedback. One thing is clear: MeshTool is not easy to use. Building a base mesh is a complex and low level process, with lots of possible pitfalls. So whatever features MeshTool develops, usability has to be a goal.

There is one more scenery tool I would like to create: a remote render farm. Right now you can make custom base meshes with MeshTool. In the future it will be possible to make edits to the source data used or global scenery (by editing OpenStreetMap itself). You can also edit the apt.dat file and submit the results to Robin.

But once you edit OSM, how do you get a new scenery pack? Using MeshTool is complex, and MeshTool is really aimed at the orthophoto crowd. Waiting 2 or 3 years for the next X-Plane release isn't a good solution.

My idea is to set up a computer to do "remote global scenery" requests. I would set the computer up to periodically pull down new data updates and recut tiles as requested, then post them publicly. This would allow users to edit the source data and then put in a tile request to get the tiles back, without ever having to know how to make scenery. The tiles would reflect all changes from all users.

Such a service wouldn't be of interest to the most advanced authors who want to create a truly original scenery pack, but for authors who want to fix specific problems, this process would be much simpler. I hear the question "how do I fix the lake behind my house" all the time; a remote render farm could be part of the answer.

Tuesday, April 20, 2010

New Tools This Week

A bug-fix release of AC3D was posted over the weekend, and now it is gone. Andy pointed out to me that I had posted a build for Windows, but not Mac and Linux, a build error on my part.

I should be able to get all of the new tools (the AC3D patch, WED 1.1 beta 1, and a MeshTool release candidate) posted this week.

Friday, April 16, 2010

System Requirements for Scenery Tools

It looks like the next generation of scenery tools (MeshTool 2.0, WED 1.1, and the latest distribution of "the tools") may have higher system requirements than their predecessors for Mac users. Those requirements would be: an Intel CPU and OS X 10.5.0 or higher.

The problem at the heart of all of these changes is that the tools use CGAL (a geometry library) and the compilers Apple distributes that are compatible with 10.4 and PPC don't work with CGAL. So I quite literally cannot compile the latest tools because of the features they offer.

I don't know how much this affects actual authors, and I don't know if it is possible (given an infinite amount of self-torture) to work around some of the compiler issues. At this point my plan is:
  • Distribute the next-gen tools in binary form for 10.5 and x86.
  • Leave links to the old tools for users who need binary PPC tools.
  • Continue to make all source code available via the GIT repo.
If someone finds a way to compile these tools for older targets using the source code, I am perfectly happy to provide distribution of those binary tools or incorporate the fixes if they are manageable.

I'm hoping to have some tools posted "real soon"...

Friday, April 09, 2010

What an Evil Disaster

After about a week of on and off hacking, I have finally knocked down one of the major stumbling blocks to getting WED 1.1 up to beta quality: exporting UV-mapped (texture mapped) bezier polygons that cross DSF borders. It works! Well, sort of.

If you have tried to program polygon cutting algorithms, you can appreciate the difficulty of an algorithm that:
  • Clips polygons robustly (including holes and other weird topology) and
  • Maintains a UV mapping while doing this and
  • Works with bezier curves and not just line segments.
WED now does all three! This was the ugliest and hardest part of the DSF exporter, and a big missing piece from going beta.

Of course, there is one problem: X-Plane can't read the bezier curves.

The problem is a simple defect in how X-Plane manages DSFs.
  • A valid bezier polygon, fully inside the DSF tile, may have control handles that go outside the tile.
  • X-Plane can't handle any DSF coordinates outside the tile.
Doh!

I am not sure what I will do about this, but in the short term, I fear X-Plane will remain limited. Probably the best short term option is to have WED at least flag such problematic bezier polygons; it is possible to approximate them or edit them to make the export work.

There is still a little bit more exporter code to write, including the line segment exporter (which is separate from the polygon exporter), but with luck the whole DSF export path should be cleaned up in the next few days.

Two-Sided Drawing and Up-Normals

Meshes in X-Plane, whether modeled in an OBJ, or generated as the results of other "3-d clutter" (road .net files, .for forest files, etc.) can be either one or two sided. So first: two-sided geometry is bad in most cases.

In order to understand why two sided geometry is bad, we must consider the alternative. The alternative to two-sided geometry is to simply create each triangle twice, with one facing in each direction. We can do this in an OBJ without making new vertices - because vertices are referenced by indices, we only need more indices, and indices are cheap.

Thus we have an alternative to two sided geometry, namely "doubled" one-sided geometry.

The first problem with two sided geometry is performance: for a small number of triangles, it is must faster to simply emit additional indices than to change the drawing mode to two-sided drawing on the CPU.

Thus in an object it is virtually never a good idea to use two-sided geometry. That ATTRibute will always be worse.

What about for the other clutter? Forests are currently always two-sided, but that's okay; X-Plane enables two sided drawing just once, then draws a huge number of trees. Same with roads. For facades, there is a cost to using two sided geometry, so only use it for facades that must be two sided, like fences; do not use it for buildings.

Now the second problem with two sided geometry is lighting: X-Plane does not calculate lighting values separately for the two sides of the two-sided geometry. So if you have directionally lit models with two sided geometry, the lighting will look wrong. This is the second reason to use doubled geometry instead.

Things Are Starting To Look Up

There is a work-around to this problem of incorrect lighting on two-sided geometry: "up normals". With up normals, the normal vector for the triangle (which is used to determine how light "bounces" off the triangle) is set to face straight up. The result is a triangle with brightest lighting at high noon, regardless of which way the triangle actually faces.

The good: the triangle looks the same on both sides and has sort of a "flat" lighting - it doesn't look wrong when the sun is setting. The bad: the triangle has "flat" lighting - it looks non-3d.

We use up normals for forests because the forests are made of two-quad trees...the trees look less fake if directional lighting hints don't make the two quads as obvious. You can simulate this in ac3d using the "make up normal" command for vegetation quads you put in your own models.

For roads, the geometry is two-sided, so we use up normals to avoid having the back of a road element look funny. Some day we may do something more sophisticated.

Fixing Facade Lighting

Facade lighting behavior will be changed in the next 950 release candidate. Before 950, facades would receive up normals, always. Starting in 950, facades will get correct normals if they are one sided and up normals if they are two-sided. This avoids artifacts with two-sided facades, but will make one-sided closed buildings look much better.

Thursday, April 08, 2010

Ray Tracing is the Technology of the Future...

...and it always will be!

Seriously, first, let's be clear: my opinions do not matter! X-Plane is a small program in a large market (game/graphics hardware) and as I've said before, flight simulators are not the early adopters of new tech. So (and this is a huge relief to me) I can do my job without correctly predicting the future of computer graphics.

Keep that in mind as I mouth off regarding ray tracing - I'm just some guy throwing tomatoes from the balcony. X-Plane doesn't have skin in the game, and if I prove to be totally wrong, we'll write a ray tracer when the tech scales to be flight simulator ready, and you can point to this post and have a good laugh.

With that in mind, I don't see ray tracing as being particularly interesting for games. I could make arguments that rasterization* is significantly more effecient, and will keep moving the bar each time ray tracing catches up. I could argue that "tricks" like environment mapping, shadow mapping, deferred rendering, and SSAO have continued to move effects into the rasterization space that we would have thought to be ray-tracing-only. (Heck, ray tracing doesn't even do ambient occlusion particularly well unless you are willing to burn truly insane amounts of computing power.) I could argue that there is a networking effect: GPU vendors make rasterization faster because games use it, and games use it because the GPU makers have made it fast. That's a hard cycle to break with a totally different technology.

I don't really have the stature in the world of computer graphics to say such things. Fortunately John Carmack does. Read what he has to say. I think he's spot on in pointing out that rasterization has fundamental efficiencies over ray tracing, and ray tracing doesn't offer enough real usefulness to overcome the efficiency gap and the established media pipe-line.

The interview is from 2008; a few months ago Intel announced that first-generation Larrabee hardware wouldn't be video cards at all. For all effective purposes from a game/flight simulator perspective, they basically never shipped. So as you read Carmack's contents re: Intel, you can have a good chuckle that Intels claims have proven hollow due to the lack of actual hardware to run on.

I will be happy to be proven wrong by ray tracing, or any other awesome new technology. But I am by disposition skeptical until I see it running "for real", e.g. in a real game that competes with modern games written via rasterization. Recoding old games or showing tech demos doesn't convince me, because you can recode an old game even if your throughput is 1/20th of rasterization, and you can hide a lot of sins in a tech demo.

Heck, while I'm putting my foot in my mouth, here's another one: unlimited detail. Any time someone announces the death of the triangle, I become skeptical. And their claim of processing "unlimited point cloud data in real time" strikes me as an over-simplification. Perhaps they can create a smooth level of detail experience with excellent paging characteristics (which is great!) but the detail isn't unlimited. The data is limited by your input data source, your production system, the limits of your hardware, etc. Those are the same limits that a mesh LOD system has now. In other words, what they are doing may be significantly more efficient, but they haven't made the impossible possible.

That is my general complaint with most of the "anti-rasterization" claims - they assume that mesh/rasterization systems are coded by stupid people - and yet most of the interesting algorithms for rasterization, like shadow mapping and SSAO, are quite clever. Consider these images: saying that rasterization doesn't produce nice images while showing Half Life 2 (2004, for the X-Box 360) is like saying that cars are not fuel efficient because a 1963 Cadillac got 8 mpg. The infinite detail sample images show a lot of repeated geometry, something that renderers today already do very well, if that's what was desirable (which it isn't).

Finally, is in favor of sparse voxel octrees (SVOs). SVOs strike me as the most probable of the various non-mesh-rasterization ideas floating around, and an idea that might be useful for flight simulators in some cases. To me what makes SVOs practical (and in defense of the unlimited detail folks, their algorithm potentially does this too) is that it can be mix-and-matched with existing rasterization technology, so that you only pay for the new tech where it does you some good.

* Rasterization is the process of drawing on the screen by filling in the pixels covered by a triangle with some shading.

Sunday, April 04, 2010

Ray Tracing: Shoot Before You Fly

I'll take a break from iPad drivel for a few posts; at least one or two of you don't already own one. (Seriously, it's simply easier to blog about X-Plane for iPad because it is already released; a lot of cool things for the desktop are still in develpment.)

In response to my comments on water reflections in X-Plane 950, some users brought up ray tracing.

My immediate thought is: I will start to think more seriously about ray tracing once it becomes the main technology behind first person shooters (FPS).

Improvements in rendering technology come to FPS before flight simulators (and this is true for the combat sims and MSFS series too, not just us). Global shadows, deferred rendering, screens-space ambient occlusion...the cool new tricks get tried out on FPS; by the time they make it into a flight simulator the technology has moved from "clever idea" to "standard issue."

Consider that X-Plane now finally has per-pixel lighting. Why didn't we have it when the FPS first did? Well, one reason is that the FPS were cheating. If you look at the papers suggesting how to program per-pixel lighting, at the time there were all sorts of clever techniques involving baking specular reflections into cube maps and other such work-arounds to improve performance. These were necessary because titles at the time were doing per-pixel lighting on hardware that could barely handle it. X-Plane's approach (as well as other modern games) is to simply program per pixel lighting and trust that your GeForce 8800 or Radeon 4870 has plenty of shader power.

I believe that the reason for the gap between FPS and flight simulators come from two sources:
  1. Viewpoint. You can put the camera quite literally anywhere in a flight simulator, and thus the world needs to look good from virtually any position. By comparison, if your game involves a six foot player walking on the ground (and sometimes jumping 10 feet in the air) you know a lot about what the user will never see, and you can pull a lot of tricks to reduce the performance cost of your world based on this knowledge. (This kind of optimization applies to racing games too.

    To give one simple example of the kind of optimization a shooter can make that a flight simulator cannot, consider "portal culling". A portal-culled world is one where the visibility of distinct regions have been precomputed. A trivial example is a house. Each room is only visible through the doors of the other rooms.* Thus when you are walking through a room, virtually no other room is being drawn at all. The entire world is only 20 by 20 meters. Thus the developers know that they have the entire hardware "budget" of computing power to dedicate to that one room and can load it up with effects, even if they are still expensive.

    (A further advantage of portal culling is a balance of effects. Because rooms are not drawn together in arbitrary combinations, the developers may find ways to cheat on the lighting or shadowing effects, and they know nothing will "clutter" the world and ruin the cheats.)**

  2. Often the FPS will have pre-built content, rather than user-configurable content. Schemes like portal culling (above) only work when you know everything about the world ahead of time and can calculate what is visible where. The same goes for many careful cheats on visual effects.

    But a flight simulator is more like a platform: users add content from lots of different sources, and the flight simulator rendering engine has to be able to render an effect correctly no matter what the input. This means the scope of cheating is a lot smaller.

    Consider for example water reflections. In a title with pre-made content, the artists can go into the world in advance and mark items as "reflects", "doesn't reflect", reducing the amount of drawing necessary for water reflections. The artist simply has to look around the world and say "ah - this mountain is no where near a lake - no one will notice it."

    X-Plane can't make this optimization. We have no idea where there will be water, or airports, or you might be flying, or where there might be another multiplayer plane. We know nothing. Everything is subject to change with custom scenery. So we can't cheat - we have to do a lot of work for reflections, some of which might be wasted. (But it would be too expensive in CPU power to figure out what is wasted while flying.)

Putting it all together, my commentary on ray-tracing is this: the FPS will be able to integrate small amounts of ray tracing first, because they will be in a position to deploy it tactically, using it only where it is really necessary, in hybrid ray-trace + rasterized engines. They'll be able to exclude big parts of the scene from the ray tracing pass, improving performance. They'll be able to "dumb down" the quality of the ray trace in ways that you can't see, again improving performance. The result of all of this will be some ray tracing in FPS when the hardware is just barely ready.

For a flight simulator, it will take longer, because we'll need hardware that can do a lot more ray tracing work. We won't know as much about our world, which comes from third party content, so we won't be able to eliminate visually unimportant ray traces. Like deferred rendering, shadow mapping, SSAO and a number of other effects, flight simulators will need more computing power to apply the effects to a world that can be modified by users.

(Is ray tracing even useful, compared to rasterization? I have no prediction. Personally I am not excited by it, but fortunately I don't have to make a good guess as to whether it is the future of flight simulation. The FPS will be able to, by effective cheating, apply ray tracing way before us, and give us a sneak peak into what might be possible.)

* There never are very many windows in those first person shooters, are there?
** To be clear: there is nothing negative about the term "cheat" in computer graphics. A way to cheat on the cost of an algorithm means the developers are very good at their jobs! "Cheating" on the cost of algorithm means more efficient rendering. If the term cheating seems negative, substitute "lossy optimization".

Saturday, April 03, 2010

Yes, the iPad is Magical

So all kidding aside, my iPad arrived today, and it is a pretty cool device. My normal attitude toward new gadgets is "great, more video driver bugs to fix", but the iPad is exciting.

I'll blog some other time about why I think the form factor is important and there is a spot for something bigger than a smart phone and smaller than a laptop.

For now I just want to point out that it flies X-Plane; X-Plane for iPad a bunch of new features, including 2-d panels while you fly, 3-d airports with full taxiway layouts, a completely rebuilt user interface, improved sky and water effects, and even some auto-gen buildings.

Also first impressions of the device itself: it's really responsive. I have tried to surf the web with my iPod touch (which is based on first-gen iPhone technology) and it's a tough experience - between the small screen, slow CPU and limited RAM. The iPad surfs the web like a desktop. A very light weight, portable desktop.

Friday, April 02, 2010

Need...More...Stripes!

Some of the coolest in-development screen-shots are the bugs. A few weeks ago while working on cloud shaders I managed to create a hybrid of an acid trip and a light-speed trip on Star Wars. Sadly the code has since been modified a thousand times and the effect is gone.

So I'm going to start posting bizarre images as they crop up. Usually they don't reveal too much about unreleased features.

This came out of some experiments with hills, mountains and UV mapping.

(Clearly what this code needs is a leopard-print texture.)

Mirrored Normal Maps

Normal maps in X-Plane 940 have a funny property: if you flip the normal map horizontally or vertically, the bumps change direction. Things that "stuck out" now "stick in" and vice versa. (If you flip the normal map horizontally and vertically, the two cancel out and the normal map is not reversed.)

You can understand this by thinking of your normal map as a physical piece of metal with bumps punched in it. Flipping it horizontally really means rotating it horizontally to see the other side - now you see the back side of the bumps. Same with the vertical flip. Flip both and you have flipped it twice and it's front-side forward again.

In a move that is either just in the nick of time or dangerously reckless, I have tweaked the normal map shader for 950 RC1 (coming out "real soon") to detect and "fix" a flipped normal. In 950 rc1 the bumps in your normal map will always point in the same direction as the normal of your mesh, even if your UV map is flipped horizontally or vertically.

What does this mean to you, the modeler? It means that you can now mirror your normal map from the left to the right side of the airplane and the normal map will still have the bumps "sticking out" like you want.

I crammed this into 950 RC 1 because it looks like it's a useful change that will restore flexibility to authors making highly detailed airplanes. Mirroring a symmetric airplane (which results in a horizontally mirrored normal map) is a pretty common thing to do, and if the text is applied as decals, this can be a big win in texture space savings.

I figured best to get the tweak in here now so people could take advantage of it. Plus, what's an RC1 without an RC2?

Thursday, April 01, 2010

The Real Deal

It goes without saying that this post was, um, inaccurate.

This, however, is real.

X-Plane 10 Will Be For the iPad Only

I wasn't planning on discussing this until April 3rd, but some sites have already picked it up, so I might as well explain the why behind it. Simply put: X-Plane 10 will be for the iPad only. I figure that by letting you know now, you can make an informed decision about whether you want to buy an iPad. (Hint: you do!)

This wasn't an easy decision to make, but here were some of the deciding factors:
  • I spend a lot of my time debugging video drivers, and it makes me very grumpy. You guys seem to think that you can plug anything with copper and wires into your PCIe socket and call it a "video card". By restricting version 10 to iPad only, we cut down development time by targeting only one GPU.

  • We love everything Steve Jobs does and kiss the ground he walks on. Steve says the pad is magical.

  • The biggest single feature request we get for X-Plane is "can I use X-Plane while operating a motor vehicle." X-Plane 9's requirement of 1 GB of RAM, a modern CPU, etc. means that most X-Plane 9 machines are not particularly portable. By making X-Plane 10 iPad only, we can provide X-Plane in a portable format that's more compatible with flying a flight simulator while driving.

One fringe benefit to cockpit builders: because the iPad is so light and thin, creation of a realistic home cockpit with X-Plane 10 will be easier than ever. Simply take some super-glue, wipe it on the back of your iPad, and simply "stick" the iPad to whatever part of the cockpit you want. It's so easy a four-year-old could do it!

Wednesday, March 31, 2010

OS X 10.6.3 Performance

OS X 10.6.3 is out. Besides adding a bunch of OpenGL extensions*, it looks like vertex performance is improved on nVidia hardware. My quick tests compare 10.5.8 to 10.6.3 (since I no longer have a 10.6.2 partition) and show a 15-30% improvement. If you have 10.5 and an 8800 you may want to consider updating your OS.

I also discovered that --fps_test=3 produces unreliable results because...wait for it...the deer and birds are randomized. If they show up during the fps test, you get hit with a performance penalty. I am working to correct this and may have to recut the time demo to work around this behavior.

If you are trying to time the sim via --fps_test=3, I suggest running the test multiple times - you should see "fast" runs and "slow" runs depending on our feathered and four-legged friends.

Phoronix reported a performance penalty with the new update; I do not know the cause of this or whether the fps_test=3 bug could be causing it. But their test setup is very different than mine - a GeForce 9400 on a big screen, which really tests shading power. My setup (an 8800 on a small screen) tests vertex throughput, since that has been my main concern with NV drivers.

My suggestion is to use --fps_test=2 if you want to differential 10.6.2 vs. 10.6.3. I'll try to run some additional bench-marks soon!

EDIT: Follow-up. I set the X-Plane 945 time demo to 2560 x 1024, 16x FSAA, and all shaders on (e.g. let's use some fill rate). I put the Cirrus jet on runway 8 at LOWI, then set paused forward full screen no HUD. In this configuration, I see these results:
Objects  10.5.8   10.6.3
none 85 fps 100 fps
a lot 46 fps 61 fps
tons 37 fps 42 fps
Note that in the "no objects" case the sim is fill-rate bound - in the other two it is vertex bound. So it looks to me like 10.6.3 is faster than 10.5.8 for both CPU use/object throughput and perhaps fill rate (or at least, fill-rate heavy cases don't appear to be worse).

* These extensions represent Apple and the graphics card company creating software interface to fully unlock the graphics card's abilities.

Friday, March 19, 2010

Weird Water

If you look at funky pictures of X-Plane on line, a fair number of them will show incorrect water reflections. I am working on some bug fixes for the reflection code for 950. Bug fixes might not even be the right term. To understand the incorrect reflections, you have to understand what the water code can and cannot do.

The water reflection code renders a reflection based on a flat plane. This limitation comes from the mathematics of the algorithm - a compromise to have water reflections that run at "real-time" speeds. (Real-time is graphics nerd speak for 20-30 fps and not 1-2 fps.)

As it turns out, the Earth is not flat. So we can pick up a number of reflection "bugs", due to the limits of the approximation we are using:
  • Over very large distances, the flat plane is a bad approximation of a water surface. The flat plane simply can't be "right" everywhere for any large scene. This isn't a bug, it's a ceiling on our maximum quality - a design constraint.
  • If we have two water surfaces at different elevations (e.g. a river with a dam) we can't have our reflection plane match both. So some scenes may have wrong reflections with multi-level water. This too is a design constraint.
  • If our reflection plane is at the wrong height or the wrong slope, it is going to produce really weird results. The reflection plane being in the wrong place despite a small scene with one level of water - that'd be a bug.
  • There is an art to positioning the plane - if we have a large scene (so the round earth means there is no one perfect plane) some locations of the plane will look better than others.
Now one fall-out of all of this is that things are going to look better if water is really flat, which is not always the case (both for some parts of the global scenery with production errors and some third party scenery). Where the water is sloped or contains bumps, we hit the multi-level case where we should not and we face reflection plane placement problems.

Finally, if the scenery mesh contains slanted water, we're really going to be hosed - almost by definition if X-Plane uses a sane water reflection plane, it won't be sloped, and thus this sloped water is going to be unaligned with the reflection plane and produce something that looks really funky.

So my work on 950 is aimed at having X-Plane be less easily fooled by complex and incorrect scenes less often. (Note that X-Plane can't tell the difference between Norway, where we really have water at multiple elevations, and bad input data to MeshTool.)

Even with these fixes, sloped water is still going to look pretty strange (because it is strange). And even with these fixes, multi-level water will still have its reflections approximated at best. But hopefully the visuals from the sim will be less jittery while flying over tricky DSFs.

I'm hoping we'll have a beta 2 in the next week or two.

Monday, March 15, 2010

Musings on CSLs

Now that Wade has XSquawkBox 1.0.3 out, I've been thinking about CSLs - that is, the collections of simplified airplanes that XSquawkBox uses to draw the other users. The CSL system was invented back in the days of X-Plane 6, and it's getting a bit long in the tooth. You can't use OBJ8 files, and it doesn't understand a lot of the modern rendering tricks that authors use with the standard tool chain.

Plane-Maker has advanced quite a bit since then too - to make the original CSL, I had to create a special one-off hacked build of Plane-Maker to export aircraft as OBJs. This capability is now built into Plane-Maker, works a lot better, and even supports animation.

X-Plane now exports a native OBJ drawing interface to plugins. Besides giving plugins access to the fully optimized native OBJ draw code, this also means that plugins can draw objects with per-pixel lighting.

One more piece of the puzzle: in France Austin announced that we were working on a new ATC engine. One goal of this new engine is to provide ATC to the AI planes, as well as your plane, so that the other aircraft interact seamlessly in one simulated environment. (In X-Plane 9, ATC only directs you, and the AI are rogue planes that try not to buzz you when you're on the runway.)

This makes me wonder: should there be a next-gen CSL format that is shared between X-Plane and third parties, based on X-Plane doing the rendering work?

Sunday, March 14, 2010

Best. Beta. Ever.

Wade just cut the final build for XSquawkBox 1.0.3, and of all of the betas I have been involved in, this was the best one ever. I say this because Wade ran the entire beta and fixed pretty much all of the bugs without me! That is to say, Wade made 1.0.3 happen. So huge thanks to Wade for making a new and much needed XSquawkBox version possible.

Here are some of the features:
  • Automatic download and update of the server list.
  • Dual VoIP radios.
  • Proxy client support for users with multiple copies of X-Plane for multiple views.
  • A ton of work on the UI - a lot of little things add up to make a much more mature, usable client.
Now if only I had time to fly on VATSIM...

New Toys

This isn't supposed to be a coding blog, but users do ask about DirectX vs. OpenGL, or sometimes start fights in the forums about which is better (and yes, my dad can beat up your dad!). In past posts I have tried to explain the relationship between OpenGL and DirectX and the effect of OpenGL versions on X-Plane.

At the Game Developers Conference 2010 OpenGL 4.0 was announced, and it looks to me like the released the OpenGL 3.3 specs at almost exactly the same time. So...is there anything interesting here?

A Quick Response

In understanding OpenGL 4.0, let's keep in mind how OpenGL works: OpenGL gains new capabilities by extensions. This is like a new item appearing on a menu at your favorite restaurant. Today we have two new specials: pickles in cream sauce, and fried potatoes. Fortunately, you don't have to order everything on the menu.

So what is OpenGL 4.0? It's a collection of extensions: if an implementation has all of them it can call itself 4.0. An application might not care. If we only want 2 of the 4 extensions, we're just going to look for those 2 extensions, not sweat what "version number" we have.

Now go back to OpenGL 3.0, and DirectX 10. When DX10 and the GeForce 8800 came out, nVidia published a series of OpenGL extensions that allowed OpenGL applications to use "cool DirectX 10 tricks". The problem was: the extensions were all NVidia specific tricks. After a fairly long time, OpenGL's architectural review board (ARB) picked up the specs, and eventually most of them made it into OpenGL 3.0 and 3.1. The process was very slow and very drawn out, with some of these "cool DirectX 10 tricks" only making it into "official" OpenGL now.

If there were OpenGL extensions for DirectX 10, who cares that the ARB was so slow to adopt these standards proposed by NVidia? Well, I do. If NVidia proposes an extension and then ATI proposes a different extension and the ARB doesn't come up with a unified official extension, then application like X-Plane have to have different code for different video cards. Our work-load doubles, and we can only put in half as many new cool features. Applications like X-Plane depend on unity among the vendors, via the ARB making "official" extensions.

So the most interesting thing about OpenGL 4.0 is how quickly they* made official ARB extensions for OpenGL that match DirectX 11's capabilities. (NVidia hasn't even managed to ship a DirectX 11 card yet, ATI's HD5000 series has only been out for a few months, and OpenGL already has a spec.) OpenGL 4.0 exposes pretty much everything that is interesting in DirectX 11. By having official ARB extensions, developers like Laminar Research now know how we will take advantage of these new cards as we plan new features.

Things I Like

So are any of the new OpenGL 3.3 and 4.0 capabilities interesting? Well, there are three I like:
  1. Dual-source blending. It is way beyond this blog to explain what this is or why anyone would care, and it won't show up as a new OBJ ATTRibute or anything. But this extension does make it possible to optimize some bottlenecks in the internal rendering engine.

  2. Instancing. Instancing is the ability to draw a mesh more than one time (with slight variants in each copy) with only one instruction to the graphics card. Since many games (like X-Plane) are limited in their ability to use the CPU to talk to the graphics card (we are "CPU bound" when rendering) the ability to ask for more work with fewer requests is a huge win.

    There are a number of different ways to program "instancing" with OpenGL, but this particular extension is the one we prefer. It is not available on NVidia cards right now. So it's nice to see it make it into the core spec - this is a signal that this particular draw path is considered important and will get attention.

  3. The biggest feature in OpenGL 4.0 (and DirectX 11) is tessellation. Tessellation is the ability for the graphics card to turn a crude mesh with a few triangles into a detailed mesh with lots of triangles. You can see ATI demoing this capability here.

There are a lot of other extensions that make up OpenGL 3.3 and 4.0 but those are the big three for us.

* who is "they " for OpenGL? Well, it's the architectural review board (ARB) and the Khronos group, but in practice these groups are made up of employees from NVidia, ATI, Apple, Intel, and other companies, so it's really a collective of people involved in OpenGL use. There's a lot of input from hardware vendors, but if you read the OpenGL extensions, you'll sometimes see game development studios get involved; Transgaming and Blizzard show up every now and then.

Wednesday, March 10, 2010

I Feel Manipulated

Tom has a new video on youtube of his just finished Falco. The video shows what screen-shots cannot: that the mouse interactions on the plane are really well crafted.

If you're just discovering X-Plane (or just discovering that X-Plane's 3-d cockpits can be very interactive), here's X-Plane's "raw" capabilities for manipulation:
  • The simplest manipulations are based on mapping the mouse from the 3-d cockpit back to the 2-d panel. This can only be done when the 3-d cockpit is textured using a piece of 2-d panel. This is the oldest way to make a clickable cockpit in X-Plane, dating back to the original X-Plane 3-d cockpits. The advantage of this method is that it's very easy to set up; the disadvantage is that the mouse click gestures tend to be "flat" in their operation.
  • As Tom's plane demonstrates, you can manipulate just about any dataref or command via a drag along a specific axis. Axes are subject to animation, so there's a lot of potential for "grabbing" things with this interface.
  • X-Plane also supports direct "click" manipulation - this can be handy for buttons where you don't want to require the user to move the mouse around. There are several types of click manipulation.
Click and drag manipulations can be tied into the plugin system - your plugin sees a manipulation as a change to a plugin-created dataref. This makes it possible to create almost any imaginable mouse effect. If you don't want to write a plugin, you can still write up the manipulators to any of X-Plane's datarefs (there are thousands) or commands (we're getting up toward the 1000 mark on these too).

To create manipulators on your cockpit, you can use the latest plugin for AC3D. A manipulator is a property on a mesh within your object - each mesh can have its own manipulation with its own properties.

X-Plane does not have an IK solver. Rather, movement of "stuff" in your cockpit is indirect.
  1. Your manipulator changes a dataref as the user drags along an axis.
  2. The dataref change shows as an animation on your mesh.
Fortunately, ac3d has a "Guess" button for the axis manipulators. If you set a mesh to be manipulated by dragging along an axis, the guess button will examine your animations and suggest an axis that will create the most "natural" looking animation for the manipulation. For example, if you have a throttle handle that rotates, the guess button will provide a drag axis perpendicular to the throttle (to push the levers); if you have a throttle lever that pushes, the guess button will make a drag axis that runs along the lever.

Monday, March 08, 2010

Conformance Test

I've been working on a conformance test for X-Plane. The idea is simple, and not at all mine: X-Plane 945 can output a series of test images that are the same on each run. The images cover a variety of rendering conditions. If a video driver is broken, the images will be corrupted.

You can learn more about how this works here: I am working on the 945 timedemo tarball now.

The main driver for this is to help NVidia, ATI, and Apple to integrate X-Plane into their dedicated testing. With X-Plane as part of their test systems, they can catch driver bugs the easy way - the day after the code is changed, rather than months later after a series of angry web posts. X-plane 945 includes a number of new features as part of its framerate test to help with this process.

My hope is that this will benefit users (who will see less bugs) and the driver writers (who can get feedback on code changes in a uniform and reproducible manner). Here are the eight images in the sample conformance test I wrote, based on the LOWI custom airport scenery.







Saturday, March 06, 2010

Plugin-Drawn Objects Do Not Exist

Version 2.0 of the plugin API (available in all versions of X-Plane 9) introduces three new routines that, for the first time, allow plugins to work with OBJs.

XPLMLoadObject and XPLMUnloadObject load an OBJ file (a model mesh) into memory, and then purge it when done. That part most users understand. The routine that causes confusion is XPLMDrawObjects. When you draw an OBJ with XPLMDrawObjects, you are not creating anything long-lasting or persistent in the world. Your object will be visible for one frame on screen, and then will disappear unless your draw it again. Your object is not part of any of the physics calculations.

To put this in perspective: when you make a new window using the Widgets API, the window is persistent - it exists until (1) you delete it or (2) your plugin is unloaded for some reason. You don't have to do anything per frame to "maintain" the window - you make it and it exists.

Objects are not like that. You cannot make an object "exist" in the X-Plane world - you can only draw it once per frame using the drawing callbacks. Essentially the draw-objects API is a lower level API.

Building a Layered System

Plugins operate in a layered environment, with lower level code on the bottom and your plugin at the top. The layer stack might only be 2 layers deep (XPLM on the bottom, plugin on top), or there might be several layers. Consider XSquawkBox:
  • The UI is drawn using the XPWidgets API. The XPWidgets API gets its drawing from the XPUIGraphics API, and the XPUIGraphics API changes OpenGL state using the XPLMGraphics API. So we've built up a layered system: basic OpenGL supports drawing, drawing supports user interface, user interface supports the plugin.
  • Similarly, multiplayer is done using a library that isn't part of the basic plugin system (but is open source): libXplaneMP. So here the XPLM supports drawing airplanes, libXplaneMP uses that to create a full multiplayer API, then XSquawkBox uses it.
The alternative to a layered system would be a "monolithic" one. Under a monolithic system, the only API for airplanes would be libXplaneMP, and the only way to create user interface would be widgets. Sandy and I usually prefer the layered approach because it provides a lot more flexibility. If you like widgets, great, use them. If not, no problem - roll your own on top of XPLMDisplay.

The Plugin System Is a Foundation

When Sandy and I cannot provide all of the layers, we have a strong bias toward providing the bottom layer, for an obvious reason: if the bottom layer isn't in the plugin system, it may be impossible for anyone else to create it. So typically if we have a choice between a high level vs. low level API, we'll put the low level API in first.

This is precisely what is happening with object drawing - we have the low level API ("draw an object") but not the high level one ("create an object in the scenery system"). Since we have provided the lowest level, it is possible to code persistent objects in your plugin by layering on top of our API. By comparison, had we only provided "create an object" it would be pretty close to impossible to draw an object for one frame - if you didn't want a scenery object, the API would be inflexible and useless.

On the Road a Lot

I've been on the road a lot for work, so my apologies to everyone whose email I am sitting on. Most of my time these days is being spent on new next-gen tech. But there are a few things I'm hoping to get done in the short term:
  1. Cut a new time-demo test. This might seem like a low priority item, but it's not. Apple, ATI and NVidia all run continuous automatic tests of their video drivers, with many applications and games. They have rooms full of computers that continuously run through 3 minute sections of Quake and Call of Duty, etc. If they introduce a driver bug while doing new development, these machines catch the problem immediately.

    The new time demo (based on 945) will have a number of features to make X-Plane a more useful test case. If we can make X-Plane into a test case, then they can catch bugs early, and that means you don't have to see them.

  2. Bring WED 1.1 to beta. The only thing holding it back is the DSF exporter, and I did have about two hours to poke at it last week. I'm hoping if I can find just a few more hours, I can finish off the exporter.

  3. Examine 950 bugs. I have half a dozen bug reports against 950 beta 1. 950 will be a small beta but also a slow one, because Austin and I have a lot of other things on our plates. If you haven't heard back from me on a bug report, probably it's still on my to-do list.

We'll see how much of that I can get to in the next week.

Tuesday, March 02, 2010

WorldEditor Export: File a Bug!

There's basically one reason why WorldEditor developer preview 2 is a "developer preview" and not a real beta: the DSF overlay export code isn't complete.

The problem is that, unlike an airport, an overlay has to be "cut" to the DSF tile boundaries. This is made slightly tricky by the fact that the overlay can have (1) bezier curved segments and (2) a UV map on those segments. My existing toolkit of polygon editing routines doesn't handle this case yet.

I have no idea when I will have time to complete this code. It is the number one piece of code that, if I had a quiet single afternoon of unexpected time to code, I'd pound it out. If I were stuck in an airport with my laptop, I'd pound it out. It should give you some idea of how busy things are that it still isn't done.

In the meantime, there is the scenery tools bugbase. By filing a bug, your issue won't get lost even if it's a while before I get to it.

A few quick rants about the bug base:
  • Most likely the first thing I'll do when I do get to your bug is just ask you for more info. Consider the bug to be as much a business card so I can make contact as well as a bug report.
  • Some bugs may get kicked out.
  • Do not file X-Plane bugs in the scenery tools bug base! The scenery tools bug base is not where we store sim bugs.
Do not bother to ask for direct bug base access for X-Plane itself. You cannot have it. The ratio of submitted "bugs" for X-Plane to actual bugs is at least 10:1. That is, 90% of you think that you should file a bug when you have a tech support question. Now you might be in that 10% (particularly if you've made it all the way down to this blog post), but we can't set up open infrastructure with those numbers. My hope is that the scenery tools are self-selecting to the point where people who are using developer-preview tools know what a bug report is.

Another Reason To Use a Few Big Textures

The file loading code in 950 beta 1 for Windows is slower than 945. Sometimes. This will be "fixed" in beta 2. Here's what happened:

The scenery system uses a number of small files. .ter files, multiple images, .objs, etc. This didn't seem like a problem at first, and having everything in separate text files makes it easier to take apart a scenery pack and see what's going on.

The problem is that as computers get bigger and faster, rather than a scenery pack growing bigger files, they are growing more files. The maximum texture size has doubled from 1024x1024 to 2048x2048. But with paged orthophotos, multicore, and a lot of VRAM, you could easily build a scenery pack with 10,000 images per DSF.

That's exactly what people are doing, and the problem is that loading all of those tiny files is slow. Your hard drive is the ultimate example of "cheaper by the dozen" - it can load a single huge file at a high sustained data rate. But the combination of opening and closing files and jumping between them is horribly inefficient. 10,000 tiny .ter files is a hard drive's worse nightmare.

In 950 beta 1 I tried to rewrite part of the low level file code to be quicker on Windows. It appeared to run 20% faster on my test of the LOWI demo area, so I left it in beta 1, only to find out later that it was about 100% slower on huge orthophoto scenery packs. I will be removing these "optimizations" in beta 2 to get back to the same speed we had before. (None of this affects Mac/Linux - the change was only for Windows.)

The long term solution (which we may have some day) is to have some kind of "packing" format to bundle up a number of small files so that X-Plane can read them more efficiently. An uncompressed zip file (that is, a zip where the actual contents aren't compressed, just strung together) is one possible candidate - it would be easy for authors to work with and get the job done.

In the short term, for 950 beta 2, I am experimenting with code that loads only a fraction of the paged orthophoto textures ahead of time - this means that some (hopefully far away part) of the scenery will be "gray" until loaded, but the load time could be cut in half.

There is one thing you can do if you are making an orthophoto scenery pack: use the biggest textures you can. Not only is it good from a rendering perspective (fewer, larger textures means less CPU work telling the video card "it's time to change textures") but it's good for loading too - fewer larger textures means fewer, larger total files, which is good for your hard disk.

(Thanks to Cam and Eric for doing heavy performance testing on some of the 950 beta builds!)

Saturday, February 27, 2010

Plugin Performance

This blog post is for amateur plugin developers. By amateur I mean: some plugin developers are professional programmers by day, and are already familiar with all aspects of the software development progress. For those developers, the SDK is unsurprising and performance is simply a matter of applying standard practice: locate the worst performance problem, fix it, wash-rinse-repeat.

But we also have a dedicated set of amateur plugin developers - whether they had programming experience before as hobbyists, or learned C to take their add-ons to the next level, this group is very dedicated, but doesn't have the years of professional experience to draw on.

If you're in that second group, this post is for you. Explaining how to performance tune code is well beyond the scope of a blog post, but I do want to address some fundamental ideas.

I receive a number of questions about plugin performance (to which the answer is always "that's not going to cause a performance problem"). It is understandable that programmers would be concerned about performance; X-Plane is a high performance environment, and a plugin that wrecks that will be rejected by users. But how do you go from worrying about performance to fixing it?

Measure, Measure, Measure, Measure.

If I had to go crazy and recite a sweaty and embarrassing mantra about performance tuning so that I could be humiliated on YouTube it would go: measure, measure, measure, measure.

If you want your plugin to be fast, the single most important thing to know is: you have to find performance problems by measurement, not by speculation, guessing or logic.

If you are unfamiliar with a problem domain (which means you are writing new code or a new algorithm - that is, doing something interesting), there is no way you are going to make a good guess as to where a performance problem is.

If you have a ton of experience in a domain, you still shouldn't be guessing! After 5 years of working on X-Plane, I can make some good guesses as to where performance problems should be. But I only use those guesses to search in the likely places first! Even with good guesses, I rely on measurement and observation to make sure my guess wasn't stupid. And even after 5 years of working on the rendering engine, my guesses are wrong more often than they are right. That's just how performance tuning is: it's really hard for us to guess where a performance problem might be.*

Fortunately, the most important thing to do, measuring real performance problems, is also the easiest, and requires no special tools. The number one way to check performance: remove the code in question! Simply remove your plugin and compare frame-rate. If removing the plugin does not improve fps, your plugin is not hurting fps.

It is very, very important to make frame-rate comparison measurements under equal conditions. If you measure with your plugin in the ocean and without your plugin at LOWI, the results are meaningless. Here's a trick I use in X-Plane all the time: I set my new code to run only if the mouse is on the right half of the screen. That way I can be sitting at a fixed location, with the camera not moving, and by mousing around, I can very rapidly compare "with code", "without code". The camera doesn't move, the flight model is doing the same thing - I have isolated just the routine in question. You can do the same thing in your plugin.

Understand Setup Vs. Execution

This is just a rule of thumb, and you cannot use this rule instead of measuring. But generally: libraries are organized so that "execution" code (doing stuff) is fast, while setup and cleanup code may not be. The SDK is definitely in this category. To give a few examples:
  • Drawing with a texture in OpenGL is very fast. Loading up a texture is not fast.
  • Reading a dataref is fast. Finding a dataref is not as fast.
  • Opening a file is usually slower than reading a file.
  • You can run a flight loop per frame without performance problems. But you should only register it once.
If you want to pick a general design pattern, separate setup from execution, and performance-tune them separately. You want things that happen all the time to be very fast, and you can be quite intolerant of performance problems in execution code. But if you have setup code in your execution code (e.g. you load your textures from disk during a draw callback) you are fighting the grain; the library you are using probably hasn't tuned those setup calls to be as fast as the execution code.

Math And Logic Is Fast

Modern computers are astoundingly fast. If you are worried that doing a slightly more complex calculation will hurt frame-rate, don't be. One of the most common questions about performance I get is: will my systems code slow down X-Plane. It probably won't - the things you calculate in systems logic are trivial in computer-terms. (But - always measure, don't just read my blog post!)

In order to have slow code you basically need one of two things:
  1. A loop. Once you start doing some math multiple times, it can add up. Adding numbers is fast. Adding numbers 4,000,000,000 times is not fast. It only takes one for-loop to make fast code slow.
  2. A sub-routine. The subroutine could be doing anything, including a loop. Once you start calling other people's code, your code might get slow.
This is where the professionals have a certain edge: they know how much a set of standard computer operations "cost" in terms of performance. What really happens when you allocate a block of memory? Open a file? If you understand everything going on to make those things happen, you can have a good idea of how expensive they are.

Fortunately, you don't need to know. You need to measure!

SDK Callbacks Are Fast (Enough)

The SDK's XPLM library serves as a mediator between plugins and X-Plane. Fortunately, the mediation infrastructure is reasonably fast. Mediation includes things like requesting a dataref from another plugin, or firing off a draw callback. This "callback" overhead contains no loops internally, and thus it is fast enough that you won't have performance problems doing it correctly. One draw callback that runs every frame? Not a performance problem. Read a dataref? Not a performance problem. (Read a dataref 4,000,000 times inside a for-loop...well, that can be slow, as can anything!)

However you should be aware that some plugin routines "do work". For example, XPLMDrawObject doesn't just do mediation (into X-Plane), it actually draws the object. Calls that do "real work" do have the potential to be slower.

Be ware of one exception: a dataref read looks to you like a request for data. But really it happens in two parts. First the SDK makes a call into the other plugin that provides the data (often but not always X-Plane itself) and then that other plugin comes up with the data. So whenI say "dataref reads are fast" what I really mean is: the part of a dataref read that the SDK takes care of is fast. If the dataref read goes into a badly written plugin, the read could be very, very slow. All of the datarefs inside X-Plane vary from fast to very fast, but if you are reading data from another plugin, all bets are off.

Of course, all bets are off anyway. Did I mention you have to measure?

* Why can't we guess? The answer is: abstraction. Basically well structured code uses libraries, functions, etc. to hide implementation and make the computer seem easier to work with. But because many challenging problems are hidden from view (which is a good thing) it's hard to know how much real work is being done inside the black box. Build a black box out of black boxes, then do it again a few time, and the information about how fast a function is has been obscured several times over!

Friday, February 26, 2010

Stutters Fixed (I Hope)

X-Plane 950 beta 1 should fix the stuttering present in 940. The bug was: the more you flew in one area, the worse the stutters would get - they'd clear out if you kept flying forward enough to scroll DSF tiles normally. Anyway, the bug was a book-keeping error based on the new "paged" airports from 940.* Fixed now - thanks to the users who got me a really clean test case to see this.

(Hint for users: if you can duplicate a scenery-related bug by saving a replay, or even if the bug can be reproduced in replay, that's really, really useful to know! It makes the bug much more repeatable, since what scenery is loaded varies a lot with flight path.)

* Before 940 we would load the entire apt.dat file into memory on startup. With Robin receiving new highly detailed airport layouts every day, this was starting to use more and more memory. 940 and 950 will load a litttle bit of information about every airport and take note of the file that contains the layout - then when the sim needs a layout, it goes back to disk to pull up just the one airport in question, saving memory. We "page" the airports into memory only as needed.

Updater Design Decisions

My blog postings have been a little thin - but not because nothing is going on. Rather it's a combination of working on next-gen stuff that is still in heavy development, travel, and (perhaps due to the travel, perhaps due to lack of sleep, or perhaps due to certain forms of Brandy brought on site by a certain British developer who won't be named, but whose handle on the org starts with a P and rhymes with "Cropsman", cough, cough) a minor fever that I've just about recovered from.

A user asked me about the design of the X-Plane updater - so this post is only going to be of interest to the few authors out there who are creating installers for their add-ons and need to update. We try to keep the file formats for X-Plane simple, e.g. "this one folder is your airplane", so that if you don't make an installer, the user won't be overwhelmed with a complex unpack-and-install routine. But if you do need to install or update, well, here's what we were thinking when we developed our installer.

We had a few needs for our installer:
  • It had to be cross-platform to Mac, Windows and Linux.
  • It had to update existing products as well as install new ones.
  • The updates had to be minimal deltas, e.g. a demo can be 600 MB but an update should only be the 30 MB of files that really changed.
  • We wanted the updater to do delta updates from any old version. (Some update systems require installing a number of patches in sequence. We have users who buy the DVD and then update a year later; we wanted to let them update in one shot.)
  • Building the patches had to be really, really fast. We use our updater to publish our betas. So while X-Plane probably has 2-3 major patches in a single year and maybe 5 or 6 bug fix patches, we will (due to betas) cut perhaps 100 or more actual "updates".
  • On that last point, cutting separate updates by platform was a deal breaker, as it would triple the number of updates we had to cut. Having a demo install and an update be the same on the server was a big win.
Some of those needs are pretty specific to Laminar Research, but some might apply to a third party. There are commercial installers that will let you generate patches. If you don't generate a lot of patches, you can probably use existing tools - basically you'll spend a lot less time up front (since the tool set exists) but you might spend more time in the long run cutting separate patches by OS.

One of the first things we decided was to not use server-side technology or special protocol. That is, we don't need any kind of smart servers to run the updater - the updates are just files hosted over HTTP on standard rented or owned apache servers. All of the work is done in the client. We did this for a few reasons:
  • It lets us throw up the install anywhere - we don't have complex needs for what's on the server. Virtually any server will do. (As the company has grown, our server needs have grown too, so this is less of a concern now, but back then we had fewer servers in service, and were buying less overall server capacity.) If a server goes down, we can press another one into service about as quickly as we can move the file set to the server.
  • We're not server programmers. Coding the installer/updater on the client side let us leverage existing company technology, etc.
Here's what we came up with. The basic idea of the installer is very simple. Note that since the installer uses HTTP files, you can "watch" the installer simply by downloading the files that it downloads.
  1. First the installer goes to a master server and gets a "version list", which tells it what it's going to actually get and what (multiple) mirrors are available. This one master file is the only file that must be hosted at X-Plane.com, and it allows us to change a very small file to press mirrors into service.
  2. Installs and updates are actually the same to us - it's a set of files that the user should have once the install or update finishes. This starts with a file directory that lists every file (by path) that needs to be downloaded, as well as its MD5 signature.
  3. The directory also contains the MD5 of every old version of any file in the version, and lists files that have to be deleted to update the version (e.g. the file panel.png used to have MD5 signature 2934b..23abc2 but is now removed).
  4. The client, once armed with the file directory, can now download all files in the directory (install) or compare the existing files on disk to the directory by MD5 and only download changes.
  5. Individual files are on the server in zip form for download and decompression.
That's pretty much it - you could write an installer in a scripting language using a tool like CURL or WGET - it wouldn't be very complex because the installer is really just a clever, painted download+unzip tool.

Dealing With Modified Files

There is a bit of complexity in this design that you might not need for a third party installer: handling modified files. Note that the directory contains the MD5 not only of the current version of the file (to detect when no update is needed and save bandwidth) but also to see when the user has modified a file that is "managed" by X-Plane.

Before the installer overwrites/deletes a file that you own, it compares the file's MD5 to the entire known list of MD5s for the entire version. If this file doesn't match a known old version, it puts up the warning dialog box that you have modified a file that is about to be overwritten.

While we recommend that add-ons use scenery packs and other "safe" ways to customize the sim, this helps detect when a user has gone in and edited the cloud textures in Photoshop, and prevents us from overwriting them without warning.