Wednesday, February 03, 2010

DSF and WED Precision

I just finished fixing some bugs with WorldEditor and DSF precision. A WED user was seeing draped polygon tiles exporting with alignment error, and the culprit turned out to be DSF precision. This begs the question, how precise are the lat/lon coordinates in a DSF?

DSFs are based on 16-bit coordinates, for a precision of one part in 65536. Now before you go screaming that 16-bit is dreadful, here's the key: those 16-bit coordinates are interpreted within an arbitrary sub-rectangle of your tile, called a "pool". If we only had one pool covering the DSF we'd have 1.5 meter precision. But by having many smaller pools, we can have high resolution within a pool.

All of the current DSF writing tools from Laminar Research are based on DSFLib, and DSFLib will create a number of pools along a pair of offset grids. In practice what this means is that you specify a number of pools across the DSF.

In WorldEditor 1.1dp2 the DSF exporter uses 8 pools. Since a DSF is about 100 km across, each pool is 12,500 meters in size, and the internal resolution of each pool is 19 cm. This is pretty good but there are cases where you can see the limits of precision. The new code (which will be in preview 3 or beta 1) uses 32 pools, for 5 cm precision.

If you use DSF2Text, you can simply use the DIVISIONS command to specify how many pools you want.

WorldEditor's internal representation is 64-bit floating point, which gives precision better than 1 mm.


Geoff said...

One thing I've noticed when flying in to a scenery with draped polygons (mostly converted MSFS sceneries) is that their rotational alignment is skewed when you finally land. The orientation is only correct when you Select Global Airport and load up directly at the airport. In the future will X-Plane be able to realign these ground polygons each time you enter a new 1x1 degree region so that these sceneries objects are angled correctly for where the aircraft currently is?

Benjamin Supnik said...

Hrm - they should not be misaligned. Please send me via email some bug reproduction materials so I can see this.

joloize said...

Hello Ben,

I tried what you said :
"If you use DSF2Text, you can simply use the DIVISIONS command to specify how many pools you want."

But it won't work...

What is this command syntax ?
I tried these two :

PROPERTY sim/divisions 32 ?