Thursday, December 20, 2007

A Tools Update

I've spent perhaps the last two weeks almost entirely looking at driver-related graphics bugs. I should note that these are often not driver bugs, and the issue of whether a bug is in the sim or a driver is a lot stickier than you'd imagine, but I'll blog about that later.

In the meantime, where do we stand on scenery development tools? Here's a rough summary.

General Strategy

The scenery tools are not targeted to X-Plane 8 or 9 - they'll export content for both (but if you use new V9-only features, your content will be v9 dependent).

There's been one structural change in v9 that's worth mentioning: version 9 allows road networks in DSF overlays. This cleans up the scenery model a lot...roads are more like objects and forests than like the base mesh, so now all of the 3-d elements can be in an overlay. This is particularly useful because you might want to adjust the road and objects nearby all at once.

Given that we have no road editing tools right now, I think there is one case where the tools will be v9-centric: I do not think I will ever provide a road-editing tools for DSF base meshes with roads. I do think I will add road editing to the list of overlay features that will eventually end up in WED.

This means that if you want to customize roads, you'll have to do it in version 9. This is an inevitable outcome of the scenery system structure...replacing the entire base mesh to edit the roads is not a great idea.

WED

WED is in beta 4, and I think it will go final as soon as I have time to make it official. At this point most of the open bugs I get are usability issues and enhancement requests, not real bugs. We'll get more usability features into WED in the future.

Please don't send me WED bugs now (or attach them as blog comments). I'll put out a formal last-call for WED bugs when I have time to dig in.

The future plan for WED is to add DSF overlay editing with all legal overlay types (which is now everything except beaches and base mesh).

AC3D

The AC3D export plugin already supports new v9 OBJ features - key frames and panel regions. These tools are available now!

Tools

The "tools" are the set of other scenery apps: DSF2Text, ObjConverter and ObjView. We'll be adding two new tools soon:
  • DDSTool is a PNG->DDS converter. There are already other programs (many free) for both platforms that can do this conversion, but we wanted something easy and drag & drop when we were doing the in-house conversion, so I coded DDSTool (based on the Squish library). We'll make it available to authors.
  • MeshTool is a base-mesh generation program. We're still working out the details on this, but basically it will generate DSF base meshes from raw DEM data and polygonal coastlines.
I am restructuring the tools to be entirely command-line-based (except for ObjView) with a single "drag & drop" UI that wraps all of the tools. So if you use the mouse, you'll use the one drag & drop application to do all your DSF2Text converions, 3DS->OBJ conversions, PNG->DDS conversion, etc. This is mainly to cut down the number of apps I have to build, but it should also declutter workflow a bit.

MeshTool

I'll post more on MeshTool once we have more details, but here are the main points:
  • MeshTool will not have a UI. It will take text files and process them. It is designed to provide a building block on which people can write more advanced tools. In other words, MeshTool gives other programmers our mesh algorithms just like DSF2Text gives other programmers our DSF-packing algorithms.
  • MeshTool will not be an editor. It will create NEW meshes from RAW DATA, not from previous meshes. There are a lot of reasons why I do not want the mesh-creation workflow to be "take the global scenery and hack it". So MeshTool lets you build your own.
  • This means you will need your own elevation and coastline data for MeshTool! You can't use what shipped in the global scenery! However, since the data that ships the global scenery is freely downloadable and widely available, I don't think this is a problem.
  • MeshTool will be able to "burn in" orthophotos into a mesh. This is probably its most important feature. Right now the only way to make orthophoto-based scenery is to use overlay polygons, but the performance of .pol files with a lot of scenery is really poor under version 9. MeshTool will let people build photo-sceneries that take full advantage of the framerate that X-Plane can deliver!
MeshTool should address some of the common issues I'm hearing with orthophoto sceneries (slow performance with a ton of .pol files, no ability to fix the coastlines or remove underlying base terrain, no ability to fix elevation).

MeshTool will create new default landuse terrain where there are no orthophotos or water. This is important because the elevation data can be customized. If you take an existing DSF and somehow edit the elevation points (for example, use DSF2Text and just start editing) and raise a grassy field, you'll have grass going up a cliff. But if you pass the raw elevation data to MeshTool, it will build a rocky cliff over steep elevation, creating more plausible scenery.

MeshTool should be in the post 9.0 final time-frame, as will all the other tool reposts.

10 comments:

alpilotx said...

Hi Ben,

Now this mesh tool sounds like a really great idea! But the only point I'm missing is: where does it get it's land use information (as you don't really tell it in you Blog)? Yes, it can do something with elevation and the slope, but that can't be everything it does! So where in this picture comes the land use in (for example if I would consider creating European Mesh using my CORINE data for the land use: which would result in some great scenery)???

Benjamin Supnik said...

Hi AlpilotX -

Of course you have immediately noticed the critical detail I left out: you provide the elevation and coastline data, but what about landuse?

The answer is: we (Laminar Research) will post our landuse and climate data for the entire world in a pre-encoded format, ready for MeshTool. Because this data is so low res, these files will be very small and easy for authors to download, and shouldn't be a hosting burden.

Now with our initial version, I do not think it will be possible to edit this data. Basically my goal is to make a very simple "just hit go" tool to create the landuse terrain. The actual process of editing landuse is exceedingly complex because the landuse data is pre-processed and run through a very complex rules file (built by Sergio and his team) that takes into account a large number of variables.

So even if we allowed landuse editing, it wouldn't be as simple as "draw swamp in photoshop, get swamp".

Our initial goal is simplicity and getting the tool out there fast to serve people trying to make orthophotos. We can look at eventually allowing specific landuse-style polygons (in the first version, only airports, water and orhtos can be specified).

But the goal of meshtool is to make the process of cutting photosceneries easier, not to provide a general GIS tool, even though that is what the global scenery generator more or less is.

(As Austin and Sergio can tell you from having worked with it, working with the actual DSF generator is a real pain and requires a lot of setup.)

Dan 31 said...

Do you know if we will have to cut the orthophoto in small pieces or if we can load a large image ? Could we make river with real water (I mean use the water reflexion) ?
I am still using v 7.63 for odthophoto.

Benjamin Supnik said...

Dan,

If your imagine is larger than the max texture size* you will have to cut it by hand and place each individual cut orthophoto.

I expect MeshTool to be a support tool for other more user-friendly tools, so hopefully someone else can automate this process.

You will be able to create water either polygonally (by using a water polygon) or via alpha (by having alpha in the orthophoto). Both methods will create "real" reflective X-Plane water that matches the water in the default scenery.

(This isn't a new capability of X-Plane, it's just a question of a tool that creates an appropriately formed DSF.)

Anonymous said...

This is a great news!

Does this mean we could generate dsf for svalbard (for example) if we can get DEM data for them, and apply the default land use data?

Benjamin Supnik said...

Anon: maybe. We do have landuse all the way to the poles. You would need to get coastline data as well as a DEM. I hesitate only because our mesh generator gets funky as you go north...it's been run to 75N successfully, but svalbard is even further north and could reveal a bug in the mesh-gen code.

Lemon said...

Hi,
Are you considering adding a sign maker to your next version, if so, I needn't carry on with mine?

Lemon

Benjamin Supnik said...

Lemon: I do have a sign maker on the todo list for WED, but I do not think it will happen any time soon. Furthermore, if you continue with your sign maker, this gives users a way to make signs now and lets me focus on other features. So I would strongly encourage you to continue with your sign maker!!!!

Anonymous said...

Hello Ben.

A simple question, I struggled a while ago building custom roads for sweden using accurate GIS data for X-plane 8.60 in the SK60 projekt. I had to do some tricky programming to be able to cut the roads to follow the edges of the underlying mesh to avoid Z-flimmering.

Would it be possible for you to add a function that takes a set of road vectors and make them follow the underlying mesh correctly?

/Jouni Lindqvist

Benjamin Supnik said...

Jouni, I am looking into that, but it's not entirely clear to me what the right solution is. For example, if there is an unexpected canyon in the terrain, you would (to get a nice flat highway) have to provide AGL heights (a bridge) that exactly counteract the canyon. One advantage of MSL heights is that terrain fluctuations don't show up directly in the road. I am open to ideas on an appropriate solution.