Wednesday, November 24, 2010

The Four Minute Mile

Sometimes these posts get off topic, sometimes in the direction of the art of computer programming, sometimes in the nature of the industry, and sometimes with pictures of the pets. This post is going to go off a bit into the subject of project management.

Randy and Tyler posted what was becoming clear (by the lack of an already existing beta): our estimated release date for X-Plane 10 was incorrect. Software project delays are pretty common, and often when a third party add-on is delayed, the community jumps to speculate about "what's going on" inside the project and tries to infer whether the delay is an indication of serious problems.

I'd like to try to reframe the issue of delays in terms of an analogy. You ask me: how fast can you run a mile? I tell you "4 minutes and 15 seconds". I then run a mile and you time me. My time: 6 minutes, 10 seconds.* What can we learn from this episode? I think we can learn two things:
  • For a computer programmer, I am surprisingly fast - a six minute mile isn't to be sneezed at when you spend your days sitting on your ass in front of a monitor drinking coffee.
  • My ability to predict my own speed is not very good. I was pretty naive to think I could run a 4 minute mile - that's what world class athletes run. My estimate was off by a fairly big error margin.
One thing we should not conclude is that because my mile time was 2 minutes slower than estimated, that I am a slow runner. The estimate sets up an expectation, but if the estimate is wrong, it's not a useful metric of efficiency.

The same applies to X-Plane; we missed our original projected ship date because the estimation of when we would be done was not a very good estimate. This isn't good for a few reasons:
  • It creates uncertainty for third parties as to when a platform will change.
  • It makes it difficult for marketing to properly plan a roll-out.
  • It makes it difficult to balance the value of more features vs. an earlier release date (since we don't know how much "time" we are trading for "features" if the time estimates are wrong).
But the delay is not at all a black mark for our team - on the contrary, they're working their asses off and creating some really great work.

When looking at a project that will be delayed (because the original schedule was wrong) there's a few things you can do:
  1. Add more people. This is quite often the wrong thing to do - please read the Mythical Man Month to understand why. Once your team is the right size, adding more warm bodies usually makes schedule delays worse and hurts efficiency.
  2. Remove features. This is the only real way to bring in a ship date.
  3. Move the date back.
When Austin and I were working on X-Plane 8, we hit a similar scheduling problem - what we had set out to do was going to take a lot longer than we thought. (Like X-Plane 10, we had just doubled the team size and begun a project that involved massive rewrites, which made it hard to ship until the work was fully complete. Sound familiar?) The difference? With X-Plane 8 we had contracted to ship with an external distributor for Thanksgiving, so we had to go for item 2 - we cut scope. What we cut was the world - that is, we shipped new global scenery only for the US, and the existing ENVs for the rest of the world. We also had to ship the artwork we had on hand, despite being unhappy with its quality. We didn't finish the rest of the world and graphics we were happy with for another 11 months.

Option 2, cutting scope is painful and hard. Sometimes it is the right thing to do. In the case of X-Plane, however, we have the luxury to move the date back. With that in mind, we're trying as hard as we can to keep feature-creep minimal and finish what we've already bit off, so we can get the release done and out the door.

* My mile time is not 6 minutes, 10 seconds...I would be astounded, and quite possibly in the ER if I could run that fast for any sustained amount of time.

7 comments:

Steve In Tucson said...

When one is involved in a major coding effort with rewrites, problems, ambitions and successes all pullingin different directions, it can often be difficult to see the forest, not just all the trees. Been there, done that, got the T-shirt (and near nervous breakdown). The world will not grind to an unpleasant halt if X-Plane 10 is not under my (Christmas tree) (please substitute the gift giving icon of choice). You guys are to be commended for stepping back and being real with us. Thanks for that. And thanks for the screamin' hard effort.

**If you code it, we will buy.**

six7 said...

Let the whiners whine. If they're not whining about release dates, they're whining about something else. You can't win with them.

Take your time. Make it right. Release it when it's ready. We'll wait.

Jaron Latona said...

Ben, I think I can speak for the vast majority when I say "Take your time, we have faith in this product." :)

Jay Douglas Carr said...

Just adding my voice to the comments so far. I'd rather wait for a stable, awesome X-plane with full ATC, lighting, textures (etc), than have a half baked version right now. I mean, I waited 5 years for Gran Turismo 5 and that didn't kill me (and it was certainly worth the wait.)

Also, for anyone interested, Wikipedia had a brief synopsis of the Mythical Man Month theory: http://en.wikipedia.org/wiki/The_Mythical_Man-Month#The_mythical_man-month

Keep up the good work guys!

Jack Skieczius said...

I have complete faith in what you guys are doing. I trust you will release it when you are ready. To tell ya the truth, i am not ready for XP10 yet.

You guys are doing great so far. and by the way, i love this blog.

Clay Moore said...

Naaaaahh. You should have planned your release-date more conservatively. Sure, I will buy X-plane 4 months late, but others will await M$ Flight (or something else).
There is a reason why launches flock around Christmas: people have cash to burn! No matter how you turn this, this delay will make you lose (some) potential customers...
Hardcore fans (like me) will buy XP10 whenever it comes. But you sure missed out a good opportunity to open the field to casual simmers...

Anonymous said...

get ideas
http://www.youtube.com/watch?v=NSjzO1cip3E
http://www.youtube.com/watch?v=RTLaEqXtHnU&feature=player_embedded
http://www.youtube.com/watch?v=dMCZcbp94mw&feature=player_embedded
http://www.youtube.com/watch?v=FwtKHbl0pjs&feature=player_embedded