Monday, August 30, 2010

How Specific Should a Bug Report Be?

I got this bug report from Austin this morning (found in the in-development version 10 code):
yter bug
0: go to kcae with no scenery installed at all
1: open the 777
2: go to aircraft:aircraft and situations
3: go to second tab
4: set total num acf to 1
5: go to first tab
6: select 'carrier catshot'
7: release the brakes, with throttle at idle
8: enjoy the cat.. ease the plane gently onto the water, gear down, pwer off
9: note that the water is 'hard'... the yter is telling us that we are on land not water. hit brakes. stop. u r walking on water!
The first thing to note is how specific and small each step is. These steps are much more precise than most of the bug reports we get. And yet, they must be that specific - the bug was so tweaky that skipping virtually any of those steps will cause it to not appear.

The biggest problem I see with bug reports sent to me is the use of "any", e.g. "open any aircraft". That's not very specific! And Murphy's law says: the one airplane I pick will just happen to be the one airplane that you haven't tried and happens to not show the bug for some strange reason.

So always be absolutely specific. When we ask for tiny steps, it's not that we don't know how to use the sim - it's just that we need to use the sim exactly the same way you did!

EDIT: bug report pages.
Be sure you understand which bug base you should be using (E.g. are you reporting a bug against the plugin SDK, the sim itself, or a scenery tool like WorldEditor.) If you don't know what program you are filing a bug against, you probably shouldn't file a bug at all - just contact tech support.


Anonymous said...

Sounds more like a miracle than a bug.

Cormac said...

I feel a little ashamed! :) I complain at work repeatedly about bug reports being too vague ... and then do the same thing to Ben with WED bug reports! I think it may be because it's usually after midnight when I submit this crap!

Benjamin Supnik said...

Cormac, your bug report had a complete scenery pack attached, if I remember correctly, which is pretty good. :-) :-)