Kuzuan
(?)Community Member
- Posted: Sat, 10 Jul 2010 00:50:45 +0000
Bug Report's Anatomy, Pt. 1/2
During the past installments we took a look at the tools and methods that we use in order to find bugs. Our task is only half done once we identify an issue because one of the most important aspects of our work is still before us: creating and filing a bug report.
Put simply, a bug report is a digital form that we fill out with information related to a particular issue in order to create a record of the problem in our bug tracking database. Often we file bugs based on the reports that all of you as beta testers post on these very forums! In effect you are acting as extensions of our eyes and ears in the game and we are able to constantly monitor a much bigger part of the project with your help.
I hope to see all of next week for Part 2 of "Bug Report's Anatomy" that will continue to flesh out the kind of information and actions that go into each bug report. To tie things together I will be showing samples of completed bug reports and holding a small contest where participants can win prizes by identifying all the mistakes present on a submitted bug report! biggrin
Previous topics:
zOMG! Behind the Scenes - The Great Cycle of Builds
zOMG! Behind the Scenes - Toolkit of Trade
zOMG! Behind the Scenes - Method in Madness
During the past installments we took a look at the tools and methods that we use in order to find bugs. Our task is only half done once we identify an issue because one of the most important aspects of our work is still before us: creating and filing a bug report.
Put simply, a bug report is a digital form that we fill out with information related to a particular issue in order to create a record of the problem in our bug tracking database. Often we file bugs based on the reports that all of you as beta testers post on these very forums! In effect you are acting as extensions of our eyes and ears in the game and we are able to constantly monitor a much bigger part of the project with your help.
- Why file Bug Reports?
Besides keeping things organized, filing bugs allows us to maintain a record of that bug for the duration of its "life". Changes, attempted fixes and additional details can be added to a bug report in order to maintain a clear history of how we have tried to resolve the issue. Managing any project of non-trivial complexity becomes extremely hard without a centralized bug tracking database. Here at Gaia we use a free, open-source, web-based bug tracking Database called FlySpray.
- Preparing for the Report
We work on reproducing the bug once we are confident that we'll be the first to file the issue. This means that we work to finalize a list of steps that can be followed by another person in order to make the bug occur as close to 100% of the time as possible. We start creating our brand-new bug report as soon as we are satisfied with how often we can reproduce the bug, our steps for reproducing it, and once we have gathered any additional details about the issue.
- The Report: Step by Step
After taking a deep breath we click the "Add new task" button and we are greeted with the following screen:
- Bug Report's Anatomy
- Summary - A catchy title is important for any book in order to attract readers. For similar reasons a unique and detailed bug summary (or bug name) is necessary with each new report. A good summary keeps things ordered, helps make your bug easily identifiable and gives a new reader a quick idea of what the problem concerns.
Task Type - As may be expected, the "Bug Report" task type constitutes the vast majority of records in our bug tracking database. The other available Task Type is "Enhancement" and is used whenever we'd like to submit a new idea or an improvement to an existing feature for considering.
Category - This field helps the Project Manager assign the bug to the right person by indicating what part of the game is most affected by the problem. The example of "Backend / Core" would be used for issues dealing with the game sever, game stability and performance.
Operating System - The vast majority of our testing is performed on Windows XP. A smaller subset of testing is also done on machines running Windows Vista and OS X.
Severity - In some sense a bug report is an alarm bell designed to alert others that a defect has been found in our software. Severity is the field that allows us to indicate how hard we're ringing this proverbial bell by describing how big of an impact we believe the bug has on a specific feature or the game in general. Below I will list the different severity levels and the meaning behind each one.
- neutral Low: The majority of the bugs we find are filed with Low severity. These bugs fail to prevent players from using any game features, but they detract from the fun or gameplay experience.
stare Medium: A smaller portion of our bugs falls under Medium severity. This level means that a bug is problematic enough to break a feature, however a user would be able to continue or work around the bug by doing something (ex: refreshing the game).
eek High: A couple of bugs in each build cycle may fall under High severity. These issues hang out in the rough parts of town and their presence prevents the player from using a feature, stops them from advancing in the game, or breaks a chunk of the game. Only bugs without a workaround earn this title.
wahmbulance Critical: The femme fatale. The show stopper. Mercifully rare, bugs that earn this severity level are ones that make the game unplayable, break fundamental game features or threaten the very stability of game servers.
This may seem like an a baffling large list of criteria to keep in mind for each issue that's filed, however experience has demonstrated many times that each part plays an important role in the overall composition of a bug report.
I hope to see all of next week for Part 2 of "Bug Report's Anatomy" that will continue to flesh out the kind of information and actions that go into each bug report. To tie things together I will be showing samples of completed bug reports and holding a small contest where participants can win prizes by identifying all the mistakes present on a submitted bug report! biggrin
Previous topics:
zOMG! Behind the Scenes - The Great Cycle of Builds
zOMG! Behind the Scenes - Toolkit of Trade
zOMG! Behind the Scenes - Method in Madness