zOMG! Behind the Scenes - Bug Report's Anatomy, Pt. 1/2

[Kuzuan]

Bug Report's Anatomy, Pt. 1/2

During the past installments we took a look at the tools and methods that we use in order to find bugs. Our task is only half done once we identify an issue because one of the most important aspects of our work is still before us: creating and filing a bug report.

Put simply, a bug report is a digital form that we fill out with information related to a particular issue in order to create a record of the problem in our bug tracking database. Often we file bugs based on the reports that all of you as beta testers post on these very forums! In effect you are acting as extensions of our eyes and ears in the game and we are able to constantly monitor a much bigger part of the project with your help.

• Why file Bug Reports?

Keeping a record of all identified bugs is extremely important during the lifetime of any project. Many bugs will be found in the code written by even the best programmers and things quickly turn chaotic without a way of keeping track of all existing bugs all the time.

Besides keeping things organized, filing bugs allows us to maintain a record of that bug for the duration of its "life". Changes, attempted fixes and additional details can be added to a bug report in order to maintain a clear history of how we have tried to resolve the issue. Managing any project of non-trivial complexity becomes extremely hard without a centralized bug tracking database. Here at Gaia we use a free, open-source, web-based bug tracking Database called FlySpray.

• Preparing for the Report

Just like it pays to look before you leap, it's important to search the bug tracking database before you file a new bug report. When a project is old and has multiple people working to identify bugs it's common to stumble across bugs in the game that have already been filed. Since only one report bug is allowed in the database it's necessary to do a search in order to save yourself time spent filing the bug in case the issue has already been reported. sweatdrop

We work on reproducing the bug once we are confident that we'll be the first to file the issue. This means that we work to finalize a list of steps that can be followed by another person in order to make the bug occur as close to 100% of the time as possible. We start creating our brand-new bug report as soon as we are satisfied with how often we can reproduce the bug, our steps for reproducing it, and once we have gathered any additional details about the issue.

• The Report: Step by Step

Once we are ready to write out report we open FlySpray and navigate to the zOMG! section of the database. One important rule to remember is that, just like there must be only one report per bug, there must be only one bug per report. As before, maintaining this 1:1 relationship is what helps maintain order and prevents the Lord of Chaos from sneaking in and stealing all the marzipan. gonk

After taking a deep breath we click the "Add new task" button and we are greeted with the following screen:

• Bug Report's Anatomy

Don't panic! The above form may seem overwhelming, but the majority of the time only six items are changed whenever a new report are filed. Below I will briefly describe the elements of a bug report that must be set every time. It's important to select these elements correctly so that searching your bug tracking database yields productive results. It's double as important to have a good summary and bug description as these can mean the difference between a quick patch and a protracted series of fix attempts! 3nodding


1. Summary - A catchy title is important for any book in order to attract readers. For similar reasons a unique and detailed bug summary (or bug name) is necessary with each new report. A good summary keeps things ordered, helps make your bug easily identifiable and gives a new reader a quick idea of what the problem concerns.
   
2. Task Type - As may be expected, the "Bug Report" task type constitutes the vast majority of records in our bug tracking database. The other available Task Type is "Enhancement" and is used whenever we'd like to submit a new idea or an improvement to an existing feature for considering.
   
3. Category - This field helps the Project Manager assign the bug to the right person by indicating what part of the game is most affected by the problem. The example of "Backend / Core" would be used for issues dealing with the game sever, game stability and performance.
   
4. Operating System - The vast majority of our testing is performed on Windows XP. A smaller subset of testing is also done on machines running Windows Vista and OS X.
   
5. Severity - In some sense a bug report is an alarm bell designed to alert others that a defect has been found in our software. Severity is the field that allows us to indicate how hard we're ringing this proverbial bell by describing how big of an impact we believe the bug has on a specific feature or the game in general. Below I will list the different severity levels and the meaning behind each one.
   
   • neutral Low: The majority of the bugs we find are filed with Low severity. These bugs fail to prevent players from using any game features, but they detract from the fun or gameplay experience.
     
   • stare Medium: A smaller portion of our bugs falls under Medium severity. This level means that a bug is problematic enough to break a feature, however a user would be able to continue or work around the bug by doing something (ex: refreshing the game).
     
   • eek High: A couple of bugs in each build cycle may fall under High severity. These issues hang out in the rough parts of town and their presence prevents the player from using a feature, stops them from advancing in the game, or breaks a chunk of the game. Only bugs without a workaround earn this title.
     
   • wahmbulance Critical: The femme fatale. The show stopper. Mercifully rare, bugs that earn this severity level are ones that make the game unplayable, break fundamental game features or threaten the very stability of game servers.
     
6. Details - Hands down, this is the most important section of any bug report as this is the place to describe the bug in detail, provide steps for reproducing it and including any supporting information such as screenshots and error log files.

This may seem like an a baffling large list of criteria to keep in mind for each issue that's filed, however experience has demonstrated many times that each part plays an important role in the overall composition of a bug report.

I hope to see all of next week for Part 2 of "Bug Report's Anatomy" that will continue to flesh out the kind of information and actions that go into each bug report. To tie things together I will be showing samples of completed bug reports and holding a small contest where participants can win prizes by identifying all the mistakes present on a submitted bug report! biggrin


Previous topics:
zOMG! Behind the Scenes - The Great Cycle of Builds
zOMG! Behind the Scenes - Toolkit of Trade
zOMG! Behind the Scenes - Method in Madness

[Erynne]

Wow. It's far more complex than I thought. At least, for me. How do quest glitches like silent NPCs go?

[Universal Paradigms]

I thought this was for Anchor bugs. sweatdrop

[Fr33d0m4ever]

Good to know they're all organized~ Thanks for sharing~ *excuse to bump*~

[Judeau_the_Modest]

Thanks for sharing. It's always interesting to see how these things are handled in other places.

I wish bug reports were kept as simple where I work. Our forms alone have more than twice the number of fields xp

[IcyKitsu]

That makes it all clear-er-ish sweatdrop

[Ungoliath]

Critcial was Error 101, right? sweatdrop (Meh, I can't remember the name and I'm too lazy to check on my item right now)

[Z741]

FlySpray? Wow, even programming bugs go squish when they get near you xd

A nice read, as always 3nodding

[Emma de Lacroix]

I wonder what kind of bugs would be Critical. Thanks for sharing all this information with us. Always an interesting read. Looking forward Bug Report's Anatomy Part 2.

[Myrielle]

I want to make something go critical so I get to be there. Myr-bug or something.

[Guardian Kim]

thanks you for writing this; D; what happened to all the emotes kuzuan?

[gataka]

: D
Let me try! Found this a few weeks back. Intended to report it earlier. I doubt it's already in the database. Anyway:
/quote="Super Bug Report...Of Doom:"/Summary: A successful Raged Shark Attack occasionally won't wake a sleeping target
Task Type: Bug Report
Category: Gameplay?
Operating System: Windows XP
Severity: Low
Details: A successful RR3 or 4 Shark Attack on a Sleep-affected target occasionally won't break the Sleep effect. Also, no damage or knockback is dealt.
Rage is used up, Shark's icon appears in the target window.

The attack should either Miss completely or break the Sleep effect to apply Shark's.
Successfully replicated on Sand Fluffs, Anchor Bugs, and Kokeshi Dolls.

Method:

• Put target to Sleep (RR1 Duct Tape)
  
• Send RR3 or 4 Shark Attack.
  
• Pay attention to the target panel: Shark's Bleed icon should appear besides the Sleep one.
  
• Repeat untill it occurs.
  
• Screenshot

//quote/
So, A+++? : D
ninja

I would so abuse that "Enhancement" option.

[joody_abot]

3nodding

[RafiCat]

Very interesting, ty for sharing. smile