Kuzuan
(?)Community Member
- Posted: Sat, 24 Jul 2010 07:14:43 +0000
Reporting for Duty
One of the major themes that has been carried across the previous entries for Behind the Scenes is the high priority we set for organization and good records keeping. The overall QA process is a much more straightforward affair for the zOMG! team because we have always been a tight-knit group. However, if you imagine that some development teams in other companies may contain hundreds of members (with some members working from different buildings or event countries), then you can see how even small problems can quickly grow in magnitude. Because of this we use a number of different reports as another layer to protect against human error or other mistakes. Each of these reports serves a unique purpose and they all come together to help us bring you a better zOMG! experience.
While QA keeps up with new development as it progresses, this is often our first chance to see the complete list of changes. Because of this, all of us review and discuss the design notes in order to write down any questions we have regarding how new features should function or what the details of a change are. These questions are put together into one report that is then sent to the programmers for additional clarification. Turn around for answers is quick and when we receive the desired clarifications we are better able to fine-tune our testing strategies.
Pre-release reports are created from the feedback on new builds that we ask our users testers to check out. This feedback serves as a sort of mine canary and allows us to judge whether we're on the right track for the wider audience.
Often we see many requests to delay fixes for bugs presented on this list. This is understandable since, by this point, everyone wants to push the new build and move onto the next step. We work closely with the programmers to try and work out a compromise on as many issues as possible in order to prevent outstanding fixes from snowballing from build to build and to improve the polish of the new build. Ultimately the reality of the given situation trumps all and we have to prioritize issues and create plans so that we can both meet our deadlines and release a quality product.
Acceptance Tests are very importance because if they pass then we have a green-light for starting to test the build further and if they fail a new build must often be assembled before we can continue testing. Because of this a report is put together and sent out to the entire zOMG! team whenever an Acceptance Test is completed. This helps keep everyone in the loop and updates them on whether testing has commenced on the build. In itself the report includes the completed file that lists all the areas we check and notes on that particular acceptance test. These notes can discussion the difficulties that occurred during the test and what problems were found.
I hope you've enjoyed this new addition to our Behind the Scenes series! We have come a long way since our beginning all these weeks ago and have covered all the major aspects of how we test zOMG!. Please post any questions you may have about this or any previous section and we'll be happy to answer them. Thank you!
Previous topics:
zOMG! Behind the Scenes - The Great Cycle of Builds
zOMG! Behind the Scenes - Toolkit of Trade
zOMG! Behind the Scenes - Method in Madness
zOMG! Behind the Scenes - Bug Report's Anatomy, Pt. 1/2
zOMG! Behind the Scenes - Bug Report's Anatomy, Pt. 2/2
One of the major themes that has been carried across the previous entries for Behind the Scenes is the high priority we set for organization and good records keeping. The overall QA process is a much more straightforward affair for the zOMG! team because we have always been a tight-knit group. However, if you imagine that some development teams in other companies may contain hundreds of members (with some members working from different buildings or event countries), then you can see how even small problems can quickly grow in magnitude. Because of this we use a number of different reports as another layer to protect against human error or other mistakes. Each of these reports serves a unique purpose and they all come together to help us bring you a better zOMG! experience.
- The Q List
While QA keeps up with new development as it progresses, this is often our first chance to see the complete list of changes. Because of this, all of us review and discuss the design notes in order to write down any questions we have regarding how new features should function or what the details of a change are. These questions are put together into one report that is then sent to the programmers for additional clarification. Turn around for answers is quick and when we receive the desired clarifications we are better able to fine-tune our testing strategies.
- Daily Bug Report
- Feedback Report
Pre-release reports are created from the feedback on new builds that we ask our users testers to check out. This feedback serves as a sort of mine canary and allows us to judge whether we're on the right track for the wider audience.
- "Must Fix" List
Often we see many requests to delay fixes for bugs presented on this list. This is understandable since, by this point, everyone wants to push the new build and move onto the next step. We work closely with the programmers to try and work out a compromise on as many issues as possible in order to prevent outstanding fixes from snowballing from build to build and to improve the polish of the new build. Ultimately the reality of the given situation trumps all and we have to prioritize issues and create plans so that we can both meet our deadlines and release a quality product.
- Acceptance Test Report
Acceptance Tests are very importance because if they pass then we have a green-light for starting to test the build further and if they fail a new build must often be assembled before we can continue testing. Because of this a report is put together and sent out to the entire zOMG! team whenever an Acceptance Test is completed. This helps keep everyone in the loop and updates them on whether testing has commenced on the build. In itself the report includes the completed file that lists all the areas we check and notes on that particular acceptance test. These notes can discussion the difficulties that occurred during the test and what problems were found.
I hope you've enjoyed this new addition to our Behind the Scenes series! We have come a long way since our beginning all these weeks ago and have covered all the major aspects of how we test zOMG!. Please post any questions you may have about this or any previous section and we'll be happy to answer them. Thank you!
Previous topics:
zOMG! Behind the Scenes - The Great Cycle of Builds
zOMG! Behind the Scenes - Toolkit of Trade
zOMG! Behind the Scenes - Method in Madness
zOMG! Behind the Scenes - Bug Report's Anatomy, Pt. 1/2
zOMG! Behind the Scenes - Bug Report's Anatomy, Pt. 2/2