For those needing a “too long, didn’t read” (TL:DR) version, the answer is: “Make your testers feel crushingly undervalued, prevent them from collaborating with others, and don’t leave enough time in the SDLC to find or correct the bugs they find.
Hopefully, you’re doing none of the above.
Jason Schreier, news editor for the popular video gaming blog, Kotaku, recently wrote an article, “Quality Assured: What It’s Really Like To Test Games For A Living”, where he interviewed around sixty QA current and former testers, many of whom shared horror stories of their struggles to assure the quality of the software they were being paid to test. The good news is that so many of the hardships these testers have faced are preventable with the correct culture and available technologies in place.
I’ve picked out five of the article’s most egregious abuses to self worth and software quality, and I’ve included some suggestions for eliminating a passive quality culture that cannot continue.
“Game testers are, by nature, underappreciated”
Sadly, I’ve interviewed numerous software testers outside of the video game industry with their own stories of feeling underappreciated, and I’m baffled every time I hear them. There are a number of reasons that testers have been made to feel this way, from having to constantly explain the differences between manual testing and automation tools, to being given so little time to actually do their jobs and then being blamed when bugs make it into production.
“The more bugs a tester finds, the higher his or her perceived value to the company.”
It’s really the “perceived” in the statement above that completely takes this assumption off the rails. Without perceived, that statement would just be factually incorrect. But with it, the statement begins to bring the flawed culture of testers vs. tools to light.
An automated tool may find more bugs than a human software tester, but are those bugs “showstoppers” to the software’s release? If a human software tester unearths one massive security flaw that would’ve exposed highly sensitive data—and a tool, or even another tester finds 10 minor bugs that can be quickly fixed and do not require delaying a future release—which find is more valuable?
Of course, if you’re hung up on “more bugs = more value” you could always give testers more time to test so they can find them, though I’ve read numerous stories where this ends up pitting testers against each other, rather than giving them incentives or bounties that can be won by working together.
“When QA speak, we are heard… there’s just no time to react.”
This heartbreaking statement came from a former tester, who, after reading his/her description of testing’s tiny place in the SDLC, you can’t blame them for leaving the testing field. Unfortunately their story is all too common, and not only in the video game industry.
What [a] lot of people probably don’t understand is that even for titles that have been in development for, say, three years, only 1.5 of those years is actually full, proper production. And of that, only nine months in QA. And of that, only three months at full QA capacity. By then, we might be at content lock. And so by the time we are able to speak out, the game is reaching Beta.
Three months of full QA capacity inside of three years of development! Is there any wonder when Schreier points out, “…today’s games seem to be shipping with more problems than ever before”?
I realize that in video game testing in particular, the product needs to be at some level of playable for it to be truly testable, but leaving three months for testing for what as Schreier labels as video games comprised of “complicated sets of interlocking systems” is simply not enough time.
In order to reduce the risk of shipping poor quality, especially when dealing with something as complex as enterprise software, testing must not only begin earlier in the SDLC, it needs to continue throughout it. (Our partners at Parasoft literally wrote the book on the benefits of continuous testing–I highly recommend it!)
“…The second, significantly harder process is trying to reproduce glitches so the company’s engineers can zap them.”
This is another huge problem that plagues teams across the globe and it results in far more than frustration and strife between developers and testers. In today’s market, testers not only need the time to find bugs and other issues, they need the ability to isolate, clone, and easily share their exact defect state, so that developers can quickly eliminate them—not shrug their shoulders and say, “it works on my machine.”
“Bugs that caused inconvenience for the player were often considered invalid because they wouldn’t affect the ability to release the game, and could be addressed later if people got upset.”
Knowingly releasing documented bugs into production is tough to justify, but when video game release dates fall during the holiday season I can see where that’s a tough call to make. However, knowingly releasing bugs that are sure to “inconvenience” your customers, and planning on fixing them “later if people got upset” is…almost enough to give me a migraine.
In this day and age, when consumers will jump at the chance to delete and replace a mobile app or other piece of software, and then take to social media to shout scathing reviews after even the slightest inconvenience you’ve caused them, treating quality as less important as speed is a huge mistake. They’re both immensely important, and one should never come at the expense of the other.
Cheers to Jason Schreier for exposing the serious quality issues above, and the many more I didn’t list. I wholeheartedly agree with him that “a widespread push for change could lead not just to healthier work environments but to more experienced, efficient testers who work directly with game developers to ensure that more bugs get fixed.” This change is already resulting in happier developers and testers releasing higher quality software faster in other industries; let’s hope it’s takes root in video games as well.
How do Skytap’s Environments as a Service help enterprise dev/test teams improve the quality and time-to-market of complex applications? Watch the 3-minute video below!
