Jakes rules for working with QA

I thought I should do a games based update, since I haven’t done one in a while.

I was talking the other day about QA people and working with QA departments and what I had experienced in the past. So here’s some thoughts on that.

QA is necessary. There are those who decree that they write code that doesn’t have bugs in it in the first place because of units tests and so on, and sure, their degree of QA requirement is lessened. But it’s important to understand that QA tests a game, not individual pieces of code. One piece of code may well do what it’s supposed to do, but what it does and how the rest of the game operates aren’t always (and often aren’t) the same.

Make QA part of the team. They just need to be, particularly when they are located in a different location from you. When I was at Raven I made a point of making sure QA where brought out to our location in Madison so we could actually put faces to the people who would be sending us bugs. Knowing who someone was could make a huge difference to how you treat a bug – before that there was way too much sarcasm going backwards and forwards.

Educate your QA people in how to write up a bug. One thing I found particularly effective was in giving them a second method of being able to get to you via a second bug list – only this wasn’t a bug list but a suggestions list. How many times have you seen a bug from QA that wasn’t really a bug, but a design suggestion? “You shouldn’t be able to jump as high as you can” etc. The idea to educate QA in whats a bug and what’s not, but to still give them the capability to make suggestions of this nature. QA plays your game over and over – they may well have suggestions worth taking on board. Giving them a method to be able to do that is a massive incentive for them to play the game looking for how to make it better than just finding the bugs.

Plus, the education in how to fill out a bug form can be the difference between finding a repro case or having enough information to find the bug and not.

Work with QA, not against them. Every time I had a bug and didn’t know what to do with it, and I contacted the QA people directly I found them to be more than willing to do the extra work to work with you to track it down. These people want to be part of the team, so the more you try and make them that, the more work and the more plugged in they will be.

If you can build an auto test rig to help out QA, it helps tremendously. Getting a bug report with the test that’s required to make it happen attached to the bug report, that runs automatically, is massive. 90% of the hard bugs from QA are ones that are ‘random’ and have no easily reproduceable situations. Lessening those makes your load much less.

There’s quite a lot of stuff you can do with an auto test rig, but thats a post for another day.

Include QA in bug triaging. They’ve been working on them, they’ve been seeing what crops up a lot – these people need to be included.

If you can, have an internal QA team that works directly with the development team – I mean literally right there, in cubes along with the rest of the guys. At EA we had a couple of the savviest QA guys as part of the team from pre-alpha onwards. They had Dev Studio on their machines and played what was there constantly, looking for lock ups or crashes. When they found one, they went to find the person who’s code it was they were looking at and brought them to see it crashed under the debugger. Mucho useful.

QA is a massively important part of a project, but often get treated as disposable add on’s and thats simply not the case. Much like Marketing, a successful partnership with this group can seriously increase the quality of what you put out there. It only takes one bug to stop people enjoying your game.

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>