[continued from Quality: The Process, Part III]

Greek to me

Every program with more than seven lines of source code has bugs. It is important that software developers do whatever is feasible to eliminate bugs. With mass market software, one can be confident that even rare bugs, when multiplied by thousands of users, will be discovered. Bugs in some specialized and vertical market software could actually cause damage or injury. In any case, when distributing shareware, bugs will cost you sales, so quality will directly help your bottom line.

The most innovative approach to elimination of bugs, which I must credit to Barry James Folsom, involved a simple corporate proclamation. As the new President, he called for a meeting and all of the several dozen developers in the company were gathered. After an introduction, he declared that none of our software would have “bugs”. From that point forward, it could only have “defects”.

It may not be terribly practical to simply redefine terms and create quality, but this dubious proclamation did have a point. When a customer or, in the case of shareware, a potential customer is using the software and it fails to work properly, that is a problem. “All software has bugs,” is not comforting, so we need to look at the software the perspective of a user.

Let’s start at the very beginning, with alpha, or more specifically, “alpha testing”.

Alpha testing is a form of black box testing that is performed in-house. In practical terms, alpha testing is simply the developer using the software in the same way that a customer would, prior to making the software available to others.

After each version of the software is ready, I close all my development tools, clear the registry and data files, and pretend to be a user seeing the program for the very first time. I start by running the program installer, and then launching the game (in our case) using the installed shortcut, as opposed to the debugger. I will then just play the game for a while, recording any problems that arise.

Once I am comfortable that the program is working as intended on my development system, I then copy the installer to at least one other test system. Rather than install the software myself, though, I enlist somebody else to do it. This can be a colleague, friend, spouse, child, parent, pet, or benevolent stranger. I provide no other instruction, and note where any questions are asked. Any problems witnessed here will also be experienced by users on a larger scale.

In a formal testing environment, alpha testing involves testers systematically checking the software according to the specified test plan, combined with actual use of the software. In a corporate environment, the test plan is executed by the QA department. In small businesses, it generally falls on the programmers to follow the test plan. In either case, anybody willing should try using the software. In a larger company, I would throw an “open house” to show the software to other employees. As an independent, simply having the game available for play is sufficient.

Alpha testing should begin as soon as the software is usable, and this will necessarily overlap with program development. At some point during the alpha phase, the software should become “feature complete”. This means that all intended features for this version are in the program and functional. It does not mean that the performance is optimized, nor does it mean that the interface is finalized, but it should do everything that it was intended to do.

[continued in Beta move on]