Quality assurance and/or testing should be an integral part of any software engineer’s process. Well, for that matter it should be part of any person’s process who produces anything to distribute. Whether you are a one man shop, a group of five or an empire of thousands the process should be there and actually is pretty much the same in all scenarios. It’s only a difference of scale.
Now I know we’ve all had those bosses that tell you all you need to do is code, test, code, test… run, build, run, build. As long as you do that everything should turn out alright, right? Wrong! Here is a major flaw in that theory. You are the designer of the system or the thing that you are producing and thus you have an unfortunate bent towards the way it should work. You will view things the way they should be viewed and do things in the order they should be done in no matter how hard you try to throw things into the mix. Believe me I’ve tried to fight against this creator effect but there is no way around it.
So what do I do if I’m a one man shop? Well, even for small stuff (like this blog) I can always get my wife to look over things for me. Trust me she is one of the perfect QA testers on the planet. Why? Because she is not a geek like me. She doesn’t know all the keyboard shortcuts and how to create macros or any other time-saving nerdy stuff like that. She is a real person, which is exactly the type of person that I want to reach! She can break a web application quicker than you can spell polymorphism! Also change perspectives when you are working on something yourself. When I’m working on a blog entry I get used to reading it in the editor view. I find that when I’m ready to publish it I’ll do one final preview in the actual skin of the sight and will usually find a few more mistakes probably because it just looks different. Another great thing to do is to get your friends and family members to look at stuff and try it out. Basically the more eyes and hands you can get on it the better your feedback will be and the more chances you will have at finding the bugs in your system or the mistakes in your blog or the flaws in whatever you are producing.
So what exactly is QA? It really boils down to statistics. Awe, Jonathan! You didn’t have to bring up Math did you? Yeah, yeah… I know it’s painful for me too. Trust me I won’t actually do any math. It’s way too late for that! Hang with me though. With any system, especially computers and software you basically cannot guarantee that there will be no flaws. That is an absolute absurdity to expect 100% quality. So our goal is to get as close as we possibly can within some acceptable parameters. How can we do this? We will perform many tests on our system and systematically remove flaws thus approaching our quality goal. Now to do this work you need people to test the system and you need them to perform tests on the system over time. To get as close to our quality goal as we can we need to perform as many tests as we can on the system to ensure we discover and eliminate as many flaws from the system as we can. Keep in mind there is a finite number of people we have to work with and a finite amount of time we have to work with. You could have a few people test the system for an extremely long time and get the same results as a very large number of people testing the system for a smaller length of time (theoretically).
Now if there is one thing I now about software (and blogging) it is that time is not something that is in abundance. Scratch that one out. That means you need people/testers. You need a lot of testers. As many as you can get. For smaller companies consider forming relationships with some of your long standing customers and creating a beta group for this purpose. One other idea that I’ve read about on Joel Spolsky’s blog that I like is called hall testing. Setup your system in a hallway and let people passing by stop and take it for a test drive at their leisure. There should of course be a way of logging feedback and errors. Then there is always the free software option… give it away! Then you will be able to get plenty of users. Of course you may have to change your business model around a bit, but such is life. 🙂
So with all of this testing going on, did you notice the mention of systematically remove flaws from the system? How do you think you should go about removing flaws from the system? In the order they are found? Should you remove all flaws from the system before delivery? Can you tell the difference between a flaw in the system and a training issue? Can you convince your boss that a flaw in the system is really a training issue? (hee hee)