Krzysztof Kozłowski
QA Expert
Have you ever been in a situation when a mobile app, website or other software you were using malfunctioned? If you live in a modern environment and are not completely cut off from technology (which is a safe bet, considering that you are most likely reading this article online and not a printout version delivered to you by post) I'm almost certain that you know what I'm on about.
Bug-free software is highly important for everyone who pays for its development and expects this investment to bring profit in the future. If you are in this group then this article should interest you.
Faults, bugs and failures in software do happen. From crashes to typos, logic flaws to insufferable graphics. If the software in question is popular enough or absolutely essential for them, users usually soldier through when faced with such a problem, find a way around it (so-called “workaround”) or wait for repairs. But when a completely new tool is buggy and unreliable users may simply abandon it.
As you can see it is crucial that when working with a software development team you test the results of their work.
Reliable developers (like ours) do test their own code, and experienced software houses (like ours :)) employ manual testers who check how digital products perform from user perspective but it is the person ordering and managing the software (be it Product Owner, Co-Founder, Project Manager etc.) who knows best what to expect from the product and how should it work. If you are a customer of a software house or oversee a development team you probably perform some form of tests anyway by simply using the software in its current stage and checking if you are satisfied with it. If not, you are brave enough to leave the tests to your users, but I wouldn't recommend it.
So, if you indeed are inspecting your software as it is created, why don't I help you learn a few tricks that will help you improve your product development process?
You'll learn how to communicate problems and issues in a fast, efficient way, which will improve the whole development process, reduce costs and risks, ensure high product quality, increase confidence in your app, as well as help with further product development process. All this by performing simple tests — no one expects you to learn ISTQB syllabi by heart (you're welcome to do it, if you wish).
It's important to remember that apart from rare unpleasant instances software development teams take pride in their work (we certainly do) and want to deliver the best software possible. But, just like in the case of proof-reading and text editing, different set of eyes is crucial for completing the whole product. Testing is an essential part of product development.
In this article we assume that you have your product properly designed. If you don't know how to get around to it check out our blog article on Product Design . Having a product designed will help properly arrange your requirements (remember that your development team should receive all information possible about what do you expect).
After you receive a functional version of your product (we employ a methodology that provides you with such a version after every completed development phase) you may begin testing it. It's best to start testing as early as possible. Why? The Systems Sciences Institute at IBM has reported that “ the cost of fixing defects increases exponentially as software progresses through the development lifecycle, it’s critical to catch defects as early as possible. The costs of discovering defects after release are significant: up to 30 times more than if you catch them in the design and architectural phase. ” According to them a bug costs $100 to fix in Gathering Requirements phase, $1,500 in QA testing phase, and $10,000 once in Production. Google provides us with another estimate that proves roughly the same principle:
As you can see it will be cheaper to fix problems at an early stage, you'll deal with the most important issues first and prevent future defects more effectively.
Don't know what to do? Just navigate the user interface as you expect your users would. Remember to demand only what should be completed at this particular phase of the project and make sure to test your product using a version that has all the expected features implemented.
Remember to meticulously note every step you take in the interface. For example, when you are testing a simple “log in with correct login and password” function, write down:
You don't have time for this? It took me less than 50 seconds to write it down and I was trying to be particularly sluggish. Remember that when you will report a bug it will concern a very particular situation, so it's good to divide what you do in small segments like the one above. If you don’t have much time to spend at the moment you can focus on the so-called “red routes” which are the critical and frequent paths that users take to complete their tasks. More on red routes here . If you expect a more thorough examination you should probably hire software testers or find a software house that provides such services.
Preparing a test scenario will help when you'll find a bug and will want to report it but, more importantly, it will also help you save time and money. How? Let's say that you follow the above steps and the app crashes. You could contact the development team and say “The app crashes. Fix it”. During the bug fixing process programmer will have to try to crash the app in whatever way they can think of to track down the defect. It will take a lot of time if the crash occurs only in a specific situation. They will probably ask you how it happened and if you will not be able to answer right away they may have to wait for your answer. What's more, you'll have to explain when the bug did appear. Trust me, if you are not in the same room it will take several minutes at minimum and pull both of you away from your duties.
Even in such a simple scenario it will take much longer than writing the steps down and then sending them to the development team with an additional note “and then the app crashed”. After receiving such message programmer can get right to bug fixing e.g. if you made a password typo and it was the cause, it won't be difficult to recreate it by simply entering non-existing password in the proper field. Then it's only a matter of fixing the bug.
The example above was too simple? Well, as I stated, you could probably easily divide everything your software does into segments not much more complicated than logging in with correct login data, especially when considering the end user perspective. It will take not that much longer to write down your steps when using more complicated functionalities but without them communicating the problem will be significantly harder, especially in the case of software-specific features. Then the additional time for communication and bug tracking may generate severe costs. Therefore I advise you to always take time for keeping track of what exactly did you do in the interface. If you are still not convinced, remember that many of those “test scenarios” will have to be created only once when you decide on how your product is to work, modified as the software evolves and re-used constantly. Bearing that in mind, explore away.
NOTE: It's not unlikely that you don't want to spend your resources on creating user steps. We don't judge, everything I wrote about the benefits of preparing test scenarios is true but you don't have to do it yourself. A competent software house will gladly prepare a set of test scenarios or test cases and update them as the project progresses.
How is the testing going? Everything is perfect? Great, it's not impossible to find nothing during tests. It doesn't mean that there isn't something out there but if tests were rigorous then you can at least have confidence in your product. But always remember to test everything again after every modification (so-called “regression testing”). Testing new additions to your products is probably fairly obvious but problems may appear in places that were bug-free before.
If something isn't working as expected you'll want to report it to the developers. I gave you an instruction on how to write down steps to recreate what occurred but sometimes it may not be enough. If the issue is platform or device-specific you may not realise it but finding that out is also an unnecessary waste of developers' time and, more importantly, a waste of your money.
Our Analyst, Łukasz and I have created a template that will help you prepare an accurate bug report. Copy the template .
Developer comes back to you with a fix? Great! Now it's time to re-test the issue. Simply perform the same actions as before and see if the problem persists. Again, remember to make sure that you are using the new, fixed version. If you are very busy or the repair work took long it's possible that you forgot how did you trigger the bug, good thing you got those steps written down, eh? And again, it wouldn't hurt to check if the fix didn't cause some new issues, so perform regression tests as well.
Looks troublesome and time-consuming? Filling in everything to report the bug from my “crash” example should take you no more than 4 minutes. Even less since you have your “steps to reproduce” already prepared (by yourself or your software house) and your speed will improve over time. It will really be worth the trouble, whether you work with your own team or a hired software house.
Does testing result in bug or no-bug situations only? Of course not, by using your own software you'll get to think about it in a different context than design panel or a cluttered whiteboard. It will be your chance to reflect upon its reliability, usability, efficiency, maintainability etc. You are bound to come up with ideas and improve your product and you'll be able to do it during its development, when it's much cheaper to include modifications.
Happy testing!
P.Wasilewski - Head of Solutions
J.Dogadalski - Business Development Lead
J.Dogadalski - Business Development Lead
T.Jurek - Head of People Ops
