Software Testing & Testineering: It’s Not Just an Act, it’s a Habit
I test, therefore I exist.
Testing is learning by experimenting, a fundamental learning process ingrained in human nature. We test even when we don’t realise it. When we decide which route to take to work, what to do on vacations, how to spend our time, we test our options first.
Newborn children understand the world by experimenting. They play around and run only to learn the meaning of falling and getting up.
Our testing abilities and how much we can learn are usually aligned.
This has to do with our cognitive ability to understand. Our curiosity, our attention to detail, and our ability to focus on the important and discard the unimportant. Our ability to improvise, make careful decisions, and be creative.
We may have many ideas about how to solve a problem, each leading to a different solution.
How do we pick the right one? We test our ideas, at least in our mind. If that’s not our first step then god help us!
Blindly implementing solutions without much thought is not a good idea. We may end up with frustration and disappointment. Testing ideas and concepts is a basic first step for successful implementation.
Until we interact with our system or product and test it, what we think we know about it is mere speculation. We have to observe our product’s actual behaviour and compare that to our expectations.
Table of Contents
A Day at Work
A day at work for a software engineer involves test engineering (testineering), either directly or indirectly.
A software engineer will write code and debug. Above all, code writing involves decision-making. We think and test before we make decisions.
This could be testing at any level (e.g., unit, integration, system), based on the results from test-cases at different abstraction levels and different levels of detail.
Different development tasks may call for different tools, different test strategies (e.g., proactive or reactive) and different test-case suites.
But at the end of the day, all software engineers have the same goal: Build something that works as expected while learning as we build and improve.
According to one school of thought, debugging and coding are the same. Coding is the process of gradually debugging a program until it does what you want.
This implicitly suggests how to write code: Start with a working program and make small modifications, debugging them as you develop.
Linux is the classic example of a project that started small and ended massive. A small working program by Linus Torvalds (a program that would simply switch between printing AAAA and BBBB) ended up with building an operating system of millions of lines of code.
Most importantly, how much you want to test as you develop your code is a crucial decision.
For instance, debugging may not be enough and further testing may be appropriate. Maybe unit/integration tests are also necessary. Or maybe it also makes sense to test at an API/UI level as some of your development tasks are about to be completed. Maybe you need performance testing as well, to guide development decisions.
Another school of thought uses tests to guide the entire SDLC. Test-driven design, test-driven development etc.
This is different from debugging and testing as you develop in that you first write your tests and then your code. When is coding finished? When your unit-tests pass.
Although it can be frustrating, debugging is intellectually rich and challenging. It involves inspecting and inferring. There are events that lead to the results you see. By studying clues you have to come up with the processes that explain the sequence of those events and results.
Like testing, debugging is experimental in nature. Once you have an idea about what is going wrong, you modify your program and try again.
If your hypothesis was correct, you can predict the result of the modification. You can make a step closer to a working program. If your hypothesis was wrong, you have to find a new one.
This is Sherlock Holmes stuff! According to one of Sherlock’s famous quotes: “When you have eliminated the impossible, whatever remains, however improbable, must be the truth.” This summarises elegantly the nuts and bolts of debugging.
Software development’s state of the art affects software testing too. Twenty-five years ago, for example, the answer to the question “Did you test your code” could be something like “Yes, my code compiled”.
While testing is not just compiling, compiling at that time was certainly a basic way of testing (if not the only way in some cases). Nowadays, however, the very idea of compiling could be meaningless.
QA Software Testing
Software testing also involves testing your code at a QA stage, that is, when development has finished.
You want to be confident that new features and code changes do not break existing features. Did they negatively affect your product in any way?
Regression testing, smoke testing, UI test automation and exploratory testing are some of the most common testing types at this stage. People who don’t write the code under test usually perform such tasks.
Why? The answer is simple: Bias. Is the writer of a novel the best person to review it and find errors? No. The writer is biased with her work. The best person to review it is an experienced reviewer, not herself.
She could review it too and correct important errors but it’s unlikely that she will find the errors spotted by an independent reviewer. Similarly, developers don’t usually do QA testing. This is the task of QA testers and/or QA engineers.
As with other stages in the SDLC, diversity really pays off during QA testing. Using more engineers to spot errors may minimise the bugs that fall under our radar. This will make our customers happier.
In conclusion, we may mix different schools of thought according to our abilities and needs.
How we test, when we test and what we test is not carved in stone. It evolves as companies evolve. It may differ considerably from one software company to another.
There exist guidelines and best practices, however, that may guide towards sensible decisions.