Nobody likes them, but they are part of software development – bugs and errors. Is this really necessary? Are the developers sleeping? And what about the quality assurance? At least they should have found the bug before delivery to the customer! These are possible thoughts after a bug was found.
As it is often the case, things are not so easy as they seem to be. There are different parties involved, lots of reasons for bugs and also lots of activities being necessary and possible in this area.
It is of course annoying for a user if he wants to do something with an application and if this does not work as desired. It is also annoying for our quality assurance team, as they ask themselves why they did not find the bug. And the same is also valid for the developers.
If a user comes across a bug we first have to find the reason and fix the bug. After this is done, special cases will be discussed in our qualty assurance review meeting to find out whether we have to adapt our processes in order to make the security net even more stable.
Detectives in QA
Some bugs are obvious, other ones really require detectives at work. Chunks of information have to be combined and sorted, details have to be collected to ease work for development and give them as many hints as possible. They have a lot of work to do in order to find the place in the code where the error happens and to develop a better solution.
Can we reproduce the bug?
If our QA team finds a bug during a full test following a test protocol, we know what they did before the error occured. If the error is discovered by chance, the first thing to do is to reconstruct which steps triggered the error. Then the question is whether it only happened once or whether the error can happens a second or third time following the same procedure. Does it happen with another document, element or other data as well? Trying different proceedings, collecting facts and documenting them has to be done as described in the following paragraphs.
Every bug reported by the customer hurts the QA soul. A portion of skepticism eases the pain, because if the error is not reproducible in our environment, we did not have a chance of discovering it. This is a consolation for QA, but makes work more difficult later on. Are there differences between the customer’s environment and ours? Are there different ways to reach the goal, and do we choose one way (perhaps out of habit) while the customer chooses another way, which we don’t even consider?
Sometimes we discover that the error was due to quantity problems. For example, deleting records can fail in a database being filled with several thousands of records. In this case we are not facing a software bug, as it works properly with limited amounts of data. The problem is located in another area.
In these cases we urgently need the customer’s co-operation, a detailed description of what he did, examples, export packages or logs in order to get helpful hints.
Browsers are different
Firefox, Edge und Chrome: The list of supported browsers is manageable, especially since Edge and Chrome have the same base and can be considered to be equal. Nevertheless, there are always surprises when different browsers behave differently in some cases. So, if a reported bug is not reproducible, using a different browser can sometimes give a different picture.
In this case using another browser facilitates reproducibility. When testing individual tickets or in a full test using at least two different browsers is mandatory for QA. Seriously performed, the effort for each test doubles – you never know…
Standard user or administrator
The testing effort doubled by the browsers can be further increased if the test is carried out with different user roles. Apart from the possibilities or limitations combined with a user role, an application can do different things when the user is logged in with or without admin rights.
If the time for testing is limited we use at least cross-combinations and do a test in Firefox as adminstrator and another one in Edge as standard user for example.
Errors following merges
A bugfix or a new feature will be provided by the developers after their testing in the so-called Main QS environment for testing by the QA team. When this test is done – with or without additional circles – the last step is a merge which brings the changes to the environment which will be delivered to the customers. All changes – code bits which were added, changed or removed – have to done in the target environment, too. This may be an easy task which takes an hour or even less. But it may also be a tedious error-prone task .
That is why a serious test with different roles in different browsers is necessary after all merges and prior to delivery.
Errors due to side effects
Building up at the front, clearing the back – this statement is known for the proverbial bull in the china shop, and it also applies to software development. This means that a change in one section of the code may have an effect on different areas of the application. One error is fixed, but a new one occurs. If the whole thing takes place in the same area of the application, e.g. in the same screen, the developer or at the latest the QA should notice it during their tests. However, the further the location of the side effect is away from the location of the original bug, the more difficult it becomes to find the side effect. It is impossible to run a full test after each new deployment as this takes several person weeks. So sometimes the only hope is the intuition of the QA team which directs their mouses and eyes to a certain point in the application for whatever reason and possibly make them discover something important.
Side effects can also happen after the upgrade of libraries in order to get rid of CVEs. The greater the difference between the current and the new version, the higher is the probability for side effects. We are eager to do updates on a regular base in order to minimize this risk.
What to and not to expect from logfiles
Log files contain a lot of information and can have some MBs in size. Nevertheless they do not contain something like “The error occurs in function XY in line 33 due to this cause…” Of course, they offer useful hints, but the developer has to find them first and then interpret them correctly.
It is very useful if we know the time when the error occurred. Then search is limited to a bunch of lines which have to hold the clue.
All's well that ends well
No matter how long the path is and how tedious it is to get along – it will end on the finish line. Once we got there, we must not take a rest as lots of new challenges are waiting for developers and quality assurance team members. We are always eager to find bugs as early as possible and to eliminate them straight away.
