Regression Testing: Essential Areas to Focus on

Regression is a major topic in Agile testing. Its importance is major because in Agile we iteratively build our product, and it is essential to ensure that the addition of a feature has not negatively impacted the existing. To evaluate this impact, we have a tool: regression.
This frequent regression is the source of many changes in practices. I am thinking in particular of:

In short, there is no sustainable Agile product without a good regression campaign. But how do we ensure that this campaign is “good”? What should a regression campaign contain? To answer this, I invite you first to look at the role of a regression campaign.

A regression campaign has many roles. Its main purpose is to detect major regressions on the product under construction. It is impossible to detect all regressions (exhaustive testing is impossible). Or even to test a new feature as thoroughly as an already implemented feature for budget and time reasons.

The concept of “major regression” is therefore everything, and I propose to return to it quickly:

For a regression to be major (and here it is highly context-dependent) it must have a significant impact or occur frequently enough to cause real discomfort. It therefore seems particularly important, according to the second point, that this regression accurately represents the uses. A regression on a scenario that is never used by users and has no non-functional impact is potentially of little value. This is the subject of Bruno Legeard’s fine article on the testing of uses defined thanks to usage logs. In this article, he is using as an example Gravity which answers totally to that point of view. That leads us to this question: is a “major regression” on a path that is not used, really major?

There is no easy answer because it depends on the context, however, we can say that a regression must therefore propose frequent user paths… but also that the content of only frequent user paths is not enough! Even if these tests can/should present the great majority of the tests of a regression campaign, it is essential to go further with the regression… this is why Bruno indicates that the Gravity tool does not lighten the workload on the regression but 80%… Which is already huge because it can help to drastically reduce the cost of maintenance! Besides, it also can help to initiate a regression campaign if there is no current one or if it is deprecated. You can create a free Gravity account to try by yourself :

The remaining 20% of the effort is dedicated to what is often referred to as “non-passing” cases or non-functional tests, such as performance and security. Indeed, security tests are necessary to avoid intrusion, data theft, or identity theft… but attacks are not as frequent as the flow of ordinary users. Similarly, a drop in performance will not change the results of the usability tests, but it will greatly deteriorate the experience of our users. A regression campaign must therefore propose complementary tests to the uses and these tests must be in line with the different risks identified… these risks are frequently linked to non-functional issues.

The regression campaign is a major element of quality during the development of a product in Agile. It is therefore necessary to have a “good” regression campaign to have, over time, a quality product. To achieve this, it is necessary to have tests that are close to what is used to avoid the illusion of no errors, but unfortunately, this is not enough; it is also necessary to integrate more specific tests that are often linked to non-functional issues (security, performance, robustness, etc.)

Find this article in French in the blog of “La Taverne du Testeur“

Author: Marc Hage Chahine