Regression — what it is, why it occurs and how to test

What is Regression in Development

Regression is a situation where functionality that worked in a previous version stops working after changes are made. The change can be anything: a bug fix, adding a new feature, refactoring, updating a library, or even a configuration change. Regression is the main enemy of stability: every change risks breaking something that has already been verified and released.

The term comes from testing: regression testing is the re-execution of existing tests after each change. If a previously passing test fails, a regression has occurred. In a broader sense, regression is not only a test failure but also any degradation in behavior noticed by the user or QA. According to Tricentis State of Testing 2023, regressions account for 35–45% of all bugs found in production.

What distinguishes a regression from an ordinary bug is the time context: a bug could have always existed, while a regression is always the result of a change. This is an important distinction because finding the cause of a regression starts with analyzing what changed between “working” and “broken.” Git bisect is the standard tool for finding the commit that caused the regression.

Types of Regressions and Examples

Local regression — a change in module A breaks functionality in the same module A. Example: a developer rewrites a sorting function and it stops correctly handling an empty array. Local regression is the easiest to detect and fix because the cause and effect are close together.

Remote regression — a change in module A breaks functionality in module B, which is not directly connected by code but is connected by data or timing. Example: changing the database schema in the “Users” module breaks a report in the “Analytics” module that uses the same table. Remote regressions are the most insidious: the developer does not suspect that their change will affect another module.

Side-effect regression — a change to a side effect (logging, caching, sending notifications) breaks expected behavior. Example: a developer adds caching to speed things up, but due to stale cache, users see outdated data. Side-effect regressions are hard to catch with automated tests because side effects are often not covered by tests.

Performance regression — code continues to work correctly functionally but is slower than before. Example: a new encryption algorithm produces the same results, but execution time has increased from 2 ms to 200 ms. Performance regressions are not caught by ordinary unit tests — benchmarks and profiling are needed.

Why Regressions Occur

The first cause is code coupling. The more modules depend on each other, the higher the probability that a change in one will cause a regression in another. Classic antipatterns: God Object (an object that does everything), Shotgun Surgery (a change in one place requires edits in a dozen places), Circular Dependency. Reducing coupling is a matter of architecture: SOLID principles, Dependency Injection, hexagonal architecture.

The second cause is lack of tests for the changed functionality. If code is not covered by tests, the developer only learns about a regression from QA or users. According to Google Testing Blog, projects with >75% test coverage have 5 times fewer regressions than projects with <25% coverage. TDD (Test-Driven Development) ensures tests are written before code, not “when there is time.”

The third cause is human factors. The developer is unaware of related functionality, does not understand all the dependencies, or is simply in a hurry. The reason is insufficient codebase knowledge sharing. Solutions: code review with developers from other modules, pair programming, architecture documentation. The project’s bus factor is inversely proportional to the number of documented architectural decisions.

Regression Testing and Its Role

Regression testing is the process of re-running existing tests after each change to verify that old functionality has not broken. It is the only way to guarantee that a new change has not disrupted existing code. Without regression testing, every release is a lottery: the developer hopes nothing broke but cannot confirm it.

Manual regression testing is the most expensive and least effective approach. As a project grows, the number of regression test scenarios grows linearly, while manual execution time grows exponentially. After 2–3 years of development, manual regression may take 2–3 weeks, making frequent releases impossible. The only solution is automation.

Automated regression testing is divided into levels according to the test pyramid:

• Unit tests — fast, isolated, covering individual functions and methods
• Integration tests — verify interaction between modules, databases, external services
• E2E tests — verify complete user scenarios through UI or API
• Snapshot tests — compare the current output of a component with a reference

According to Google Testing Blog, the optimal ratio is 70% unit tests, 20% integration tests, 10% E2E. Deviating from this proportion reduces the effectiveness of regression testing: too many E2E tests slow down the pipeline, too few unit tests leave micro-bugs undetected.

Strategies for Automating Regression Testing

The first strategy is Full Regression. All project tests are run. The most reliable but also the slowest approach. Suitable for small projects (up to 10,000 tests, run time <30 minutes). For large projects, full regression can take hours, making the CI/CD pipeline impractical.

The second strategy is Selective Regression. Only tests related to the changed code are run. A code dependency graph is used to determine relationships. Tools: Bazel (Google), Nx (JavaScript), sbt (Scala). Selective regression saves 60–80% run time but requires accurate dependency graph construction — errors lead to missed regressions.

The third strategy is Prioritized Regression. All tests are ranked by priority: critical path (most important user scenarios), high risk (code with a history of bugs), changed code (code affected by the change). The highest priority tests run first — if they pass, the developer gets fast feedback. Time-boxed run: critical tests are checked in 10 minutes, the rest run in the background.

How to Prevent Regressions in a Project

The first and most important step is a culture of writing tests. Every change should be accompanied by a test that verifies the change works and a test that verifies nothing is broken. TDD (Test-Driven Development) gives the best results: the developer first writes a failing test, then code that passes it. This guarantees that the test exists before the code.

The second step is a CI/CD pipeline with mandatory test execution. A pull request cannot be merged until all tests pass. Tests cannot be “skipped” due to urgency — urgent changes go through an accelerated but mandatory test suite. According to Google DevOps Research, teams with mandatory CI/CD have 3 times fewer regressions in production.

The third step is production monitoring. Even the best tests do not guarantee 100% protection against regressions. Observability tools (Sentry, Datadog, New Relic) should track key metrics after each deployment: error rate, latency, throughput. Automated rollback when thresholds are exceeded is a safety net if a regression does reach production.

The fourth step is code review with a regression mindset. The reviewer should ask: “What other modules could break because of this change?” It is not enough to check that the code is correct — you must check that it will not disrupt related functionality. The code review checklist should include a “regression check in related modules” item.