Non-functional tests: what defines whether the system is good in addition to working

A system can pass all functional tests and still be a failure.

It does exactly what the specification asked for. Every button works, every calculation is correct, every flow completes itself. And even so, it's slow to the point of irritation, it crashes when a lot of people use it, it leaks data on the first attack attempt and it's so tangled that no one can change it without breaking three other things. It works, and it's bad.

This difference between "does what it's supposed to" and "does it well enough to be actually used" is exactly the territory of non-functional testing. They are, in my view, the part of quality that most distinguishes professional products from disguised prototypes.

Functional versus non-functional

The distinction is simple to state and profound in its consequences.

Functional tests check what the system does: given an input, does it produce the correct output? Does the registration save? Is the payment processed? Does the report provide the right numbers? These are questions about behavior and correction.

Non-functional tests check how the system does: how fast, how secure, how reliable, how scalable, how easy to use and maintain. They don't ask "does it work?", but rather "does it work well?". Speed, safety, stability under load, accessibility, these are all non-functional.

The trap is that functional is easy to specify and charge for, so it gets almost all the attention. The non-functional is diffuse, easy to postpone, and therefore is usually neglected, until it becomes the reason why the system fails in production.

The dimensions that matter

"Non-functional" is a large umbrella. It is worth knowing the main dimensions, because each one is a different way for the system to disappoint even when it is working.

Performance is the most visible: response times, speed under use. A correct but slow system silently loses users. Scalability and capacity respond to whether it can handle growth and spikes in demand, rather than only working when few people use it.

Reliability and availability are about continuity: does the system stand up over time, recover from failures, not corrupt data? Security checks whether it resists malicious use, and this is where the most expensive defect category lives, because a security flaw doesn't frustrate the user, it exposes him.

Usability and accessibility ask whether real people, with different capabilities and contexts, can actually use it. And maintainability, the most invisible of all, asks whether the team can evolve the system without each change becoming a minefield.

Why non-functional is deferred (and why it's a bug)

There is a structural reason why the non-functional is neglected: it does not appear in the demonstration.

When you show the system to a client or manager, you show functionality. "Look, do this, do that." Nobody demonstrates "look how it can withstand ten thousand users" or "look how it resists an attack". These qualities are invisible until the day they are missing, and then they appear as a crisis.

The result is a false economy. The load, security, accessibility test is postponed, because "first it needs to work". Then the system goes into production, crashes at the first peak, leaks data or excludes part of the users, and the cost of correcting late is multiple times higher than it would have been to test in time.

In the public sector this is especially serious. A citizen service system that works but is slow, insecure or inaccessible does not serve those who need it most. And when dealing with personal data, the non-functional security flaw comes directly up against LGPD, it stops being a technical problem and becomes a legal problem.

Testing the non-functional requires a different mindset

Testing functionality is relatively straightforward: define inputs, check outputs. Testing the non-functional is more difficult because it requires defining what is "good enough", and that is a decision, not a fact.

How fast is fast enough? How many simultaneous users can it "handle"? What level of availability is acceptable? These questions have no universal answer; depend on the context of the business and users. The first job of non-functional testing is, therefore, to transform vague qualities into measurable goals. "Fast" is not tested; "responding below such time for 95% of cases" is tested.

The practical consequence is that non-functional requirements need to be defined early, along with functional ones. Treating them as an afterthought is guaranteeing that they will be tested too late or never.

The mistakes that make the effort useless

The first mistake is not setting goals. Without a target, "testing performance" becomes collecting numbers without knowing whether they are good or bad. The goal has to come before the test.

The second is to test in an unrealistic environment. Measuring performance on a machine other than production, or security without realistic attack scenarios, generates misleading results. The non-functional is especially sensitive to the environment, the numbers are only valid if the conditions resemble reality.

The third is to treat it as a single event. Performance, security and reliability degrade over time, with each change to the system. A security test done once at launch says little about security six months and fifty releases later. The non-functional needs recurrence.

Quality is what remains when functionality is taken for granted

There is a moment in the maturity of a product, and of an organization, when "it works" stops being an achievement and becomes a starting point. From then on, real quality is defined by non-functional: it's fast, it's safe, it holds up, it stands up, it's accessible, it's possible to evolve.

Startup teams celebrate “it works.” Mature teams know that this is just the floor. What distinguishes a professional system from an amateur one is rarely what it does, it's how well it does it, under pressure, over time, for everyone. This "how well" is precisely what non-functional tests protect.

Ignoring them doesn't save effort. It just transfers the cost to the worst possible moment: production, with the user present.

If your product "works" but you've never measured how fast, safe or resilient it is, there's probably a non-functional risk waiting to emerge. I have other articles on the blog about performance, security, reliability and software quality that address each of these dimensions.