Writing secure software is not trivial. Modern software stacks and systems have many parts, abstraction layers, and complexities that make it difficult to avoid all bugs, including security bugs creating vulnerabilities.
The UK NCSC has done a meta analysis that shows that the number of lines of code per software increases — anyone who’s looked at the size of a JavaScript’s node_modules directory can easily attest to that — and the same is true for the number of defects per line of code. So developers have their work cut out for them.
Vulnerabilities are not all born equal
Some are complex edge cases that are difficult to identify (and, hopefully, exploit), and others are trivial bugs that reflect poor security practices. And it’s this second category that UK NCSC’s proposal is trying to shine some light on.
We know many vulnerabilities are complex and difficult to avoid. But vulnerabilities that are trivial to find (and that occur time and time again) are ones the NCSC are aiming to drive down at scale. These ‘unforgivable vulnerabilities', a phrase coined by Steve Christie in his 2007 MITRE paper, ‘are beacons of a systematic disregard for secure development practices. They simply should not appear in software that has been designed, developed, and tested with security in mind’.
→ Steve Christie in his 2007 MITRE paper
The report summarises three categories of vulnerabilities:
A vulnerability that should not exist in software, as the difficulty of implementing mitigations is deemed negligible.
This could be because:
- the mitigation is fully documented
- it is cheap to implement
- the technical implementation of the mitigation does not rely on too many (or too complex) prerequisites
Note: There may be multiple causes of a single vulnerability, with each cause having a different mitigation.
A vulnerability that exists in software because implementing known mitigations is deemed hard.
This could be because:
- the vulnerability is subtle
- there is lack of knowledge about it
- the cost to implement the mitigation is too high
- the technical implementation of the mitigation relies on too many (or too complex) prerequisites
There is a third class of vulnerability, which we have called Non-exploitable. We don’t discuss this class further in this paper, but arguably this is a vulnerability that is not a security risk.
This could be because:
- there is no code path to be able to exploit it, or
- there are mitigations in place to prevent the vulnerability being exploited, or
- it is unlikely that chaining vulnerabilities will allow exploitation
Ease of implementation
The report defined an “ease of implementation” score, based on:
| Category | Description | Score | ||
|---|---|---|---|---|
| Cost | including direct costs (such as licensing a library) and indirect costs (such as additional time needed to code/refactor). | 1️⃣ cheap | 2️⃣ some costs | 3️⃣ expensive |
| Knowledge | how widely known and understood is the mitigation. | 1️⃣ well known | 2️⃣ intermediate | 3️⃣ obscure |
| Feasibility | a feasibility analysis evaluates the mitigation’s potential for success, and how easy the mitigation is to achieve. This includes evaluating the advantages and disadvantages of the mitigation, and any technical requirements or prerequisites for the mitigation (such as hardware support). | 1️⃣ easy | 2️⃣ possible | 3️⃣ difficult |
The “implementation score” is the sum of the three sub-scores above:
Implementation Score = Cost score + Knowledge Score + Feasibility Score
| Implementation Score | Difficulty | |
|---|---|---|
| 3-4 | Easy | Likely an unforgivable vulnerability 🔥 |
| 5-6 | Medium | Possibly and unforgivable vulnerability |
| 7-9 | Hard |
Here is an example for Input Validation.
| Score | Evidence | |
|---|---|---|
| Cost | 1 | Low/no direct and indirect costs |
| Knowledge | 1 | Widely understood and widely available |
| Feasibility | 1 | Many implementations for most/all languages |
| Total | 3 (Easy) | → unforgivable vulnerability 🔥 |
So a vulnerability caused by a lack of Input Validation would likely be deemed an unforgivable vulnerability.
Further reading
UK NCSC’s proposal goes more in depths with example for all the CWE Top 25 Most Dangerous Software Weaknesses of 2023, and rate them using their scoring system:
| Top-level mitigation | Implementation Score | Difficulty |
|---|---|---|
| Input Validation | 3 | Easy |
| Output Encoding | 3 | Easy |
| Reduce the Attack Surface | 5 | Medium |
| Enforcement by Conversion | 5 | Medium |
| Sandbox or Jail | 5 | Medium |
| Secure Programming | 6 | Medium |
| Compilation or Build Hardening | 6 | Medium |
| Separation of Privilege | 6 | Medium |
| Libraries or Frameworks | 6 | Medium |
| Secure Architecture and Design | 7 | Hard |
| Language Selection | 8 | Hard |
All the details on how and why they arrived at these numbers are in the report. It’s absolutely worth a read.