There are many views on what constitutes a Vulnerability Assessment versus a Penetration Test. The main distinction, however, seems to be that some believe a thorough Penetration Test involves identifying as many vulnerabilities as possible, while others feel that Penetration Tests are goal-oriented and are mostly unconcerned with what other vulnerabilities may exist.
I am in the latter group, and what follows is my argument for why you should be too.
language matters
Language is important, and we have two terms for a reason. We already have an (aptly named I might add) security test for compiling a complete list of vulnerabilities, i.e. a Vulnerability Assessment. If there isn’t a clear, communicable distinction between this test type and a penetration test then we shouldn’t be using separate terms. Such a distinction does exist, however, and it’s a crucial one.
clarified definitions
Vulnerability Assessments are designed to yield a prioritized list of vulnerabilities and are generally for clients who already understand they are not where they want to be in terms of security. The customer already knows they have issues and simply need help identifying and prioritizing them.
The more issues identified the better, so naturally a white box approach should be embraced when possible. The deliverable for the assessment is, most importantly, a prioritized list of discovered vulnerabilities (and often how to remediate).
Penetration Tests are designed to achieve a specific, attacker-simulated goal and should be requested by customers who are already at their desired security posture. A typical goal could be to access the contents of the prized customer database on the internal network, or to modify a record in an HR system.
The deliverable for a penetration test is a report of how security was breached in order to reach the agreed-upon goal (and often how to remediate).
a physical analog
A good analog for this is a Tiger Team working for the government, like richard marcinko used to run with Red Cell. Think about what his missions were: things like gain control of a nuclear submarine and bring it out into the bay.
So imagine that he’s getting debriefed after a successful mission where he broke in through the east fence, and someone were to ask him about the security of the western side of the building. The answer would be simple:
We didn’t even go to the west side. We saw an opening on the east-facing fence and we went after our target.
If the person doing the debrief were to respond with, “You didn’t check the other fences? What kind of security test is it where you didn’t even check all the fences?”, the answer would be equally direct:
Listen, man, I could have come in a million ways. I could have burrowed under the fences altogether, parachuted in, got in the back of a truck coming in–whatever. You told me to steal your sub, and that’s what I did. If you wanted a list of all the different ways your security sucks, you should have hired an auditor–not a SEAL team.
the question of exploitation
Another mistake people make when discussing vulnerability assessments vs. penetration tests is to pivot immediately to exploitation. The basic narrative is:
Finding vulnerabilities is a vulnerability assessment, and exploiting them is a penetration test.
This is incorrect.
Exploitation can be imagined as a sliding bar between none and full, which can be leveraged in both vulnerability assessments and penetration tests. Although most serious penetration tests lean heavily towards showing rather than telling (i.e. heavy on the exploitation side), it’s also the case that you can often show that a vulnerability is real without full exploitation.
A penetration testing team may be able to simply take pictures standing next to the open safe, or to show they have full access to a database, etc., without actually taking the complete set of actions that a criminal could. And vulnerability assessments can slide along this scale as well for any subset of the list of issues discovered.
This could be time consuming, but exploitation doesn’t, by definition, move you out of the realm of vulnerability assessment. The only key attributes of a VA vs. PT are list-orientation vs. goal-orientation, and the question of exploitation is simply not part of that calculation.
the notion that penetration tests include vulnerability assessments
It’s also inaccurate to say that penetration tests always include a vulnerability assessment. Recall that penetration tests are goal-based, meaning that if you achieve your goal then you are successful. So, you likely perform something like a vulnerability assessment to find a good vuln to attack during a pentest, but you could just as easily find a vuln within 20 minutes that gets you to your goal.
It is accurate to say, in other words, that penetration tests rely on finding a one or more vulnerabilities to take advantage of, and that people often use some sort of process to systematically discover vulns for that purpose, but because they stop when they have what they need, and don’t give the customer a complete and prioritized list of vulnerabilities, they didn’t actually do a vulnerability assessment.
