Are vulnerability assessments giving your organisation a false sense of security? Ian Otieno, Cyber Security Account Director at The Missing Link discusses misconceptions about vulnerability scanning, and why there’s a lot to consider when establishing the appropriate layers of protection within public sector constraints.
What is penetration testing?
According to the NCSC, Penetration Testing is: "A method for gaining assurance in the security of an IT system by attempting to breach some or all of that system's security, using the same tools and techniques as an adversary might." Thorough penetration testing services will accurately simulate attacks against your systems and applications to identify and then help you close any security gaps.
So, what should a successful penetration test look like?
I’m here to help you explore this question in a little more detail. A successful penetration test must include several elements:
- It’s performed using both manual and automated technologies.
- It tests for several types of vulnerabilities, for example devices which haven’t been patched, flaws in applications, and even end-user weaknesses.
- It will explore whether the discovered vulnerabilities can be exploited further, such as by elevating privileges or deepening access.
- It will take ~4 hours to assess an internal network of about 10-15 IP addresses.
In the end, you should receive a report detailing any vulnerabilities in your environment, prioritised according to how risky they are. Your provider should share recommendations and a process for remediating these vulnerabilities.
The great penetration test lie
Security consultancy is cutthroat. Companies are always offering to provide the same penetration test cheaper and quicker. A poll of session attendees at tri-WARP found that less than two-thirds of the audience felt they were getting genuine business value from them, even though companies are trying to make penetration testing cheaper and quicker.
But, with the context provided above, how comprehensive can an internal penetration test on a network with 250+ live IPs be when it’s conducted in an 8-hour window? It doesn’t add up, and what appears as a sensitive consideration of public sector constraints is often masking a potentially dangerous, lower-quality service. I like to call it “vulnerability scanning as a service.”
Vulnerability scanning as a service
To cater to time and budget constraints, we are seeing consultancies resort to automation to carry out most of their penetration testing.
There are three main problems with this approach.
- Relying on automated tools removes human, out-of-the-box thinking and creativity. Being hands-on with the keyboard is how new vulnerabilities are discovered.
- Vulnerability scanning is reactive, relying on a library of issues that have been provided to whatever solution/s are used. That means providers are relying on pre-determined rules rather than human creativity.
- We’re not using the same tools an adversary – the people who might try and compromise your network – might. Therefore, you’re not getting an accurate assessment of your security posture.
Automated scanning does have a role when you’re conducting a security assessment – it covers patching levels. However, it should be delivered alongside a manual penetration test, conducting a true-to-life assessment that uses actual techniques being used by threat actors. We can push the boundaries of what is possible using emerging technologies and working to clear goals of engagement so we can help better secure the public sector.
Where some penetration testing providers fall short
Knowing about the move toward vulnerability scanning as a service, how did we get here in the first place? I’ve observed a few trends fuelling the fire:
- Consultancies are pushing day counts (and by extension price) down to win the bid. This isn’t a good thing. Quality is what matters, and a shorter timeframe means experts don’t have time to conduct in-depth manual testing.
- We’re seeing an increased reliance on the quick easy wins, with consultancies filling their reports with “bloat finds” that don’t have any significant business impact, for example, SSL misconfigurations on an internal network.
- Not only is the quality of the actual penetration testing dropping but so is the documentation of business impact and security reports. Some consultancies are no longer sharing constructive remediation paths.
Strategies to improve the quality of your penetration testing
Don’t let yourselves down! Call out bad penetration test reports and ask whether the results have been discovered manually or through automated scanning tools.
Keep an eye out for penetration tests that fail to remediate old, but extremely common, issues. Critical oversights like missing security patches because software is due to be decommissioned shortly can lead to costly compromises.
Understand what a good penetration test looks like, and why it’s worth paying more. This can help ensure the penetration testing you choose for your organisation is successful and fulfilling. There’s a lot to consider when selecting a penetration testing provider but paying a little more can help you avoid the dreaded (and potentially more costly) vulnerability scanning as a service.
How can we all improve?
We can work together to improve the security of our nation. It’s on us, as consultancies, to provide a better service. As public sector security professionals, you’re responsible for scrutinising the services provided to you, and knowing what you’re getting for your money.
The NCSC has done a great job of setting standards, promoting more secure policies, and raising awareness across all sectors. But, as technologies change and APT groups continuously advance, we need to constantly improve.
The industry can only change to better service public sector organisations by breaking the mold of what is defined as the requirement. I have a few ideas about how to make this a reality, for example:
- Certifications that reflect emerging technologies like cloud deployments, in-depth mobile app testing and IoT testing; and,
- An agreed standard day count to be enforced by the NCSC. This could look like a minimum number of hours or days by IP, not allowing known vulnerable systems to be excluded from scopes, random audits of penetration testing methodologies or enforced upgrades before servers become end of life.
Collaboration is key. Let’s work together to make better, higher-quality penetration testing reality and abolish vulnerability scanning as a service forever.
