Penetration testing - a useful definition

For the purposes of this article, we will define penetration testing as:

"A method for gaining assurance in the security of an IT system or the physical components of your operation by attempting to breach some or all of that system's security.

It uses of a variety of manual and automated techniques to simulate an attack on an organisation’s information security arrangements – either from malicious outsiders or by your own staff."

Why is it used?

One of the roles of management is to manage cyber security risk(s) by mitigating the vulnerabilities (weaknesses inherent in the system which could be exploited) and threats through both technological and organisational measures.

Penetration testing provides a level of assurance that the system functions as intended, within an acceptable level of risk and budget for the organisation.

What are the benefits?

• Provides an objective and independent view of how secure your systems are.
• Helps identify vulnerabilities and security issues which may exist in your systems to date.
• Helps your internal teams think about the security implications of their actions e.g. vulnerabilities in developed software may be negatively impact the users of your systems.
• Helps to identify real world scenarios by which your systems could be breached by an determined attacker.
•  Can form part of a broader approach to gaining assurance that your systems function as intended and remain secure.
  

What are the limitations?

Many organisations rely upon penetration testing as the sole mechanism to identify potential vulnerabilities which may exist in their environment. 

Penetration testing should not be seen as the only activity conducted to validate that a system works as intended but should form part of an organisations cyber security assurance approach. 

Some limitations to consider:

• Testing engagements are typically bound by time which means that there is a limit to what an experienced tester can validate and test.  Real world attackers are not time bound and can continue to probe over a prolonged period of time to find and exploit vulnerabilities.
• Testing engagements will have a defined scope e.g. certain elements of a system may not be validated due to their complexity or their impact on live services.  This could reduce the level of assurance that you can gain from a penetration test.  It is recommended that you track limitations in penetration testing scope and define future tests to 'plug these gaps'.
• You are reliant on the skills and experience of the tester to ensure vulnerabilities are identified.  If you use specific technologies, software or hardware, it is recommended that the testers experience is validated.
• Penetration testers don't have a full understanding of your control environment.  That means when reports are produced findings may often be risk rated significantly higher than you may expect.  You therefore need to be able to translate these findings into language / risk ratings which are more aligned to your organisations policies and practices. 
  

What are the different types of test?

1. Black Box Testing: During this type of test, the testers receive no information about the workings of the system being tested.  The purpose of this type of test is to emulate an external attacker who may be motivated to gain access to your internal systems.  Whilst black box testing simulates the approach taken by a real world attacker, it may also limit what the tester could identify meaning some vulnerabilities may not be identified during the test.
2. Grey Box Testing:  Utilises a combination of black and white box testing scenarios whereby the tester is provided with some (but limited) insight into the inner workings of the system being tested.
3. White Box Testing: A variety of information is shared with the tester to ensure that they have a comprehensive view of the system before testing commences.

Test Type

Whilst we haven't included an exhaustive list of tests types, they can be typically broken down into the following:

1. Scenario driven testing aimed at identifying weaknesses in systems:  The purpose of this type of test is for the tester to work within a specific scenario to determine if vulnerabilities can be identified within the 'in scope' target systems.  Such tests may include:
   1. Web application test: Focus on web facing systems / applications used by an organisation, its users and end customers.  Such systems may hold sensitive customer information such as payment card information or medical records.
   2. Infrastructure test: Focus on infrastructure (internal and externally facing) which underpins the businesses core systems and services.
2. Scenario driven testing aimed at assessing the adequacy of your detection and response controls: The purpose of this type of test is to simulate potential attacks and validate whether an organisation can detect and respond to a potential incident on a timely basis.
   

How do I scope a test?

1. Engage with key internal stakeholders:  It is important that the right team is assembled to support the test, from key business stakeholders, technical teams and security team members.  This ensures that all team members understand their role and how the test will be used to improve the organisations security controls. 
2. Identify the business objective and purpose of the test:  identify what assurance you are looking to gain from the test and how this supports and underpins your organisations business objectives.  Doing so allows you to position technical findings which may arise at the end of a test, in a language which is consistent with your broader organisational goals and risk management framework.
3.  Define the scope and boundaries of the test: Consider the test basis and test types (as outlined above) and align this to the business objectives for your test.
4. Determine and agree any legal, regulatory or legislative requirements which must be met by the test: It is important that such requirements placed upon you or the penetration testing firm are clearly identified.
5. Engage with a experienced penetration testing firm: Utilise their experience to help scope and shape the test so that it delivers the best outcomes for your organisation.  For example, has the test been scoped in the best way?  Could other tests be combined or run in parallel to give you greatest insight on your operation and the different elements which contribute to its effective running?  
6. Define clear rules of engagement: Once you have formally engaged a penetration testing firm, it is important that you agree a clear set of principles by which both organisations with adhere to e.g. timing of testing during the day / out of hours, escalation procedures.

In addition to the five steps outlined above, we recommend the organisations look to establish an "Assurance Map" which positions how this test provides assurance over key risks identified across an organisations operations. 

What should I expect during and after a test?

1. Communication, communication, communication:  This point should never be overlooked.  Given the nature and role of a penetration tester in your organisation, it is extremely important that you are in regular dialogue with them.  For example, if you detect some strange activity on your systems - is that being caused by the tester or are you under real attack?
2. Early sight of findings: It is important that you receive any findings during the test period rather than having to wait until a formal report is produced.  This ensures that you can evaluate the impact of findings quickly but also take action where it is required.
3. A comprehensive and actionable report: Reports should be written in a way which is comprehensive both for technical and non technical teams so that the impact of findings are clear.  Reports should specify clear actions to ensure that there is no ambiguity on the steps which need to be taken to resolve identified issues.
   

What should I do with the results of the test?

As outlined above, penetration testing is a form of assurance.  It is therefore important that the results of penetration tests are used to drive continuous improvement across the organisation where issues are identified. 

Whilst much of the test report content may be sensitive, it is important that the learning and  the results of such tests are made available (and appropriately summarised/redacted) to key stakeholders within your security, technical, operations and senior leadership teams.  

We have outlined a number of steps which we recommend organisations follow on receipt of the penetration test findings:

1. Keep a central log of all penetration test results: Ensure that you track key elements such as date of test, severity and risk rating of findings, recommendations, remediation owner, target and actual remediation date, retest date and results.  Use this information to hold teams to account for closure and remediation of findings.
2. Translate technical findings into the commercial realities of your business: Testers don't have a full understanding of your environment e.g. controls which may mitigate the risks they have identified.  Scoring systems such as the Common Vulnerability Scoring System (CVSS) are often used to report on vulnerabilities however they do need to be translated into a language which is more relevant to your organisation.
3. Drive ownership and accountability through remediation activity: Penetration tests more often that not, find the symptom not the root cause of vulnerabilities.  For example, poor coding practises and a lack rigorous testing techniques in the software development life cycle can introduce a significant number of bugs into production systems.  If recurring items are identified through penetration tests, use the results to drive improvements in the underlying controls.
4. Don't just focus on the most material issues identified from penetration tests: Whilst it is positive that organisations focus on addressing high(er) risk findings on penetration test reports, time and effort should be spent reviewing and addressing those findings deemed to be less important e.g. 'low risk' findings. 
   
   Why?  Overtime such findings build up and can introduce vulnerabilities which are greater than the sum of their parts.  In addition, systems become more costly to maintain as they have large volumes of bugs.   
5. Challenge the 'status quo':  Use the outputs from testing to inform changes in testing in future cycles e.g. to perform deeper, more insightful testing which helps to validate how secure and resilient an organisation really is.