Security Assessment and Testing

1.1 Assessment, audit, testing: three words, three realities

Assessment vocabulary is confusing because different professions use close terms. Three notions must be crisp.

An assessment is a broad evaluation of controls against management expectations. Its scope, schedule and reporting are set by management; it often precedes an audit to spot and fix weaknesses.

An audit is a structured review of information, observations and data to measure compliance with a defined standard. The evidence examined is called artifacts (log, policy, interview result). A formal audit requires independent evaluators, outside the management chain responsible for the system: this independence ensures no undue influence skews judgment.

Testing compares the actual behaviour of a system, process or activity, under defined conditions, with its expected behaviour. Testing is a building block that can serve an assessment or an audit, but can also stand alone as security monitoring. Key point: expected performance must be defined BEFORE testing, otherwise you cannot conclude.

Finally, an evaluation is formal (against a legal, regulatory or contractual standard, by independents) or informal (to gather observations and insight, not directly for compliance). Informal often gives management a preview of what a formal audit would reveal.

1.2 Anatomy of a finding and testing perspectives

An audit result is delivered as findings, compiled into a report given to the chartering organisation. A typical finding has five elements: Condition (what the audit observed), Criteria (the measuring standard), Cause (why the gap exists), Effect (the gap and its significance) and Recommendation (the corrective action). A finding's accuracy depends on artifact integrity: they must reflect the true state of the control against its objective.

Testing follows an internal or external perspective. Internal testing assumes the identity of a trusted insider or an attacker who has already crossed the perimeter (see NIST SP 800-115); it covers application configuration, authentication, access control and hardening, and may include privilege escalation. Insider threats remain a major source of compromise. External testing takes the view of an untrusted individual outside the perimeter, to find weaknesses exploitable from outside.

Practical rule: if both external AND internal are planned, run external first, because the information available to internal testing is generally richer. The internal/external distinction blurs with zero trust and deperimeterization, which apply controls based only on the privileges needed for a task.

1.3 Assessment standards and frameworks

Standards and frameworks form a virtuous circle with laws and market expectations: all incentivise good practice. Several anchors structure security evaluation.

NIST Risk Management Framework (RMF, SP 800-37 Rev. 2) is the reference for US federal agencies and many of their contractors. Two useful companions: SP 800-53 Rev. 5 (security and privacy controls) and SP 800-171 (protecting controlled unclassified information at non-federal entities).

NIST Cybersecurity Framework (CSF) offers a voluntary, risk-driven maturity model, simpler and incremental; it emphasises detecting an incident. ISO 27000 serves as an audit framework to evaluate an ISMS and is widely adopted outside the United States.

Finally, the AICPA control-evaluation framework, known through its SOC reports, evaluates an organisation against the Trust Services Criteria. Beyond that, many organisations fall under several sector frameworks at once: PCI DSS (cards), IEC 62443 (industrial control), Cyber Essentials (UK), NERC CIP (US critical infrastructure), CSA STAR (cloud), SWIFT CSCF (finance). The UK NCSC also proposes 12 supply chain security principles (understand the risks, establish control, check, continuous improvement).

1.4 SOC reports: 1, 2, 3 and Cybersecurity

The SOC (Service Organization Control) framework, defined by SSAE 18 (partly updated by SSAE 20), lets a provider attest to its controls for its clients. Four reports, not to be confused.

SOC 1 attests to internal controls over financial reporting (ICFR). It cannot cover disaster recovery or privacy.

SOC 2 (also called the Trust Services Criteria report) evaluates security controls against five criteria: Security, Availability, Processing Integrity, Confidentiality, Privacy (memo CIANA+P). It is confidential and may cover only a subset of the criteria depending on the provider's needs.

SOC 3 gives a public, less technical summary of a SOC 2: a releasable showcase, often on the provider's website.

SOC for Cybersecurity specifically targets the cybersecurity programme (plans, processes, services) based on the 2017 TSC.

Two types apply to SOC 1 and SOC 2. Type I evaluates the design (suitability of design) of controls at a point in time: an expression of due care. Type II additionally evaluates the operating effectiveness of controls over a period (usually 6 to 12 months): the expression of due diligence. SOC 2 Type II is generally preferred, and it is the report to request from a third-party provider integral to operations. SOC 3 and SOC for Cybersecurity are single-type.

1.5 Trust Services Criteria and attestation standards

The Trust Services Criteria (TSC) are the criteria an auditor uses to evaluate the suitability of design and operating effectiveness of controls relevant to security, availability, processing integrity, confidentiality and privacy. Developed by AICPA and CICA, they were realigned in 2017 with the 17 principles of the COSO framework. Security is the mandatory common criterion; the others are added per the provider's needs.

On attestation standards, AICPA issues the SSAEs, compliance standards that certified auditors apply when an organisation engages them to attest to its controls. The old SAS 70 (Type I and Type II) was replaced by SSAE 16 then SSAE 18, drawing on the international ISAE 3402. An SSAE 18 / ISAE 3402 report typically has five sections: the auditor's opinion, the organisation's written attestation, the description of controls and objectives, the tests of operating effectiveness, and additional information.

Lesson of SAS 70: it was designed for ICFR, but many repurposed it for security, availability and privacy needs. SOC reports now clarify usage. Finally, evidence collection (field work) can happen on site or remotely, on on-premises, cloud or hybrid assets; the cloud shared responsibility model helps determine who owns which control.

2.1 Three methods: testing, examination, interview

Evaluating a control is done by any combination of testing, examination and interview; modelling, simulation and analysis may be added. Each method measures a system's compliance with its requirements and specifications. Proper method selection drives the level of assurance.

Examination means reviewing, inspecting, observing or analysing an assessment object (specifications, mechanisms, activities). An administrative control like a policy is checked by reading the document; a business process is observed (running backups, wearing a badge). Examination does not require the assessor to direct the operation or interact with stakeholders. Reviewing real performance documentation (access registers, transactions, logs) is one of the most valuable tools.

Interview identifies the frequent gap between the documented process and actual practice. Interview subjects are defined during chartering and named in the pre-audit checklist; depth and breadth must be clarified. Interview documentation (questionnaires, recordings with consent, notes) must be secured per its classification.

Testing compares actual behaviour to expected behaviour under defined conditions. Once confined to development, it now serves throughout a system's operational life. A complete control may require several methods: a mere examination is not always enough, and additional tests or interviews may be needed. The organisation must establish the assessor's need to know, without granting unlimited out-of-scope access.

2.2 Scoping and sampling the evaluation

Assessment activities are planned by the security assessment policy, which sets the rules for planning, conduct and reporting. Management provides high-level direction: identify how the context has changed since the last assessment, along three key elements - the threat landscape, business logic and compliance regime. This yields a focused, down-selected set of assessments to run (tailoring).

The assessment is then tailored to the control assessed, by its type, intended depth and the specifics of the compliance requirement. Since you can rarely examine everything, you rely on a sample. Statistical sampling uses the laws of probability to infer a control's performance over the whole population. Judgmental sampling (also purposive or authoritative) relies on the assessor's expertise to choose systems or events, prioritising those that present the greatest risk.

The organisation cannot ignore its confidentiality, integrity and availability obligations during the assessment: convenient as unlimited assessor access may be, that privilege may exceed scope.

2.3 Log reviews

All major frameworks stress logging practice. ISO 27001:2013 control 12.4.1 requires event logs (user activities, exceptions, faults, security events) to be produced, kept and regularly reviewed. NIST SP 800-92 (Guide to Computer Security Log Management) treats log review as a component of log management. Review serves not only security assessment, but also the identification of incidents, policy violations, fraud and operational problems near the time of occurrence. Reviewing historic audit logs can determine whether an identified vulnerability has already been exploited.

Several regulations mandate diligent reviews: GLBA (protecting financial institutions' customer data), HIPAA (regular audit log reviews, 6-year retention), SOX (regular log review to spot violations), PCI DSS (track all access to network resources and cardholder data).

Log management follows four key practices: prioritise log management across the organisation; establish policies and procedures; create and maintain a secure log management infrastructure (a SIEM helps with storage and analysis, able to absorb peaks during an incident, pentest or scan); provide adequate support (training, tools) to staff. Log security (ISO 12.4.2) protects against tampering and unauthorised access; logs can contain sensitive data (passwords, email content) and must be protected for confidentiality and integrity, with compliant retention. Rootkits often aim to alter logs to erase their traces.

2.4 Compliance test vs substantive test

Tests fall into two families. The compliance test determines, in the assessor's opinion, whether the control exists and operates correctly. Example: compare a sample of employees entitled to access a system with the list of named users; the expected result is that the two lists match.

The substantive test evaluates the proper operation of the process itself. In the same example, the tester performs an onboarding to grant access to a new user, attempts an authorised transaction with those credentials, then offboards the identity; they then review the logs to verify all transactions executed and were tracked correctly. The substantive test gives a higher degree of assurance that the process behaves as expected, but takes more time and resources.

Distinguish this from compliance testing in the broad sense (also called conformity testing), which validates that a system has been designed and configured to meet the organisation's security requirements: that is the right answer when the question is about validating against defined standards. Note finally: the substantive test is also the technique by which an auditor obtains evidence to support their opinion.

3.1 Vulnerability assessment and CVSS

Vulnerability assessment is an essential task in evaluating a system's security. A vulnerability is a weakness that, if exploited, could compromise the confidentiality, integrity or availability of an asset. Caution: the existence of a weakness does not mean it is exploitable by a threat. A vulnerability is not an exploit.

Vulnerability scanners automate the identification of known vulnerabilities in configurations, settings and software. They can be authenticated (with credentials, a more precise inside view) or unauthenticated (an external attacker's view). They also detect the use of insecure protocols like FTP - even if compensating controls (network segmentation, usage restriction) can mitigate the risk. Evaluating identified vulnerabilities means analysing their impact given the organisation's unique configurations and circumstances, and handling false positives (erroneous alerts).

To prioritise, use CVSS (Common Vulnerability Scoring System), which scores technical severity from 0 to 10: None (0), Low (0.1-3.9), Medium (4.0-6.9), High (7.0-8.9), Critical (9.0-10). CVSS measures intrinsic severity; it pairs usefully with EPSS, which estimates the probability of exploitation. Typical SLA prioritisation reserves the shortest deadlines for Critical and High vulnerabilities.

3.2 Ethical penetration testing: RoE and scope

Penetration testing simulates a threat actor's actions using information the actor would have. It is a controlled process, governed by Rules of Engagement (RoE) detailing the circumstances of the attempt. Useful to determine whether controls are effective, it poses significant risk if poorly conducted. RoE define scope, methods, constraints, the individuals authorising the test, incident reporting procedures and liability limits.

Key legal point: without a signed contract, the tester faces criminal and civil charges in many jurisdictions, and can be held liable for disruptions caused. Hence the term ethical penetration testing: the test is tied to a contractual, legal and ethical basis. Sophisticated attackers use the same techniques, but their intent makes their activity unethical and probably illegal. The code of ethics requires the tester not to disclose their activities and findings without the sponsor's written consent, on top of contractual NDAs.

Scope depends on objectives. PCI DSS, for instance, may require covering the cardholder data environment and demand both network AND application testing. Other scenarios include a wireless component, social engineering or physical intrusion. The test may be blind (the tester is blind to technical details) or double-blind (the defending team - the blue team - is also not warned), which exercises detection and response but requires deconfliction if the defenders discover the test.

3.3 Box testing and the five-phase methodology

The level of knowledge granted to the tester defines the box testing type. White-box (or crystal-box) gives full knowledge of the system (IP addresses, versions): maximum coverage, mimics an insider threat, fastest and often cheapest, but controls that hide information from outsiders (like NAT) are not assessed. Black-box grants no knowledge: the tester runs reconnaissance, which confirms controls meant to conceal information, at the risk of missing assets. Gray-box blends the two: partial information, the tester discovers the rest. Reminder: a real attacker is not constrained by time or billing, unlike the tester whose contract lasts a few weeks.

The methodology follows five phases. 1) Chartering: drafting RoE, formal risk assessment, integration into the contract if third party. 2) Discovery: reconnaissance to map the environment (sometimes unnecessary if the RoE already define the systems). 3) Scanning: identifying and fingerprinting weaknesses to craft exploits. 4) Exploitation: delivering the exploit and documenting; when withdrawing, the tester ensures they do not increase the risk of exploitation by a malicious actor. 5) Reporting: compiling activities, executive summary, recommendations, returning all material (including credentials obtained). There is no single standard: the exact sequence must be documented in the RoE.

3.4 Red, blue, purple team and ethical hacking

Depending on objectives (e.g. response readiness), several teams take part. The red team simulates a malicious attacker to assess security posture: its goal is to identify vulnerabilities, exploit weaknesses and gauge the effectiveness of existing measures. The blue team represents the defenders (internal security team): it monitors, detects and responds to the simulated attacks, evaluating defensive strategies, incident response capabilities and resilience. The purple team combines both: it fosters collaboration and knowledge sharing between attackers and defenders, to close defensive gaps and recommend improvements.

Ethical penetration testing should not be confused with ethical hacking, which gathers the efforts of amateur or professional researchers to probe and detect vulnerabilities. Bug bounty programs encourage these informal attempts: hacking is ethical there as long as the researcher respects the program's terms and does not disclose results to third parties. Google Project Zero is an emblematic example. The legal context matters: in the US, DMCA section 1201 makes circumventing digital protections an offence, limiting independent evaluation of cryptographic protections.

3.5 Continuous testing: BAS, ATT&CK, chaos engineering

A penetration test is a point-in-time activity: it reflects one set of tests at a moment, while the environment and threats evolve fast. Several continuous testing approaches complement the frameworks that expect continuous monitoring.

MITRE ATT&CK, a knowledge base of adversary tactics and techniques, helps test resilience by emulating attacker behaviour. Breach and attack simulation (BAS) tools automate these tests to run continuously against the infrastructure; coupled with threat intelligence and integrated with change management, they offer a different level of assurance than traditional pentesting. BAS combines vulnerability scanning and automated penetration testing, with a constantly updated attack database. It enables more frequent testing than quarterly scans or annual pentests: if a simulated attack is blocked, controls work; if it succeeds, it signals a deficient control or a new risk.

Chaos engineering forces production systems to fail to evaluate detection, response and recovery capabilities. Its no-notice tests heavily stress people and systems: teams are not warned and treat the outage as a real attack. Handle with care: many systems are not engineered to withstand a production shutdown without cost, and the approach does not simulate a true attacker (disruptions are scripted by insiders).

4.1 Code review: objectives and SAST / DAST

Code review, more precisely source code analysis and review, aims to verify six things: all required functions exist; no foreign code is present; no dead or unreachable code remains; no backdoors or trapdoors; coding standards are met; all code is of trustworthy provenance. These objectives are not for development only: they assume rigorous configuration management, without which dead code can be reactivated and made exploitable by a small change elsewhere.

Testing can be manual (code peer review: a developer reviews a peer's work) or automated. Two families dominate. SAST (static application security testing) analyses code AT REST, without executing it: it models execution to detect buffer overflow, inconsistent data conditions, misuse of templates, outdated libraries, misconfigurations. It is a white-box approach, ideal for shift-left. DAST (dynamic analysis) tests the program WHILE RUNNING to observe actual behaviour: a black-box approach. A variant exists on compiled binaries (static binary analysis), less precise than on source.

The shift-left philosophy (Agile, DevOps, DevSecOps) moves security tasks to the left of the project timeline: architecture review and threat modeling (prerequisites to testing, not testing) upstream, then SAST during development. ISO 27002 14.2.1 prescribes secure development rules and training in secure coding standards.

4.2 Misuse case and negative testing

A use case describes an intentional way of using a system to accomplish authorised work. Misuse case testing examines the opposite assumption: a set of actions that could cause integrity failures, malfunctions or compromises, whether deliberate and malicious or accidental. It is built into test plans, often focusing on system behaviour facing unexpected field values (wrong format, data far from the expected input). Simple example: a user enters their phone number in a field meant for their name. Modelling hostile agents and their misuse cases focuses the security professional's attention.

Negative testing, in contrast to a positive test (which checks the system works as expected and fails on any error), provides evidence of application behaviour facing invalid or unexpected data. Any provocation of failure should surface in testing rather than in production. The optimal response to a negative test is to gracefully reject the data without crashing. It is optimal to combine positive and negative tests.

Don't confuse the two: negative testing detects crashes in different situations; misuse case takes a hostile actor's viewpoint. Safety-critical systems blur the line (fail-safe requirement). Advances in automated testing, like fuzz testing (mutation or generation of inputs), allow tens of thousands of cases at the edges of the system's acceptable envelope - for owners and attackers alike.

4.3 Test coverage and test levels

The level of structural testing is measured by coverage metrics, which indicate the percentage of the software structure evaluated. By convention, speaking of coverage implies 100%: statement coverage means 100% of statements have been executed at least once. The main types: statement coverage (each statement at least once, insufficient alone); decision/branch coverage (each branch takes each outcome, minimum for most software, insufficient for high-integrity); condition coverage (each condition takes all its values); multi-condition coverage (all combinations of conditions); loop coverage (loops run zero, one, two and many times); path coverage (each feasible path, rarely achievable given the number of paths); data flow coverage (each feasible data flow). The target level should be proportionate to the software's risk.

Tests are also structured by level: unit (smallest isolated component), integration (modules together), system (whole system end to end) and acceptance (UAT, validation against business needs). Add cross-cutting tests: regression (verify a change broke nothing), positive, negative, and white-box / black-box approaches. Reaching 100% coverage is rarely realistic: cost and time-to-market pressure make it hard, especially on large code, even if not strictly impossible.

4.4 Interface testing

An interface is a point of interaction with a system. Examples: the UI (human interaction, GUI with windows and menus, or command-line CLI) and APIs (system-to-system communication: REST for the web, IPC, RPC). Evaluating the functionality and security of interfaces is a crucial aspect of system testing: security controls must be implemented in them to ensure confidentiality (access controls) and non-repudiation (logging of actions). Interface testing is part of standard development testing and plays a role in security activities (vulnerability assessments, pentests, breach simulations).

Interface testing verifies that components (software and hardware) pass data and control to each other correctly, per their design. It differs from integration testing: it checks that distinct components are in sync and that functions like data transfer happen as designed. In particular, it verifies that application-server interactions execute correctly, that errors are properly handled, what happens if the user interrupts a transaction, or if a web server connection is reset. On the server side, it validates communication between web, application and database servers, and software/hardware/network compatibility. On external interfaces, it checks supported browsers and error conditions when the external application is unavailable.

4.5 Synthetic transactions and RUM

To evaluate the performance, response time and availability of applications, sites and services, two complementary approaches exist.

Synthetic transactions are automated activities run against a monitored target to evaluate its performance. For a web application, this can be logging in with a test account to verify the response to login or queries. The term transactions does not mean only financial transactions: any system activity (DNS resolution, IP assignment via DHCP) qualifies. Synthetic performance monitoring (or proactive monitoring) has external agents run scripts following a typical user's steps (search, view a product, log in, pay). It is testing by scripts, not by live actions, which distinguishes it from other test forms. It covers website monitoring, database monitoring, TCP port monitoring and SLA validation, and runs 24/7 even with no real traffic.

RUM (real-user monitoring), also called real-user measurement or end-user experience monitoring (EUM), is by contrast a form of passive monitoring: it captures and analyses every transaction of every real user of a site or application. Top-down, client-side RUM (small JavaScript scripts or local agents) sees directly the experience the user lives and the link between site speed and satisfaction. RUM also applies to people-intensive operations (call centres). Synthetic and RUM complement each other: synthetic watches availability during low-traffic periods, RUM measures real experience.

5.1 Account management and IAM review

An account is the set of credentials used to access a resource. Since access control is the bedrock of IS security, collecting process data focuses first on account management. It supports business functions by: assigning account managers; setting conditions for group or role membership; specifying authorised users; requiring approval for authorisations (creating, enabling, modifying, disabling, removing access); monitoring account usage; notifying the manager when access is no longer needed; reviewing accounts for compliance. Access control is an example of a blended control: policies enforced by procedures, implemented by technology, sometimes with physical controls (badges, smart cards).

IAM review rests on three central information sets. The identity store is the centralised repository of all information about an identity; it also records decisions to grant, remove or downgrade permissions. The integrated IAM system maintains its authentication and authorisation databases (the 3rd A of AAA: authentication, authorization, accounting). System and application logs, monitoring agents and sensors record indicators of user actions. User behaviour modelling can detect indicators triggering account reviews. As jobs change and, ultimately, people leave, accesses are modified, disabled, deprovisioned then deleted. Auditing these sources provides rich material for security assessment.

5.2 Management review and approval

Management is responsible for defining and executing the organisation's mission, including security and privacy. To ensure controls reduce risk to an acceptable level, it conducts reviews and approves security process data. ISO 27001 requires top management to review the ISMS at planned intervals to ensure its continuing suitability, adequacy and effectiveness - an expectation echoed by NIST, ITIL and COBIT 5.

Management review should include (but not be limited to): exemptions from normal activities, information from previous reviews, ongoing outcome metrics, audit results, whether security objectives have been met, feedback from interested parties, and risk assessment reporting with the treatment plan. The goal is to support continual process improvement. Many organisations still rely on subjective judgments to prioritise, whereas major frameworks require statistical methods, metrics and benchmarks - which assumes logging and monitoring provide the necessary data-driven information.

System owners have similar responsibilities for their own stores: they identify their risk tolerance, compliance obligations, and decide to authorise a system's use. They are essential to determine whether selected controls meet their objectives. Environment changes (new business lines, systems, compliance, mergers/acquisitions) can affect control effectiveness, hence continuous monitoring generating metrics and indicators.

5.3 KPI and KRI: measuring the past and anticipating the future

You only manage well what you measure. Security metrics serve as key performance indicators (KPI) and key risk indicators (KRI). Mind the metric vs indicator distinction: a metric is a plain measurement (the number of patches applied), while an indicator uses metrics and provides context (an indication). Example: the ratio of patched endpoints to the total is an indicator of patch management effectiveness; if you expect 100% coverage by D+7 and the figure is below, the indicator signals corrective action.

KPI and KRI differ in their time orientation. KPIs look at the past: the activity already happened, were goals met? They help correct future actions, can reveal underperforming segments, justify rewards and reassure a client base. KRIs look at the future: they capture how the changing risk landscape might affect the organisation, using modelling, analysis or educated estimation. They allow assessing a risk's likelihood and triggering proactive action - for example, a rise in malicious traffic volume blocked by a firewall may indicate the need to upgrade it before it is overwhelmed. KRIs often support long-term decisions (budget, resources) and must be reviewed via formal change management when risk evolves.

5.4 Backup verification and availability services

Availability services are another source of process data, because their performance drives the organisation's ability to meet its commitments and compliance obligations. Availability data is routinely audited.

Data and system backups are essential integrity and availability controls. The first step is to identify critical data and systems, then design a backup strategy suited to recovery needs. Once procedures are in place, it is crucial to verify the integrity of the backup PROCESS AND the backed-up data. Both are evaluated by a test restore from the backup media, to ensure all required data is present and readable. If data is missing, it may indicate misconfiguration (excluded directories or systems) or media corruption. Some backup systems include automatic integrity checking, generate alerts on loss and retry the backup.

On compliance, ISO 27001 Annex A:12.3 is the principal control, but the need for backup appears across many controls. FIPS 200 covers the Contingency Planning (CP) and Media Protection (MP) families. HIPAA requires both a data backup plan and a business contingency plan, and the Availability TSC criterion sets similar expectations for a SOC 2. Developing an availability programme extends risk management practice: execution is IT's job, but it is directed by the risk of data loss or failure.

6.1 SETA: education, training, awareness

The security environment changes fast: establishing and maintaining a security education, training and awareness (SETA) programme is vital, because users are both a critical line of defence and a high-value target. The three activities differ. Education presents a body of knowledge applicable to a broad range of problems; as the body evolves fast, a professional's education never stops (ISC2 members must demonstrate continuing education). Training targets acquiring specific knowledge to perform a task (configuring and using a technology or control correctly). Awareness is more general: NIST SP 800-50 distinguishes the two by saying training teaches skills, while awareness focuses attention on an issue.

A SETA's structure and effectiveness depend on the organisation's policy and strategy. A needs assessment, initial and continuous, determines the training strategy and justifies resource allocation. It must involve key roles with specific training needs: executive management (understand directives and laws), security personnel (experts in policy and best practices), system owners (strong understanding of controls), system administrators and IT support (technical knowledge), operational managers and users (awareness and rules of behaviour). FIPS 200 identifies the Awareness and Training (AT) family; ISO 27001 A.7.2.2 has a similar expectation.

Evaluating effectiveness is hard: counting attendees shows no behaviour change. Anti-phishing campaigns, which send fake emails and track responses, are one way to quantify the effect. Successful SETA programmes have meaningful evaluation measures (completion rates, long-term retention, coverage, audience tailoring), reused for retraining and re-evaluation.

6.2 BC and DR: two distinct processes

Disaster recovery (DR) and business continuity (BC) are often paired but distinct. DR means planning and implementing strategies to restore IT systems, data and operations after a disaster (natural disaster, cyberattack, hardware failure): it ensures quick recovery and minimises downtime, protecting data integrity and business functionality. BC focuses on maintaining essential business functions during and after a disaster: it goes beyond IT (employee safety, communication protocols, alternative work sites), to support core operations, customer service and limit financial losses. Together, BC and DR form the organisation's resilience.

Process data related to BC and DR is of great interest to controls assessors. Major standards expect risk-driven BC/DR planning: NIST SP 800-34 (Contingency Planning Guide) charts a road map; ISO 27031 addresses IS continuity and aligns with ISO 22301 (Business Continuity Management Systems). Many sectors (finance, nuclear, aviation, critical infrastructure) impose high resilience expectations, sometimes with a regular, documented full-scale demonstration.

A well-established dictum reminds us that the disaster you get is never the one you plan for. The plan, as an artifact, demonstrates commitment, but the true test is showing the organisation can execute it. Hence the importance of training teams in conducting recovery, with progressive complexity and breadth.

6.3 Testing BC/DR plans: from desk check to full cutover

BC/DR testing and training is process data of first interest to the assessor: it demonstrates control effectiveness and feeds continual improvement. The training programme should progress in complexity and organisational breadth, each successful event preparing the next. A typical progressive programme is ordered as follows.

The desk check: each member reviews operational documentation, rosters, their availability and their version of the DR plan; expected result: everyone has a current plan. The walk-through: using manuals, DR teams explain their roles and activities; result: they understand their role relative to each other and deconflict overlaps. The tabletop exercise: team leaders are presented a disaster scenario and discuss their response; result: deconfliction of roles and identification of major plan weaknesses. The simulation: a disaster is simulated OUTSIDE the production environment, DR activities are performed there; result: demonstration of recovery capabilities within the scenario context. The parallel: a disaster is simulated while production continues; result: demonstration of full recovery without disrupting production. The full cutover (full-interruption): a disaster is invoked that actually affects production; result: recovery of the production environment within established parameters.

Classic trap: a fire evacuation drill is a walk-through (step-by-step review of procedures without real-time execution), not a simulation. The results of these activities are documented and made available to assessors and other parties for process improvement.

7.1 Remediation and continual improvement

The CISSP must establish a process, called remediation, to address findings from tests, assessments and audits. This mobilises project management skills: prioritise tasks, set timelines and milestones, track progress to completion. Audit findings can feed other parts of the programme (risk analysis): a deficient control reveals a poorly mitigated risk, which should trigger a review of the risk management process.

Control-environment changes fit a continual improvement approach. The most common model is PDCA (Plan-Do-Check-Act), also called the Shewhart or Deming wheel, in four steps. Plan: decide what needs to be done, set objectives and processes. Do: execute the plan (on a small scale). Check: evaluate results (statistical measures, observations, evaluations). Act (Adjust): from the Do and Check phases, identify root causes, reassess risk, reset the baseline; then the cycle restarts.

Six Sigma applies a similar five-step model (Define, Measure, Analyze, Improve, Control - DMAIC), repeated until the desired quality level. All major frameworks apply continual improvement to better align control performance with risk. Maturity (CMM) follows the same logic. No single model fits every organisation: tailoring considers culture, risk management, prioritisation, compliance and change management. Continual improvement is as much a mindset as a process.

7.2 Exception handling

When a finding or indicator establishes that a control does not reach the agreed level of risk reduction, the usual response is to improve the control. In certain circumstances, the appropriate response is instead to adjust risk tolerance to align with control performance. The change management process must engage the system owner to determine the proper course.

When an evaluation reveals an issue that cannot be remediated, it must go through an exception process, modelled on policy exceptions (used when a system cannot meet a requirement). Exceptions may be granted, but only temporarily. If a permanent exception is requested, or if the same issue prompts repeated requests, it signals misalignment with the organisation's objectives, warranting updates to policies or governance decisions.

Concretely: if the situation is temporary and the system owner accepts the risk, change management documents the circumstances and flags the exception for review at an agreed date or event. If the owner refuses the additional risk, they must either resource the additional control or take other measures covering the new risk level. In all cases, exceptions must be documented, authorised and reviewed.

7.3 Ethical disclosure of vulnerabilities

Web applications, sites and services likely contain vulnerabilities. A community of researchers works to identify and report them responsibly, before a malicious actor exploits them: this is ethical or responsible disclosure. Organisations with web resources must be ready to receive these often-unexpected reports from external people. Three postures coexist.

Full disclosure: whoever discovers a weakness publishes it as fast as possible to all affected parties. Its advocates argue that not disclosing everything immediately leaves organisations at risk, and that public embarrassment pushes vendors to fix. Responsible disclosure: the discoverer first reports to the responsible organisation and gives it time to fix before publication; the delay is debated, but the idea is supported by Linus Torvalds, Microsoft, Google. Nondisclosure: do not disclose, sometimes the only legal and ethical path, when agreements (NDA), an ongoing investigation, privacy rights or protected financial data require it; safe harbor programs tried to create sharing channels but run afoul of GDPR.

Project Zero (Google, 2014) illustrates responsible disclosure: the team passes details to the vendor and, with no patch within 90 days, makes the weakness public; over 95% are fixed within the window. In the US, the Vulnerabilities Equities Process (VEP) arbitrates, case by case, the disclosure of vulnerabilities known to agencies. Add mandatory reporting (legal duty to report certain circumstances, e.g. crimes involving minors) and whistleblowing (ethical reporting, variably protected by jurisdiction, hence the value of professional insurance). The professional must clarify the legal status of whistleblowing before any disclosure.

8.1 Internal audit and methodology

Assessments and audits can be conducted internally or by independent third parties. An independent evaluator gives more confidence in opinions, attestations and findings. Internal audit aims to determine whether controls meet the organisation's risk expectations; it also helps improve operational efficiency and prepare external audits. It is often conducted by the organisation itself (self-assessment), which demands rigour and a willingness to ask hard questions - many breached organisations discover their culture did not encourage this introspection.

Internal audit advantage: the auditors' familiarity with processes, tools and personnel. But that familiarity is also a risk: issues may be unintentionally overlooked, and the internal auditor may lack the independence for an objective assessment (temptation to downplay a major issue that would affect performance or relationships). Some organisations keep auditors on a separate team; for smaller ones, this is not always feasible.

The methodology follows phases. Chartering: management sponsors and resources the evaluation. Scoping: management determines depth, breadth, schedule, reporting format and who does the work (stakeholder engagement and risk assessment run throughout). Testing: conducting physical, technical and administrative controls (vulnerability assessment, penetration testing). Reporting: the report serves as a benchmark, a corrective tool or an artifact for external audit; the level of detail fits the audience, disclosure stays management's responsibility, and artifacts are protected. Remediating: associating results with corrective action; in the US, the Plan of Action and Milestones (POA&M) tracks the weakness, the proposed remediation, resources and schedule (NIST SP 800-37).

8.2 Preparing and conducting a SOC audit

For a service provider never audited before, preparing a first SOC examination typically follows two phases. The security professional plays a key advisory and execution role here.

The audit preparation phase begins by familiarising management and key personnel with the process and purpose of SOC examinations. The team then proceeds as with any major project: clearly establish schedule, scope and success criteria; determine the controls in the examination's scope, inventory current controls and identify missing ones; inventory documentation; use gap analysis techniques (readiness review) to identify and prioritise issues; develop and carry out recommendations to resolve discrepancies; verify resolution; and establish the audit and reporting approach best suited to external needs.

The audit phase starts once preparation is complete: the organisation is ready for its first formal audit. Having outside auditors examine the organisation in detail can be disruptive; good planning reduces it. It requires sound project management: build a detailed audit plan; collect all required artifacts in advance; provide access, workspaces and rooms; plan introduction meetings, progress reviews and outbriefings; conduct required tests and provide artifacts; allow auditors time for off-site analysis; resolve impediments collaboratively; provide draft and final report versions for management review; and conduct a post-audit (lessons-learned) review for the next cycle.

8.3 External audits: types and conduct

An external audit is an assessment by a third party demonstrating the organisation's controls meet a compliance standard - a standard that imposes consequences for non-compliance (financial penalties, criminal sanctions, activity limits). External audits are regular (risk, governance, finances) and employ an independent organisation with no conflict of interest; sometimes the compliance body audits. In all cases, the audited organisation is responsible for demonstrating compliance. External audits are often mandatory for regulatory compliance; their advantage is being thorough and unbiased, with auditors of specialised skill (e.g. SOC 2 under SSAE 18).

Chartering is the audited organisation's governing body's responsibility, setting schedule, scope, resources and coordination leads. Scope and objectives depend on the audit type and standard. The most common types: compliance audits (test specific controls vs a standard), financial audits (accuracy of financial reporting), operational audits (a process's internal controls), information systems audits (controls over IS development and operation), integrated audits (operational + financial), forensic audits (discover and report fraud or criminal activity).

Process: the auditee knows the applied standard in advance, allowing support preparation. Pre-audit planning identifies skills and resources, prepares an audit checklist (areas to investigate, artifacts, people to interview, schedule) shared with the auditee to close gaps. Audit execution collects artifacts, runs tests and interviews (on site and remotely). Audit reporting analyses and compiles a draft report shared with the auditee to correct errors and omissions, before presenting final results. The auditor must have sufficient expertise, possibly engaging a third party for specialised tests (authorised by the audit charter).

8.4 Third-party audits, managed services and supply chain

Third-party relationships cover supply chains and providers: CSPs, ISPs, managed security services, consultants, but also hardware/software vendors (considered part of the supply chain). Third-party audit is a key risk management tool: the security practitioner can conduct the audit or request an audit report from the third party. The shared objective is to identify third-party risks that could impact your organisation and assess the effectiveness of mitigations. Common examples: SOC 2 and CSA STAR. Auditing the practices of third parties with access to your sensitive data is vital, because many privacy laws hold the data controller legally responsible for a breach, even one caused by a data processor (e.g. a cloud).

Two processes should drive identifying and characterising all third parties (risk, compliance, or both). Contract terms must establish security and compliance responsibilities; otherwise, that gap must be closed, contracts renegotiated. A third party may itself have relationships (its own providers or clients): the first party must identify the mix of risk and compliance and resolve it with the third party.

Managed services hand a third party responsibility for all or part of a set of functions. The security professional engages on two fronts: those delivering IT/security services (identity, incident response, BC/DR) and those who, on other services, have shared access to the organisation's assets. Ideally, a contract and SLA establish a cooperative relationship giving visibility into the provider's audits and allowing your own testing. At minimum, SLAs must grant summary-level information on security audits. Finally, organisations also exist in others' supply chains: supply chain management principles apply upstream (suppliers) and downstream (customers); the ISO 28000 series addresses supply chain security management. Evidence collection (field work) happens in person or remotely, on on-premises, cloud or hybrid assets, the shared responsibility model clarifying who owns which controls (e.g. physical security of the infrastructure provider's data centres).