Updated: Sep 20
A SOC assessment is not as simple as pass or fail, but some outcomes are better than others. There's common themes behind where things go wrong and how some companies get it right.
What is considered a good outcome for SOC assessments?
What is the best way to avoid bad outcomes?
In SOC reporting, the auditor provides an attestation. This is different to a certification that applies for ISO 27001. The attestation confirms that the auditor believes:
1. the SOC report is fairly presented;
2. that there are controls in place that are designed effectively; and
3. for Type 2 reports, that those controls are also operating effectively.
SOC 2 Issues
The service auditor can note exceptions from their assessment, which may result in one or more qualifications. One or more qualifications is generally considered a "fail". Exceptions without any qualifications are quite normal and generally no cause for alarm.
What are qualifications?
Qualifications are statements by the service auditor in contradiction to the attestations above. These are noted in the auditors opinion after the attestation by saying "except for", followed by a list of any qualifying statements.
1. Fair presentation: The service auditor can theoretically note a qualification that they do not believe the report is fairly presented to its end users. In practice, you don’t see these qualifications because the report would be updated prior to issuance in order to be fairly presented.
2. Design effectiveness: The service auditor may note the controls in place are not designed effectively to address the criteria or control objectives. This is rare for a Type 1 report as usually these would be addressed prior to issuing the report. However, it does happen in Type 2 reports. The processes may have changed in a way that no longer meets the criteria and/or the service auditor may identify a weakness that was not observed before.
3. Operating effectiveness (Type 2 only): A qualification against the operating effectiveness is usually due to the aggregation of multiple exceptions noted. Exceptions are where the controls do not operate as described, or evidence was not retained to demonstrate the controls operated. Qualifications usually arise when it's a pervasive issue rather than a one off. Multiple exceptions in the same control or multiple controls in the same process that affects the same criteria or control objective.
In essence, a qualification is saying the auditor is unable to provide reasonable assurance that the criteria or control objectives are being met. Rather than being a blanket statement (pass or fail), it is specific to the matter(s) identified, to allow users to consider those.
What’s the best way to “Pass” your SOC 2 Audit?
There's a few main themes behind where things go wrong, and ways that companies help avoid them to achieve a “clean” SOC report:
Assign ownership: Track all of your SOC controls with ownership assigned, clear definitions of how the controls operate, and the evidence that you need to retain for the audit process. Ensure control owners are clear on the requirements and periodically check-in to keep the SOC requirements front of mind.
Consider change implications: A common fail point is changes to the process during the live reporting period that don't consider the SOC implications. SOC is designed to be flexible and allow changes, but it's important to ensure the new process addresses SOC requirements including retention of evidence of the controls operating. Check in with your service auditor when going through significant changes.
Compensating controls: All processes and controls are prone to fail from time-to-time. Compensating controls help ensure failures don't result in qualifications. Compensating controls include additional checks and/or other controls that address the same control objective.
Raising awareness: SOC reports require a level of formality and consistency of operating practices. A common fail point is when a new event or circumstance arises that wasn't considered when designing the controls. Raising awareness in the organisation of the SOC requirements helps to identify and address these when they occur.