Controls and Audit Evidence

Designing Effective Controls

A control that exists only in a policy document is not a control — it's an intention. An effective control is one that operates consistently, produces verifiable evidence, and can be independently tested by an auditor. The gap between designed controls and operating controls is where compliance failures happen.

The control lifecycle

Every control follows a lifecycle:

Design → Implement → Operate → Monitor → Evidence → Audit

• Design: Define what the control does and how it works
• Implement: Deploy the control in the environment
• Operate: The control functions as designed
• Monitor: Verify the control is operating correctly
• Evidence: Collect proof of operation
• Audit: An independent party verifies the evidence

A control that fails at any stage is not effective. A beautifully designed control that isn't implemented is worthless. An implemented control that isn't monitored may silently fail. A monitored control without evidence can't be audited.

Types of controls

Preventive controls stop incidents from occurring:

• MFA enforcement prevents unauthorized access
• Input validation prevents injection attacks
• Access controls prevent unauthorized actions

Detective controls identify incidents that have occurred:

• Logging and monitoring detect suspicious activity
• Vulnerability scanning identifies unpatched systems
• Access reviews detect over-permissioned accounts

Corrective controls remediate incidents after they occur:

• Incident response procedures contain and remediate
• Backup and recovery restore from data loss
• Patch management fixes vulnerabilities

A mature security program uses all three types. Relying solely on preventive controls is fragile — no prevention is perfect. Detective and corrective controls provide resilience when prevention fails.

Control design principles

1. Specificity: A control should be specific enough to be testable:

Vague: "Access is reviewed regularly"
Specific: "Quarterly access reviews are conducted for all production systems.
  The review verifies that each user's access is appropriate for their role.
  Findings are documented and remediated within 30 days.
  Evidence: Signed review spreadsheets with reviewer name and date."

2. Automation: Automated controls are more reliable than manual controls:

# Manual control: Someone remembers to review access monthly
# Automated control: AWS Access Analyzer continuously identifies external access

# Automated evidence collection
resource "aws_access_analyzer_analyzer" "main" {
  analyzer_name = "production-access"
  type          = "ACCOUNT"
}

3. Evidence generation: Every control should produce evidence as a byproduct of operation:

4. Independence: The person operating the control should not be the same person verifying it:

• The engineer who deploys a change should not be the only person who verifies it
• The person who configures a control should not be the only person who tests it
• Separation of duties prevents conflicts of interest and catches errors

Control failure modes

Understanding how controls fail helps you design more resilient ones:

1. Design failure: The control is designed incorrectly and doesn't address the risk
2. Implementation failure: The control is designed correctly but deployed incorrectly
3. Operational failure: The control works initially but degrades over time
4. Bypass failure: The control is circumvented through an unexpected path
5. Evidence failure: The control operates but produces no verifiable evidence

The most common failure in audit is evidence failure — the control works, but the organization can't prove it.

Evidence Collection and Audit Preparation

Audit season is stressful because evidence is scattered across systems, stored in inconsistent formats, and often requires manual collection. The solution is to treat evidence collection as an ongoing operational process, not a quarterly fire drill.

What auditors look for

Auditors evaluate evidence against three criteria:

1. Relevance: Does the evidence relate to the control being tested?
2. Reliability: Is the evidence trustworthy and unmodified?
3. Sufficiency: Is there enough evidence to conclude the control operated effectively?

Weak evidence: "We have a policy that says MFA is required"
Strong evidence: "IdP configuration export showing MFA enforced for all users,
  dated within the audit period, with a system-generated timestamp"

Types of audit evidence

System-generated evidence (most reliable):

• CloudTrail logs showing API activity
• IdP configuration exports
• Vulnerability scan reports
• Access review records from IAM tools
• Deployment logs from CI/CD pipelines

Manually created evidence (less reliable but necessary):

• Signed policy acknowledgments
• Meeting minutes from security reviews
• Incident response tickets
• Change approval records
• Training completion records

Testimonial evidence (least reliable):

• Interviews with control operators
• Written attestations
• Email confirmations

Auditors prefer system-generated evidence because it's harder to fabricate and provides a complete record. Manual evidence is acceptable but requires more scrutiny.

Automated evidence collection

The most effective approach is to collect evidence automatically through APIs:

# Example: Automated evidence collection for access reviews
import boto3
import json
from datetime import datetime

def collect_access_review_evidence():
    iam = boto3.client('iam')
    
    # Get all users and their access keys
    users = iam.list_users()['Users']
    evidence = []
    
    for user in users:
        access_keys = iam.list_access_keys(UserName=user['UserName'])['AccessKeyMetadata']
        policies = iam.list_attached_user_policies(UserName=user['UserName'])['AttachedPolicies']
        
        evidence.append({
            'user': user['UserName'],
            'create_date': user['CreateDate'].isoformat(),
            'access_keys': len(access_keys),
            'policies': [p['PolicyName'] for p in policies],
            'collected_at': datetime.utcnow().isoformat()
        })
    
    # Store with timestamp for audit trail
    filename = f"access-review-{datetime.utcnow().strftime('%Y-%m-%d')}.json"
    with open(filename, 'w') as f:
        json.dump(evidence, f, indent=2)
    
    return filename

Evidence storage and retention

Evidence must be stored securely and retained for the required period:

• SOC 2 Type II: Evidence for the entire audit period (6-12 months)
• ISO 27001: Evidence for the certification cycle (3 years)
• HIPAA: 6 years
• PCI DSS: 1 year minimum

# Evidence storage best practices
storage:
  location: S3 bucket with versioning enabled
  encryption: AES-256 at rest
  access: Read-only for auditors, write-only for collectors
  retention: Configured per compliance requirement
  integrity: Checksums to detect modification
  organization: /evidence/{framework}/{control}/{date}/

The evidence mapping matrix

Maintain a matrix that maps each control to its evidence sources:

control: MFA Enforcement
frameworks: [SOC2 CC6.1, ISO27001 A.9.4.2]
evidence_sources:
  - type: system-generated
    source: Okta configuration export
    frequency: monthly
    storage: s3://evidence/soc2/mfa/okta-config/
  - type: system-generated
    source: Okta MFA compliance report
    frequency: monthly
    storage: s3://evidence/soc2/mfa/mfa-compliance/
  - type: manual
    source: Signed MFA policy acknowledgment
    frequency: on hire
    storage: s3://evidence/soc2/mfa/acknowledgments/

This matrix ensures no control is without evidence and no evidence is without a purpose.

Continuous audit readiness

The goal is to be audit-ready at any time, not just during audit season:

1. Automate collection: Evidence is collected automatically on a schedule
2. Centralize storage: All evidence is stored in a consistent, searchable location
3. Tag and organize: Evidence is tagged by framework, control, and date
4. Regular self-assessments: Quarterly internal reviews verify evidence completeness
5. Gap identification: Automated checks flag missing or stale evidence

# Check for stale evidence (older than 90 days)
find /evidence -name "*.json" -mtime +90 -exec echo "STALE: {}" \;

# Check for missing monthly evidence
for month in $(seq -w 1 12); do
  if [ ! -f "/evidence/2026-${month}-access-review.json" ]; then
    echo "MISSING: 2026-${month}-access-review.json"
  fi
done

When evidence collection is automated and continuous, audit preparation becomes a review exercise rather than a frantic collection effort.