mukul975/building-detection-rules-with-sigma

Building Detection Rules with Sigma

When to Use

Use this skill when:

• SOC engineers need to create detection rules portable across multiple SIEM platforms
• Threat intelligence reports describe TTPs requiring new detection coverage
• Existing vendor-specific rules need standardization into a shareable format
• The team adopts Sigma as a detection-as-code standard in CI/CD pipelines

Do not use for real-time streaming detection (Sigma is for batch/scheduled searches) or when the target SIEM has native detection features that Sigma cannot express (e.g., Splunk RBA risk scoring).

Prerequisites

• Python 3.8+ with pySigma and appropriate backend (pySigma-backend-splunk, pySigma-backend-elasticsearch, pySigma-backend-microsoft365defender)
• Sigma rule repository cloned: git clone https://github.com/SigmaHQ/sigma.git
• MITRE ATT&CK framework knowledge for technique mapping
• Understanding of target SIEM log source field mappings

Workflow

Step 1: Define Detection Logic from Threat Intelligence

Start with a threat report or ATT&CK technique. Example: detecting Mimikatz credential dumping (T1003.001 — LSASS Memory):

title: Mimikatz Credential Dumping via LSASS Access
id: 0d894093-71bc-43c3-8d63-bf520e73a7c5
status: stable
level: high
description: Detects process accessing lsass.exe memory, indicative of credential dumping tools like Mimikatz
references:
    - https://attack.mitre.org/techniques/T1003/001/
    - https://github.com/gentilkiwi/mimikatz
author: mahipal
date: 2024/03/15
modified: 2024/03/15
tags:
    - attack.credential_access
    - attack.t1003.001
logsource:
    category: process_access
    product: windows
detection:
    selection:
        TargetImage|endswith: '\lsass.exe'
        GrantedAccess|contains:
            - '0x1010'
            - '0x1038'
            - '0x1fffff'
            - '0x40'
    filter_main_svchost:
        SourceImage|endswith: '\svchost.exe'
    filter_main_csrss:
        SourceImage|endswith: '\csrss.exe'
    filter_main_wininit:
        SourceImage|endswith: '\wininit.exe'
    condition: selection and not 1 of filter_main_*
falsepositives:
    - Legitimate security tools accessing LSASS
    - Windows Defender scanning
    - CrowdStrike Falcon sensor

Step 2: Validate Sigma Rule Syntax

Use sigma check to validate the rule:

# Install pySigma and validators
pip install pySigma pySigma-validators-sigmaHQ

# Validate rule
sigma check rule.yml

Alternatively, validate with Python:

from sigma.rule import SigmaRule
from sigma.validators.core import SigmaValidator

rule = SigmaRule.from_yaml(open("rule.yml").read())
validator = SigmaValidator()
issues = validator.validate_rule(rule)
for issue in issues:
    print(f"{issue.severity}: {issue.message}")

Step 3: Convert to Target SIEM Query

Convert to Splunk SPL:

from sigma.rule import SigmaRule
from sigma.backends.splunk import SplunkBackend
from sigma.pipelines.splunk import splunk_windows_pipeline

pipeline = splunk_windows_pipeline()
backend = SplunkBackend(pipeline)

rule = SigmaRule.from_yaml(open("rule.yml").read())
splunk_query = backend.convert_rule(rule)
print(splunk_query[0])

Output:

TargetImage="*\\lsass.exe" (GrantedAccess="*0x1010*" OR GrantedAccess="*0x1038*"
OR GrantedAccess="*0x1fffff*" OR GrantedAccess="*0x40*")
NOT (SourceImage="*\\svchost.exe") NOT (SourceImage="*\\csrss.exe")
NOT (SourceImage="*\\wininit.exe")

Convert to Elastic Query (Lucene):

from sigma.backends.elasticsearch import LuceneBackend
from sigma.pipelines.elasticsearch import ecs_windows_pipeline

pipeline = ecs_windows_pipeline()
backend = LuceneBackend(pipeline)
elastic_query = backend.convert_rule(rule)
print(elastic_query[0])

Convert to Microsoft Sentinel KQL:

from sigma.backends.microsoft365defender import Microsoft365DefenderBackend

backend = Microsoft365DefenderBackend()
kql_query = backend.convert_rule(rule)
print(kql_query[0])

Step 4: Map to MITRE ATT&CK and Add Coverage Metadata

Tag every rule with ATT&CK technique IDs in the tags field:

tags:
    - attack.credential_access        # Tactic
    - attack.t1003.001                # Sub-technique
    - attack.t1003                    # Parent technique

Track detection coverage using the ATT&CK Navigator:

import json

# Generate ATT&CK Navigator layer from Sigma rules
layer = {
    "name": "SOC Detection Coverage",
    "versions": {"attack": "14", "navigator": "4.9", "layer": "4.5"},
    "domain": "enterprise-attack",
    "techniques": []
}

# Parse Sigma rules directory for technique tags
import os
from sigma.rule import SigmaRule

for root, dirs, files in os.walk("sigma/rules/windows/"):
    for f in files:
        if f.endswith(".yml"):
            rule = SigmaRule.from_yaml(open(os.path.join(root, f)).read())
            for tag in rule.tags:
                if str(tag).startswith("attack.t"):
                    technique_id = str(tag).replace("attack.", "").upper()
                    layer["techniques"].append({
                        "techniqueID": technique_id,
                        "color": "#31a354",
                        "score": 1
                    })

with open("coverage_layer.json", "w") as f:
    json.dump(layer, f, indent=2)

Step 5: Test Rule Against Sample Data

Create test data and validate the rule catches the expected events:

# Use sigma test framework
sigma test rule.yml --target splunk --pipeline splunk_windows

# Or manually test in Splunk with sample data
# Upload Sysmon process_access log with known Mimikatz signature

Validate false positive rate by running against 7 days of production data in a non-alerting saved search.

Step 6: Deploy to Production SIEM

Deploy the converted query as a scheduled search or correlation rule:

Splunk ES Correlation Search:

| tstats summariesonly=true count from datamodel=Endpoint.Processes
  where Processes.process_name="*\\lsass.exe"
  by Processes.src, Processes.user, Processes.process_name, Processes.parent_process_name
| `drop_dm_object_name(Processes)`
| where count > 0

Elastic Security Rule (TOML format):

[rule]
name = "LSASS Memory Access - Credential Dumping"
description = "Detects suspicious access to LSASS process memory"
risk_score = 73
severity = "high"
type = "eql"
query = '''
process where event.action == "access" and
  process.name == "lsass.exe" and
  not process.executable : ("*\\svchost.exe", "*\\csrss.exe")
'''

[rule.threat]
framework = "MITRE ATT&CK"
[[rule.threat.technique]]
id = "T1003"
name = "OS Credential Dumping"

Step 7: Version Control and CI/CD Integration

Store rules in Git with automated testing:

# .github/workflows/sigma-ci.yml
name: Sigma Rule CI
on: [push, pull_request]
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - run: pip install pySigma pySigma-validators-sigmaHQ
      - run: sigma check rules/
      - run: sigma convert -t splunk -p splunk_windows rules/ > /dev/null

Key Concepts

| Term | Definition | |------|-----------| | Sigma | Vendor-agnostic detection rule format (YAML-based) that compiles to SIEM-specific queries via backends | | pySigma | Python library replacing legacy sigmac for rule conversion, validation, and pipeline processing | | Backend | pySigma plugin that translates Sigma detection logic into a target platform query language (SPL, KQL, Lucene) | | Pipeline | Field mapping configuration that translates generic Sigma field names to SIEM-specific field names | | Logsource | Sigma rule section defining the category (process_creation, network_connection) and product (windows, linux) of the target data | | Detection-as-Code | Practice of managing detection rules in version control with CI/CD testing and automated deployment |

Tools & Systems

• SigmaHQ: Official Sigma rule repository with 3,000+ community-maintained detection rules on GitHub
• pySigma: Python-based Sigma rule processing framework with modular backends and pipelines
• ATT&CK Navigator: MITRE tool for visualizing detection coverage mapped to ATT&CK techniques
• Uncoder.IO: Web-based Sigma rule converter supporting 30+ SIEM platforms for quick translation

Common Scenarios

• New CVE Detection: Write Sigma rule for exploitation indicators (e.g., Log4Shell JNDI lookup patterns in web logs)
• Hunting Rule Promotion: Convert ad-hoc Splunk hunting query into Sigma rule for ongoing automated detection
• Multi-SIEM Migration: Converting 500+ Splunk correlation searches to Sigma for migration to Elastic Security
• Purple Team Output: Convert red team findings into Sigma rules for immediate defensive coverage
• Threat Intel Operationalization: Transform IOC-based threat reports into behavioral Sigma rules

Output Format

SIGMA RULE DEPLOYMENT REPORT
━━━━━━━━━━━━━━━━━━━━━━━━━━━
Rule ID:      0d894093-71bc-43c3-8d63-bf520e73a7c5
Title:        Mimikatz Credential Dumping via LSASS Access
ATT&CK:       T1003.001 - LSASS Memory
Severity:     High
Status:       Deployed to Production

Conversions:
  Splunk SPL:    PASS — Saved search "sigma_lsass_access" created
  Elastic EQL:   PASS — Detection rule ID elastic-0d894093 enabled
  Sentinel KQL:  PASS — Analytics rule deployed via ARM template

Testing:
  True Positives:    4/4 test cases matched
  False Positives:   2 in 7-day backtest (svchost edge case — filter added)
  Performance:       Avg execution 3.2s on 50M events/day