jmagly/sigma-hunting

sigma-hunting

Applies Sigma detection rules against collected log sources to identify threat activity. Supports the bundled forensics-complete Sigma rule library and custom rules. Converts Sigma rules to backend-specific queries for Elasticsearch, Splunk, and grep, enabling hunting across both real-time platforms and offline log files.

Triggers

Alternate expressions and non-obvious activations (primary phrases are matched automatically from the skill description):

• "ATT&CK [technique]" / "MITRE [TID]" → technique-specific threat hunt
• "T1059" / "T1053" (etc.) → ATT&CK technique ID lookups
• "Sigma rules" → rule-based threat hunting

Purpose

Sigma provides a vendor-neutral rule format for expressing detection logic. Writing backend-specific queries for every log source and SIEM is time-consuming and error-prone. This skill translates Sigma rules to the appropriate query format for the available tooling, applies them against collected logs, and reports matches with ATT&CK technique context.

Behavior

When triggered, this skill:

1. Identify available rule sources:

   • Bundled rules: agentic/code/frameworks/forensics-complete/sigma/
   • Custom rules: .aiwg/forensics/sigma/custom/
   • Check for sigma-cli or pySigma installation: sigma --version 2>/dev/null
   • If sigma-cli is unavailable, use built-in grep-based conversion for simple rules

2. Identify target log sources and backends:

   • Detect available log sources: journald, flat files, Elasticsearch index, Splunk index
   • Match Sigma logsource categories to available sources:
     • category: process_creation → syslog, auditd logs, or EDR telemetry
     • category: network_connection → firewall logs, VPC flow logs, Zeek conn.log
     • category: webserver → nginx/Apache access logs
     • product: linux → auth.log, syslog, journal
     • product: windows → Windows Event Log exports (.evtx or JSON)

3. Select applicable rules:

   • Filter rule library by logsource compatibility with available sources
   • If a specific MITRE technique is requested (e.g., "hunt for T1059"), filter by tags: attack.t1059*
   • Apply severity filter: default to medium, high, critical rules only
   • List selected rules and their ATT&CK technique mappings before execution

4. Convert rules to grep (offline log files):

   • Parse Sigma YAML detection field
   • Convert selection keywords to extended grep patterns:

     grep -Ei 'pattern1|pattern2' /var/log/auth.log
     

   • Handle condition: selection and not filter by piping through a second grep with -v
   • Note: grep conversion handles keyword-only rules; complex field-mapped rules require sigma-cli

5. Convert rules via sigma-cli (when available):

   • Elasticsearch backend:

     sigma convert -t elasticsearch -f lucene rules/sigma/linux/ > hunt-queries.txt
     

   • Splunk backend:

     sigma convert -t splunk rules/sigma/linux/ > hunt-spl.txt
     

   • Execute converted queries against the target index or log source

6. Execute hunts and collect matches:

   • Run each rule against the target log source
   • Record: rule name, ATT&CK technique, match count, first and last match timestamp, sample matching lines
   • Group results by ATT&CK tactic for reporting

7. Triage matches:

   • Flag rules with zero matches (coverage gap) vs rules with matches (hits)
   • For each hit: extract relevant fields (source IP, username, process name, command line)
   • Cross-reference extracted values with IOC list from ioc-extraction skill

8. Apply custom rules:

   • Load any .yml files from .aiwg/forensics/sigma/custom/
   • Validate YAML structure and required Sigma fields before execution
   • Report custom rule coverage alongside bundled rule results

9. Write hunt report:

   • Save to .aiwg/forensics/findings/<hostname>-sigma-hunt.md
   • Include: rules applied, hits per rule, ATT&CK tactic coverage map, sample evidence per hit, coverage gaps

Usage Examples

Example 1 — Full hunt against local logs

sigma hunt

Example 2 — Hunt for specific technique

hunt for T1078

Example 3 — Convert rules for Elasticsearch

sigma rules --backend elasticsearch --output hunt-queries.txt

Output Locations

• Hunt report: .aiwg/forensics/findings/<hostname>-sigma-hunt.md
• Converted queries: .aiwg/forensics/sigma/converted/
• Rule hit evidence: .aiwg/forensics/evidence/sigma-hits.txt

Configuration

sigma_hunting:
  bundled_rules_path: agentic/code/frameworks/forensics-complete/sigma/
  custom_rules_path: .aiwg/forensics/sigma/custom/
  default_severity_filter:
    - medium
    - high
    - critical
  default_backend: grep
  available_backends:
    - grep
    - elasticsearch
    - splunk
  sigma_cli_path: sigma

References

• @$AIWG_ROOT/agentic/code/addons/aiwg-utils/rules/research-before-decision.md — Identify available log sources and rule libraries before executing hunts; check tool availability
• @$AIWG_ROOT/agentic/code/frameworks/forensics-complete/rules/evidence-integrity.md — Hunting must not modify log sources; run rules against read-only copies
• @$AIWG_ROOT/agentic/code/frameworks/forensics-complete/rules/red-flag-escalation.md — Escalate to human when Sigma hits indicate active compromise or critical severity findings
• @$AIWG_ROOT/agentic/code/frameworks/forensics-complete/skills/ioc-extraction/SKILL.md — Cross-reference Sigma rule hits against extracted IOCs for confirmation
• @$AIWG_ROOT/agentic/code/frameworks/forensics-complete/skills/log-analysis/SKILL.md — Log analysis skill provides the correlated timeline that Sigma hunting adds detection coverage to