Active
--:--:-- UTCSyed Dayaan Shah
← Back to case filesCase · SOC-014
Detection & ResponseSeverity · high2-week tuning sprint

SIEM alert triage — separating signal from noise under pressure

Inherited a Splunk ES instance drowning in false positives. Rebuilt the triage flow against a live stream of mixed alerts — brute force, impossible travel, benign scanners, DGA — and dropped mean-time-to-decision from 6 min to under 90 s.

Evidence · Post-tuning dashboard · 14-day trendLIVE
Splunk · SOC OverviewLast 14 days

Open alerts

12

MTTD

6m 41s

FP rate

18%

Escalations

3

Alert volume · daily

↓ 62% after detection tuning

MTTD (lab)

14m → 3m

FP rate

-61%

New searches

6

ATT&CK coverage

+33%

[01]Objective

Cut analyst time-on-alert by tuning noisy detections and building a repeatable triage loop.

[02]Live Lab · SIEM triage lab
Interactive · runs in your browser
  1. 01 · Brief
  2. 02 · Your move
  3. 03 · Result
  4. 04 · How I stopped it
  5. 05 · How you prevent it
Stage 1 / 5 · Brief

Six alerts just landed in the queue. Some are real, some are noise. You have one minute of analyst time budgeted per alert — real SOCs don't get more.

Tag each as TP (true positive, escalate) or FP (false positive, tune the rule).

[03]Stack · Tools & Platforms

Tools

  • Splunk ES
  • Sysmon
  • MITRE ATT&CK Navigator
  • PowerShell

Platforms

  • Splunk
  • Windows Server 2022
  • Active Directory
[04]Method
  1. 01

    Baselined the last 30 days of alerts; ranked by FP rate × analyst-minutes-per-alert.

  2. 02

    Built a decision tree per detection type (auth, network, host, identity) so triage is checklist-driven.

    Fig · 02 · Triage queue · disposition dispositionsLIVE
    Sentinel · Alert queueAssigned to op-sds-04
    TimeAlertSevDisposition
    09:41Impossible travel · a.patelHIGHTrue positive
    09:38PowerShell EncodedCommandHIGHContained
    09:22Failed logon burst (svc_backup)MEDBenign · pw rotation
    09:04New OAuth grant · unknown appHIGHTrue positive
    08:51Login from TOR exitMEDBlocked by CA
    08:12AV: EICAR test fileLOWBenign · training
  3. 03

    Wrote SPL for behavior-based signals — impossible travel with device-context guard, vssadmin canaries, DGA entropy.

    Fig · 03 · Detection editor · tuned SPL ruleLIVE
    detection-rules/impossible_travel.splSPL · tuned
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    index=azuread sourcetype="SigninLogs"
| where ResultType==0
| eval prev_ip=prev(IPAddress),
       prev_time=prev(TimeGenerated),
       prev_geo=prev(Location)
| eval dist_km = geodistance(prev_geo, Location)
| eval mins    = (TimeGenerated - prev_time)/60
| where mins < 60 and dist_km > 500
| where UserPrincipalName !in (svc_travelers_lookup)  // tuning
| stats count by UserPrincipalName, prev_ip, IPAddress
  4. 04

    Re-emulated with Atomic Red Team and measured every detection end-to-end.

[05]Outcome

MTTD cut from 14 min to 3 min in lab; six new correlation searches; false-positive rate down 61%.

[06]Lessons learned
  • Every noisy alert costs the whole SOC — killing FPs is a force multiplier.
  • Decision trees make junior analysts as fast as senior ones on the top 80% of alerts.

Next case · NET-022

Packet inspection — hunting a C2 beacon inside noisy egress

Open →