Most SOCs have a detection backlog. New telemetry sources arrive, new threats emerge, and the team writes rules faster than they tune them. Within six months, the rule corpus is unreliable and analysts stop trusting alerts.
A detection engineering workflow fixes this by treating detections like code: versioned, reviewed, tested, and instrumented.
The four-step loop
Every detection moves through four stages.
Stage 1: Hypothesis. Start with a specific attacker behavior, ideally mapped to a MITRE ATT&CK technique. "Detect lateral movement" is not a hypothesis. "Detect T1021.002 SMB lateral movement using non-admin accounts at off-hours" is.
Stage 2: Build. Write the rule in a version-controlled repository using a structured format like Sigma. Include the hypothesis, the data source, the logic, the expected false-positive rate, and a test case.
Stage 3: Test. Run the rule against historical data. Measure true positives, false positives, and coverage. Most rules need three to five iterations before they hit acceptable noise levels.
Stage 4: Instrument. Deploy with metrics attached. Every alert produced by the rule logs its disposition (true positive, false positive, benign). After 30 days, review the disposition mix and either tune, retire, or promote the rule.
The habits that matter
Three small habits separate detection programs that scale from ones that drown.
First, write the hypothesis before the rule. Detections built from a clear hypothesis are easier to tune, easier to document, and easier to retire when the threat shifts. Detections built from "this alert looks useful" almost always end up in the false-positive pile.
Second, set a coverage target per ATT&CK tactic. Aim for at least one tested detection per technique you consider in-scope. The visible coverage map drives conversations with leadership and clarifies which gaps matter most.
Third, retire detections aggressively. Most teams add detections and never remove them. A rule that has not fired in 90 days, or that produces 95 percent false positives, is technical debt. Kill it.
Where this shows up on CAS-005
The CAS-005 exam tests detection engineering inside the Security Operations domain. Performance-based questions ask you to read telemetry, identify the attacker behavior, and either write or evaluate a detection rule. The exam does not test you on a specific SIEM product, but it does expect you to know Sigma syntax and MITRE ATT&CK tactic/technique mapping cold.
Leave a Reply