Atomic Red Team: simulating ATT&CK and finding logging gaps

The alert that never came

A red team operator opens a PowerShell window on a Windows workstation and types a single line. It downloads a script into memory and runs a credential-dumping routine modelled on Mimikatz, the tool that has appeared in nearly every serious intrusion of the past decade. The command finishes in under a second. The operator waits. And waits. No alert fires in the security operations centre. No email lands in an analyst's inbox. As far as the organisation's defences are concerned, nothing happened at all. That silence is the whole point of the exercise. The operator was not an attacker but a defender running a test from Atomic Red Team, an open-source library of small, self-contained attack simulations built and maintained by the security firm Red Canary. Each test reproduces one specific adversary behaviour so a team can check a simple, uncomfortable question: if a real attacker did this, would we see it? The answer, far more often than most organisations would like, is no.

What "atomic" actually means

The word "atomic" is doing precise work here. An atomic test is the smallest useful unit of attacker behaviour: a single action, executed in isolation, mapped to a single technique. Rather than running a sprawling, multi-stage break-in, you fire one narrow probe. Did the system generate a log for it? Did that log reach your monitoring platform? Did a detection rule turn it into an alert? The project grew out of a specific frustration. According to Red Canary's own account, most security teams lacked the internal expertise to simulate particular adversary tactics on demand, so they had no cheap way to know whether their tools worked. Atomic Red Team debuted at the Carbon Black User Exchange in October 2017 with roughly 75 tests, Red Canary recounts in a first-year retrospective. Michael Haag originally pitched the idea internally under the codename "Charlie," and the concept was inspired by a 2016 BruCon talk by Chris Nickerson and Chris Gates on automated adversary simulation. The security researcher Casey Smith became one of its most visible early contributors. The library has since grown to more than 1,800 tests contributed by a global community. Each test is a small chunk of structured text in YAML format, a human-readable way of storing data. A single atomic contains a plain-language name and description, the platforms it runs on (Windows, macOS, or Linux), any configurable inputs, the exact command to execute, and, crucially, a cleanup command that undoes the change afterward. That last piece matters: these tests genuinely modify the system, creating files, spawning processes, and touching the registry, so each one ships with instructions to put things back. Take technique T1059.001, the PowerShell entry in MITRE's catalogue. The public documentation for it lists 22 separate atomic tests, ranging from a benign "PowerShell spawning cmd" up to downloading and running the reconnaissance tool BloodHound or a Mimikatz variant straight into memory. Each is deliberately readable, so a defender can look at the command and understand exactly what telemetry it ought to produce before they run it.

The map underneath: MITRE ATT&CK

Atomic Red Team is only as coherent as the framework it is pinned to, and that framework is MITRE ATT&CK. ATT&CK (the letters stand for Adversarial Tactics, Techniques, and Common Knowledge) is a free, globally used knowledge base of adversary behaviour "based on real-world observations," in MITRE's phrasing. It is not a theory of how attacks might work; it is a catalogue of how they actually have. The catalogue is organised in two layers. Tactics are the attacker's goals, the "why" of a given move: gaining initial access, escalating privileges, evading defences, moving laterally, exfiltrating data. Techniques are the specific "how." As of version 17, released in April 2025, the Enterprise matrix holds 14 tactics, 211 techniques, and 468 sub-techniques, and that release added an ESXi platform to cover attacks on VMware's hypervisor. The framework is versioned and revised roughly twice a year as new adversary tradecraft is observed in the wild. This is what makes the pairing powerful. ATT&CK gives you a shared vocabulary and a checklist of what attackers do; Atomic Red Team gives you a runnable test for a large fraction of those entries. When someone says an environment has "coverage for T1003, credential dumping," everyone in the room knows precisely what is meant, and can run the same test to verify it. A common misconception is worth clearing up here. Colouring an ATT&CK matrix green because you have written a detection rule for a technique is not the same as detecting that technique. A rule can be misconfigured, it can watch a log source that was never actually being collected, or it can be so narrow that it misses obvious variants. The green square is a claim. An atomic test is the experiment that checks the claim.

Running one, from command to cleanup

In practice, most teams do not run these commands by hand. They use Invoke-AtomicRedTeam, a PowerShell module that wraps the whole library and handles setup, execution, and teardown. The workflow is disciplined by design.

A defender typically starts by reading the test without running it:

Invoke-AtomicTest T1059.001 -ShowDetails

This prints exactly what each test will do. Next comes a prerequisite check, because some tests need a particular file or tool present first:

Invoke-AtomicTest T1059.001 -CheckPrereqs

Then a single, specific test is executed, often by number so nothing unexpected runs:

Invoke-AtomicTest T1059.001 -TestNumbers 1

And finally, the cleanup that removes whatever the test created:

Invoke-AtomicTest T1059.001 -TestNumbers 1 -Cleanup

The whole cycle usually takes minutes. That speed is deliberate: teams are meant to run tests often, in a tight loop, rather than treating security validation as an annual event. The project is blunt about the risks, though. Its own documentation warns that "executing atomic tests may leave your system in an undesirable state" and that the operator is responsible for understanding each test and having permission to run it. This is not something to fire off against production without authorisation. Because a test can legitimately download a hacking tool or dump credentials, running it in the wrong place is indistinguishable from a real intrusion, and could trip incident response, break something, or violate policy.

Finding the gap, and naming it correctly

Here is where the work gets interesting, and where the discipline separates competent teams from careless ones. When you run an atomic test and no alert appears, you have learned something, but not yet what. There are two very different failures hiding behind that silence, and confusing them wastes weeks. The first is a collection gap: the telemetry was never generated or never made it to your monitoring platform. The second is a detection gap: the data arrived, but no rule turned it into an alert. As practitioners writing about the workflow put it, if a test's exit code is zero and yet no process or file telemetry appears anywhere, the problem is collection or filtering, not a failed atomic. If the telemetry is sitting there in the logs but nothing fired, the problem is your detection logic. The remedy is different in each case: one sends you to fix logging pipelines and agent configuration, the other sends you to write or tune a rule. The classic collection gap on Windows is a case study in itself. Windows can log every new process it starts as Security Event ID 4688. But by default that event's most valuable field, the full command line, is empty. It only populates after an administrator explicitly enables a separate Group Policy setting, and even then, 4688 carries no file hashes, no parent-process command line, and only reusable process IDs that are awkward to correlate. This is why so many teams deploy Sysmon, a free Microsoft tool whose Event ID 1 captures the full command line, the parent process, cryptographic hashes, and a unique process identifier in a single structured record.

An atomic test makes this abstract problem concrete in seconds. Run a PowerShell atomic whose whole malicious character lives in its command-line arguments, then go look at what your logs captured. If all you see is "powershell.exe started" with no arguments, you have just watched your own blind spot appear in real time, and you know exactly which setting to change. That is a far more persuasive artefact to bring to a change-management meeting than a vendor's slide claiming full coverage.

The 79 percent problem

How common are these gaps? The numbers are sobering, and they come from measurement rather than anecdote. In its fifth annual State of SIEM Detection Risk report, published in June 2025, the vendor CardinalOps analysed production data from hundreds of real enterprise environments running major security information and event management (SIEM) platforms, the central systems where logs are collected and alerts are generated. The finding: on average, these organisations had detection coverage for just 21 percent of the techniques in MITRE ATT&CK, leaving 79 percent uncovered. That figure is not a story about missing data. The same report found that these SIEMs were ingesting an average of 259 distinct log types and nearly 24,000 log sources, enough raw telemetry, by CardinalOps' assessment, to detect more than 90 percent of ATT&CK techniques. The organisations were sitting on the information needed to catch the vast majority of attacker behaviours and simply were not turning it into working detections. On top of that, the report estimated that around 13 percent of the detection rules that had been written were broken outright, silently failing due to misconfigured fields or missing log sources. The gap between 90 percent of techniques being detectable and 21 percent being detected is the entire reason a tool like Atomic Red Team exists. It is the difference between having a smoke detector on the shelf and having one wired to the ceiling with a fresh battery. The report attributed much of the shortfall to the manual, error-prone nature of detection engineering: rules are written by hand, rarely tested end to end, and quietly rot as environments change. Atomic testing is one of the few cheap, repeatable ways to catch that rot before an attacker does.

What atomic tests cannot tell you

It would be a mistake to treat a green run of the atomic library as a clean bill of health, and the project's honest advocates say so. The most important limitation is structural: atomic tests are, by definition, isolated. Each one exercises a single technique out of context. Real intrusions are chains, one technique feeding the next over hours or days, and a defence can catch every individual link in a lab while still missing the pattern of the whole attack. As one comparison of open-source emulation tools notes, Atomic Red Team tests do not emulate any specific campaign unless an operator deliberately chains them together, and some tests are simply incomplete. This is the line between two related but distinct activities. Atomic testing validates that your controls can see a given behaviour. Adversary emulation stitches those behaviours into a realistic campaign that mimics a specific threat group's playbook from start to finish. For that heavier work, teams often turn to MITRE's own CALDERA, a more elaborate platform that runs an agent on target machines and orchestrates multi-step operations, and which even offers a plugin to pull in Atomic Red Team's techniques. CALDERA is more powerful and considerably more demanding to operate; the two tools complement rather than replace each other. And neither replaces a human. Red Canary itself concedes that skilled red teams "will come up with clever attacks designed to subvert your organisation's specific defensive controls," the kind of bespoke, improvised tradecraft no library of predefined tests can anticipate. Automated testing tells you whether you can catch the known moves. It says nothing about the move nobody has catalogued yet. There is also an operational trap worth naming. Because these tests generate genuine attacker telemetry, a team that runs them without coordinating with its own analysts can waste real incident-response hours chasing its own footprints, or worse, train its analysts to reflexively dismiss the very behaviours they should be hunting. The point is to close gaps, not to manufacture noise.

What to watch

The deeper value of atomic testing is cultural more than technical. It converts security from a matter of assertion into a matter of evidence. Instead of trusting that a control works because a vendor said so or because a rule exists somewhere in the SIEM, a team can run a two-minute test and watch, in its own logs, whether the thing fired. Detection engineering starts to look less like faith and more like science: form a hypothesis, run the experiment, read the result, fix what broke, run it again. The open question is whether that loop can be made to keep pace with the threat. CardinalOps found coverage crawling upward by only about two percentage points a year while ATT&CK itself keeps expanding, adding new platforms and techniques with every release. Closing gaps by hand, one atomic at a time, may not scale against an attack surface that grows faster than the people defending it. The most interesting developments to watch are the attempts to automate the whole cycle, continuously running tests, measuring coverage, and even drafting detection rules, so that the silence after a simulated attack becomes something a team notices in minutes rather than discovers, too late, during the real one.

---

Sources

1. Red Canary, Atomic Red Team GitHub repository (README), 2026.
2. Red Canary, Red Canary Introduces Atomic Red Team Testing for Defenders, 2017.
3. Red Canary, One Year of Atomic Red Team! Looking Back and Ahead, 2018.
4. MITRE, MITRE ATT&CK homepage and Enterprise matrix, 2025.
5. MITRE, ATT&CK Updates: April 2025 (v17 release notes), 2025.
6. Atomic Red Team, T1059.001 Command and Scripting Interpreter: PowerShell tests, 2026.
7. Red Canary, Invoke-AtomicRedTeam GitHub repository, 2026.
8. Help Net Security, Enterprise SIEMs miss 79% of known MITRE ATT&CK techniques (CardinalOps 5th annual report), 2025.
9. iTech Guides, Command Line Event Log: Enable Windows 4688, Sysmon & PowerShell Logging, 2024.
10. GoCodeo, How to Use Atomic Red Team for MITRE ATT&CK-Based Threat Testing, 2025.
11. Picus Security, A Data Driven Comparison of Open Source Adversary Emulation Tools, 2024.
12. Red Canary, Comparing open source attack simulation platforms for red teams, 2020.