Compliance Evaluation Lifecycle
Once you've authored rules, grouped them into a baseline, and assigned that baseline to a collection, two questions come up straight away: when do machines actually get the rules, and when do they re-check? This page answers both — the events that push a baseline to agents, the timer that re-evaluates it, how a baseline is withdrawn from a machine that should no longer run it, and how machines that were offline at the time catch up when they reconnect.
Start with Compliance & Baselines for the model — rules, baselines, assignments and the per-rule states. This page is the operational companion: timing and delivery, not authoring.
Push and evaluate: two halves of one cycle#
Compliance works on a push-then-evaluate model. The server pushes a baseline's rule bodies down to each agent, and each agent then evaluates those rules against itself locally and reports a per-rule state back. The two halves are deliberately decoupled:
| Half | Who drives it | What travels |
|---|---|---|
| Push (delivery) | The server, on the events below | The full rule set — detection and remediation scripts — cached on the agent. |
| Evaluate (execution) | The agent, on its own timer and on an Evaluate Now ping | Just a signal to run; the agent evaluates from its cached copy and reports results up. |
Because the agent keeps its own cached copy of the rules, a machine can keep evaluating on schedule even if it briefly loses contact with the server — and the server only needs to re-push when something actually changes.
When a baseline is pushed to agents#
The server pushes rule bodies to agents on the occasions below. Together they cover both "the assignment changed" and "the fleet changed" — so a machine ends up with the right rules no matter which one moved:
| Trigger | What happens |
|---|---|
| You create or edit an assignment | Linking a baseline to a collection (or changing that link) pushes the baseline to every currently-connected member of that collection. |
| You edit a rule's scripts or auto-remediate flag | Changing a detection or remediation script — or flipping autoRemediate — re-pushes every assignment whose baseline contains that rule, so agents don't keep running a stale copy. |
| You change a baseline's rule list | Adding rules to a baseline or removing rules from it re-pushes every assignment of that baseline to the connected members of its collections, so machines switch to the new list instead of continuing to run (and, with auto-remediate on, enforce) the old one. Renaming a baseline, or saving it with the same rule list, pushes nothing. |
| An agent connects | When an agent establishes its connection to the server, the server walks every collection the agent belongs to and pushes each of those collections' assignments to it. This is how a machine that was off during a change catches up. |
| An agent joins a dynamic collection | When a machine newly matches a dynamic collection that already has a baseline assigned — for example it comes online and lands in All Online Agents — the server pushes that collection's assignments to it right away, without waiting for a reconnect. |
| Collection membership changes | When a collection that has a baseline assigned gains or loses members — you add or remove machines in a static collection, a dynamic collection is refreshed or its filters are edited, or the collection is deleted — every connected machine that joined receives the collection's assignments and rules, and every connected machine that left is told to stop running them (see below). Machines that stayed in the collection are left alone. |
| Evaluate Now | An operator-initiated re-check from the console. This is a lightweight "evaluate now" signal rather than a full re-push (the agent already holds the rules). |
Dynamic collection membership is recalculated as agents check in, so a machine can enter a collection like All Online Agents after it has already connected. Without a dedicated trigger, that machine would hold no assignment and its Compliance section would sit on No Compliance Results until the next reconnect. The join trigger closes that gap: the moment an agent enters an already-assigned dynamic collection, its baselines are pushed and it starts evaluating.
Pushes only reach agents that are online at that moment. An agent that is offline when an assignment or rule changes is simply skipped — it picks the change up automatically the next time it connects, via the agent connects trigger above. Re-pushing rules an agent already has is harmless, so overlapping triggers never cause duplicate work.
When a baseline stops running on a machine#
Because each agent keeps a cached copy of its assignments, removing an assignment on the server has to be actively delivered to the machine as well — otherwise it would keep evaluating, and with auto-remediate on keep fixing, a baseline you no longer want applied. Athena does this by sending the agent the complete list of assignments it should hold right now; the agent drops every cached assignment that is not on that list and never evaluates or remediates it again. An empty list withdraws everything.
| You do this… | …and these machines stop running the baseline |
|---|---|
| Delete an assignment (Compliance → Assignments) | Every member of the assignment's collection. |
| Delete a baseline | Every member of every collection the baseline was assigned to — its assignments are removed with it. |
| Remove a machine from an assigned static collection | That machine (unless another collection it belongs to carries the same baseline). |
| A machine stops matching an assigned dynamic collection | That machine, as soon as the collection's membership is re-evaluated — when it checks in, or when the collection is refreshed or its filters are edited. |
| Delete a collection that has a baseline assigned | All of its former members. |
Only connected machines receive the list at that moment. A machine that was offline is not missed: every time an agent connects, the server sends it the complete list first (even when it is empty) and then pushes the assignments it should have — so a baseline withdrawn while the machine was powered off stops running on its next connect.
When an agent drops a withdrawn assignment it writes an informational line to its
agent log naming the assignment and the server that withdrew it,
ending in removed, no longer evaluated. Its compliance results for that baseline
stop updating from then on.
Withdrawal is carried out by the Windows agent 1.2.96 or later. Older Windows agents and the current Linux agent ignore the list and keep evaluating the cached copy of a withdrawn baseline — update your Windows agents so deletions take effect on the machines. Newer agents only ever drop assignments when a server sends the list, so running a new agent against an older server never loses assignments.
If a machine reports to more than one Athena server, each server's list is treated as the complete set. Keep the same compliance assignments on every server the machine talks to; otherwise one server's list will remove the assignments that only the other server holds.
The re-evaluation timer#
After the initial push, the agent owns its own re-evaluation cadence — there is no server-side
clock ticking on behalf of each machine. Every assignment carries an
evaluationIntervalMinutes value (set when you create the assignment; it defaults to
60 and must be a positive number). The agent re-runs each assigned baseline against
itself once that interval elapses, then reports the fresh per-rule states.
Each assignment keeps its own interval and its own "last evaluated" timestamp, so a machine that belongs to several collections evaluates each baseline independently on its own schedule — a fast-cadence security baseline and a slower inventory baseline don't interfere with each other.
Evaluations run one at a time on the agent. Because compliance checks are PowerShell — and the same machine may also be running commands, scans and deployments — the agent deliberately serialises them to stay non-disruptive rather than running them all at once.
Compliance:LocalSchedulerEnabled
The agent-side re-evaluation timer is on by default. It can be turned off with the
Compliance:LocalSchedulerEnabled setting (set it to false) as an opt-out
for testing or troubleshooting — when disabled, agents stop re-evaluating on their own timer and
only respond to an explicit Evaluate Now. Leave it on for normal
operation.
Forcing a re-check with Evaluate Now#
You don't have to wait for the interval to come around. Evaluate Now tells agents to re-run a baseline immediately and is available in two places in the console:
| Where | Scope |
|---|---|
| Compliance → Assignments, per assignment row | Re-checks that baseline across every machine in the assigned collection. |
| Agent Details → Compliance section, per agent | Re-checks the selected machine's assigned baselines on that machine only. |
Evaluate Now sends a lightweight "evaluate now" signal — the agent already holds the rule bodies from an earlier push, so nothing heavy travels. Online agents receive the ping and re-evaluate right away; the console reports back how many machines were pinged.
How offline machines catch up#
Compliance is built to tolerate machines that are powered off, asleep, or briefly disconnected — nothing is silently lost:
| Situation | What happens |
|---|---|
| Assignment or rule changed while the agent was offline | The push skips the offline agent; the change is delivered automatically on its next connect. |
| Assignment or baseline deleted (or the machine left the collection) while the agent was offline | On its next connect the agent receives its complete assignment list and drops the withdrawn baseline, so it is never evaluated or remediated there again. |
| Evaluate Now clicked while some targets were offline | Online targets are pinged immediately; each offline target has the request queued, and the queue is drained — the machine re-evaluates — the next time it reconnects. The console shows how many were dispatched now versus queued for later. |
| Agent can't reach the server to report a result | The result is held locally on the agent and delivered on the next successful connection, so a scheduled evaluation is never dropped just because the link was down. |
The net effect: whether a machine was online for the change or not, it converges on the current baseline and the current results without any manual re-push.
Telling stale evaluations apart#
Because pushes and evaluations are decoupled, it helps to know which version of a rule a machine actually ran. Every rule body is fingerprinted with a hash when it's created or changed, and each result an agent reports carries the hash of the script it ran. When you edit a rule, the re-push and the machine's next evaluation update that fingerprint — so the console can flag a result that was produced against an older version of a rule, rather than leaving you to guess whether a change has propagated. See Compliance rules for how the fingerprint is stamped.
Quick answers#
| Question | Answer |
|---|---|
| I just assigned a baseline — when does it run? | Immediately on every online member of the collection, then again each evaluationIntervalMinutes. |
| A machine that just came online now shows results — why? | Connecting (and joining any assigned dynamic collection) pushes its baselines and kicks off an evaluation. |
| I edited a rule — do I need to re-assign? | No. Editing the detection/remediation script or the auto-remediate flag re-pushes the affected assignments automatically. |
| I added or removed rules in a baseline — do I need to re-assign? | No. A changed rule list is re-pushed to the connected members of every collection the baseline is assigned to; offline members get it on reconnect. |
| I deleted an assignment — does the machine stop remediating? | Yes, on Windows agent 1.2.96 or later: connected members drop it immediately, offline members on their next connect. See When a baseline stops running. |
| How do I force a re-check right now? | Use Evaluate Now on the assignment row or on a single agent's Compliance section. |
| A target was offline when I clicked Evaluate Now — is it lost? | No. It's queued and re-evaluates on reconnect. |
| Why isn't a machine evaluating at all? | Confirm it's online, that it's a member of an assigned collection, and that Compliance:LocalSchedulerEnabled hasn't been turned off. |