How Response Rules and the Confidence Engine work
Tell CyberSentry what to do when a Microsoft 365 account looks compromised: lock it, revoke its sessions, open a ticket or notify your team, with each rule able to run in Monitor first. The always-on Confidence Engine scores every finding and maps the score to the same actions.
What this page is
Response Rules is CyberSentry's IF/THEN layer. When a sequence of things happens in order for one user, the rule runs a set of effects: lock the account, revoke sessions, open a ticket, notify. The Response Rules card lists your rules, New Rule adds one, and the starter templates begin in Monitor, where a rule only records what it would have done.
The Confidence Engine row
The engine sits at the top of the list as a row of its own, named Confidence Engine, with its arming state beside the name (Monitor, Live or Disabled), an Auto badge and a line reading Always-on scoring with the number of recorded events. The toggle on the right only moves the engine between off and Monitor. Live is a separate, deliberate step on the engine's own page. Clicking the row opens that page.
Source filter
A Source filter on the Response Rules card narrows the list to the rules that reason over one kind of signal, such as Identity (M365). All sources shows every rule.
The table
Every rule shows its Name, Source, Sequence, Mode, Status, Scope and Priority. Mode and Status are two different things: Status says whether the rule evaluates at all (Enabled or Disabled), and Mode says whether a match acts (LIVE) or is only recorded (Monitor (dry run)). A rule with more than one ordered step reads Multi-stage in the Sequence column, and Scope reads All clients or the number of clients it is limited to.
Rules that mirror a built-in detector
Some rules mirror one of CyberSentry's built-in detectors for calibration and visibility. The built-in detector is still the one that actually alerts, so tuning such a rule is safe, and it cannot be armed to Live until the built-in detector is retired.
Disabled, Monitor, Live
Three states, picked at the top of a rule. Disabled means the rule is off and it neither runs nor records anything. Monitor records what the rule would have done. Live runs the effects for real, and Save waits for an arming confirmation first.
Browse
On this page
What a rule triggers and does
Triggers
The when half of the rule. The conditions must occur in the order you set, within the match window. The starter templates show the range: one flags a non-interactive token redemption that the session-fork detector caught, and contains the account. Another looks for ten or more failed sign-ins followed by a success from an unexpected country, and escalates to your team.
Actions
The then half. Actions run from top to bottom, and you add them one at a time or drop in a preset bundle with Add a bundle. Open ticket carries its own Ticket board field. The Stop if an action fails switch halts the remaining actions on the first failure, so a rule does not report contained when only half of it ran.
Match window, scope and guards
Match window and priority
Match window (minutes) is how far apart the steps can be and still count as one sequence. Priority decides which rule runs first when several match the same event: lower numbers run first.
Scope and the cap
Applies to is All clients or Manual, and Manual reveals a client picker. Auto-action cap (accounts / hour) is the most distinct accounts one rule will auto-act on in a rolling hour, a safety cap so a misfiring rule cannot lock everyone at once. Left blank it reads Unlimited.
The guards
Stop processing further rules on match makes lower-priority rules skip that same account when this rule fires for it. Never act on break-glass / service accounts keeps a rule away from a client's designated emergency-admin accounts. Never act on our own (MSP) access keeps it off your own team's access to the client. A warning also lists clients in a rule's scope that have no break-glass account set.
Recent activity and testing a rule on history
Recent activity
A rule's own ledger. In Monitor every match lands here showing what the rule would have done. A Disabled rule records nothing, and the card says so.
Test against history
A dry run of the rule over one client's recent activity, to preview its blast radius before you enable it. Pick a client and a window, then press Run test. The result says how many accounts the rule would have acted on, how many were blocked by the guards and the cap, and nothing is locked, revoked or notified by a test.
Arming the engine, thresholds and bands
Arming the engine
The engine's own page opens from its row on Response Rules. Promote to Live is the deliberate second step that lets the bands act for real, and it asks you to confirm. When the engine is Live the same button becomes Return to Monitor.
Auto Response Thresholds
The glass box: how a signal becomes a confidence percentage, and what that percentage fires. Every alert shows its math. On an instance that has not tuned anything, the page says it is using the built-in defaults.
Confidence Bands
A band is a threshold plus a set of actions: when an alert's confidence reaches a band, its actions run. Bands are evaluated highest-first. Each band lists what it fires, such as Lock account, Revoke sessions, Open ticket or Notify, and can be set to Require corroboration before it acts. Enabled turns a band off without deleting it.
Factor weights
Factor Weights
Each piece of evidence carries a weight, and the weights of everything observed for one event add up to the confidence percentage. The factors are grouped as Location / network, Token / session, Action stream, Device and Context / amplifiers, and the Strong pill marks the ones meant to move the needle hard. Negative weights are dampeners, which pull the score down as evidence against. The Enabled box turns a factor off without deleting it.
Would-have-fired observations
Would-Have-Fired Observations
While the engine is in Monitor, every confidence evaluation records the band it would have hit. The Would-Have-Fired Observations table shows how often each band would have fired, for how many accounts and at what average confidence, so you can watch where the bands sit before promoting anything to Live.
Why it works this way: How Response Rules and the Confidence Engine work
The triage pass never sends a notification, for any alert. It clears noise, annotates, and raises the alert for a human, for Response Rules and for the Confidence Engine to read. The one thing it opens is a ticket: when its verdict on an open alert is escalate, it opens that alert's ticket, once.
Besides that ticket, two doors can act on a pattern on their own: a response rule armed past Monitor that carries an Open ticket or Notify action, and the Confidence Engine promoted to Live, whose bands carry the same action keys. An alert gets only one ticket, whichever door fires first.
So a fresh or lightly-armed instance opens no tickets and sends no real-time push at all. That is the intended default rather than a gap, and the alert stays on the Alerts list, on its own detail page, and in the periodic Cleared-by-Elise digest, notified or not.
Monitor is not a half-armed Live. A rule in Monitor evaluates every match and writes what it would have done into its own Recent activity; the engine in Monitor writes the band it would have hit into Would-Have-Fired Observations. Nothing is locked, revoked, ticketed or notified from either.
A band's actions and a rule's actions are the same set of effects written two ways. A rule lists them in order in its own Actions card; a band names them as chips beside a confidence threshold.
Editing a band changes what happens at that confidence level across the whole instance, because the engine is instance-wide. Response rules are the scoped half: All clients, or a picked list.
The Activity card on the Confidence Engine's own page lists the engine's recent firings, one line each, with the time, the user, the client and an outcome badge. Would run means Monitor mode recorded what it would do and did nothing. Executed means Live ran the actions. Blocked, followed by a check name in brackets, means a safety check stopped that firing, and the name is shown as written. In Monitor, Blocked can carry any of these names: blastRadiusCap (too many accounts for that client already acted on in the past hour), systemicEvent (many accounts show the same signal), belowArmedBand (the firing is below the one band Live is armed for), containmentFloorConfidence or containmentFloorCorroboration (not enough confidence or proof to lock), eventStale (the event is too old), mspAccess (your own team reaching the client) and breakGlass (an emergency, staff or excepted account). In Live, only eventStale, mspAccess, breakGlass and blastRadiusCap on a band with no destructive action show as Blocked. When Live refuses a containment for belowArmedBand, systemicEvent, a containment floor or the blast-radius cap, the line shows Executed, and the reason is kept in the record only.
Questions this page answers
How does the confidence score work (Factor Weights)?
Each piece of evidence - impossible travel, a token replayed, a suspicious OAuth consent, an MFA change, and so on - carries a weight. The weights of everything observed for one event add up to the confidence %. "Strong" factors move the needle hard; negative weights are dampeners (evidence AGAINST, e.g. a familiar device). Every alert shows its own math, so you can see exactly which signals produced its score and tune the weight of any factor.
What are Confidence Bands?
A band maps a confidence threshold to a set of actions: "at ≥ X% confidence, fire these." Bands are evaluated highest-first, so the strongest band that an alert reaches wins. Each band lists the actions it fires (raise alert, notify, open ticket, revoke sessions, lock account, …) and can require corroboration - a second independent signal - before it acts. Editing a band changes what the whole fleet does at that confidence level; the seeded defaults are conservative.
Why wasn't I notified about this alert?
Elise's triage sends no real-time alert push of its own. If her verdict on an open alert was escalate, she opened its ticket, and your team got the new ticket bell and email. An alert push comes from a Response Rule armed with the "Notify" action, or from the AUTO engine once promoted to Live. A fresh instance, or one with the engine still in Monitor, sends no alert pushes until you arm one of those. A cleared alert notifies no one. The alert stays on the Alerts list, this page, and the Cleared-by-Elise digest either way.
What does "Blocked" mean in the Activity list?
Each line in Activity is one time the confidence engine fired. "Would run" means Monitor mode recorded what it would do and did nothing. "Executed" means Live ran the actions. "Blocked (code)" means a safety check said Live would NOT act on that firing, so nothing was done. The code is the name of that check, shown as written. For example: blastRadiusCap (too many accounts in one hour), systemicEvent (many accounts show the same signal), belowArmedBand (the firing is below the armed band), containmentFloorConfidence or containmentFloorCorroboration (not enough confidence or proof to lock), eventStale (the event is too old), mspAccess or breakGlass (your own team or an emergency account).
Was this helpful?