Tuning a UEBA Deployment

The first weeks of any deployment produce thousands of anomalies, nearly all legitimate. The instinctive response — raise thresholds until it is quiet — reduces noise and detection together, and leaves a system that sees only the most extreme activity.

There is a sequence that works better, and it takes a quarter.

The order

One. Verify the data before touching the model. Is every source delivering? Is entity resolution correct? Are timestamps sane? Is coverage what you think? A substantial fraction of apparent model noise is data noise, and tuning the model around it entrenches the problem.

Two. Identify approved workflows. Every organisation has legitimate processes that look like violations: a finance job exporting nightly, an integration account moving bulk data, a backup process touching everything. Enumerate them and exclude them explicitly, with documentation and a review date.

This step removes more volume than any threshold change and costs no detection.

Three. Fix the features. Which features are contributing most of the deviation? Frequently one badly chosen feature — an hour-of-day feature on a distributed workforce, a volume feature on a source with duplicate events — generates the majority of the noise.

Four. Add context before suppressing. An alert that cannot be interpreted looks like noise. Enrichment frequently converts an apparently useless detector into a useful one without changing the detection at all.

Five. Correlate. One action producing twenty alerts is one finding. Correlation into entity-level or session-level incidents reduces volume without reducing coverage.

Six. Only then adjust thresholds, per detector rather than globally.

Most deployments start at step six.

Set the threshold from capacity

Decide how many entities a reviewer can genuinely examine daily, with attention. Then set thresholds to produce approximately that number.

This sounds crude and it is more honest than choosing a round number. A queue larger than review capacity is not a detection system; the alerts that matter are invisible inside it.

For most organisations the honest figure is single digits to low tens per day.

Exclusions need governance

Exclusions accumulate and become permanent holes nobody remembers creating.

Document each one: what is excluded, why, who approved it, when it expires.

Set expiry dates. An exclusion that must be renewed is reviewed; one that persists silently is forgotten.

Review quarterly. Business processes change and exclusions outlive them.

Never exclude an entity entirely. Exclude a specific behaviour for a specific entity. Blanket entity exclusions are how privileged accounts end up unmonitored, and they are almost always created for a good reason during a noisy week.

Count them as a metric. A rising exclusion count means either the business is changing or the detections are too blunt.

What not to do

Do not raise thresholds globally. It hides the fact that one detector is responsible for most of the volume.

Do not close alerts in bulk. The habit forms in a week and does not switch off for the one that matters.

Do not tune to zero. A detector producing nothing is either perfectly preventive or dead, and you cannot tell which.

Do not tune during an unstable period. A migration or a reorganisation produces genuine anomalies; tuning them away means the model will not see the next one either.

Knowing when it is tuned

The queue is fully reviewed daily. The binding constraint.

Precision is measurable and stable. You know what fraction of alerts are something, because adjudication is captured.

Volume does not spike on predictable dates. Month-end and holidays are handled.

Analysts can explain most alerts within a minute. If they cannot, the problem is enrichment, not the model.

New detectors can be added without breaking the budget. Headroom means the deployment can grow.

A tuning log

Tuning changes accumulate and nobody remembers why any of them were made. A log costs nothing and prevents a specific, common failure: reversing a change that was made for a good reason.

Record for each change: date, what was changed, why, who approved it, what the expected effect was, and a review date.

Then check the effect. A week later, did volume move as expected? Tuning changes frequently do something other than intended, and without the check nobody notices.

Group changes. Making six adjustments at once means you cannot attribute the result to any of them. One change at a time, with a week between, is slower and it is the only way to learn what works.

Review the log quarterly. Patterns emerge — one source generating repeated exclusions, one detector requiring constant adjustment — that are invisible change by change.

Common false positives

The categories that dominate an untuned queue, in rough order of volume:

Approved automation — backup, integration, monitoring and deployment accounts doing their jobs.

Bulk operations by design — reporting, data warehouse loads, archive processes.

Administrative breadth, where wide access is the role.

Organisational change — rollouts, migrations, reorganisations.

Calendar effects — month-end, holidays, audit windows.

Cold-start entities with immature baselines.

Data defects presenting as behaviour.

Working through these in order removes most volume before any threshold is touched, and none of it costs detection.

Blind spots and assumptions

That tuning is a phase. It is continuous. Every new source, every organisational change and every product update disturbs it.

That low volume means good tuning. It may mean broken data. Volume monitoring per source distinguishes them.

That the vendor's defaults are a starting point. They are a starting point for a different environment.

That analysts will report a bad queue. They adapt to it, quietly, by dismissing faster.