How Risk Scores Are Built

Every UEBA product produces a score. Analysts triage by it, dashboards rank by it, and executives ask why it went up. Almost nobody can explain what it means, including many of the people who built it.

How a score is assembled

The typical pipeline has four stages.

Per-feature deviation. Each feature is compared against the baseline and converted to a deviation measure — a z-score, a percentile, a rarity value. This is the only stage with a clear statistical meaning.

Normalisation. Deviations across features with different scales are mapped onto a common range so they can be combined. The mapping function is a design choice and it is rarely documented.

Weighting and combination. Features are weighted and aggregated. Weights are usually set by the vendor, sometimes tunable, and almost never derived from your data. This stage is where most of the arbitrariness enters.

Presentation mapping. The combined value is mapped onto 0–100, frequently with a non-linear curve chosen so that the distribution "looks right" on a dashboard.

By the final stage, the number has passed through at least two arbitrary transformations. It is an ordering, not a measurement.

What the number does and does not mean

It does rank entities relative to each other under one model at one time. If A scores 90 and B scores 40, A's activity was more unusual by that model's definition. That is genuinely useful for prioritising a queue.

It does not express a probability. A score of 85 does not mean an 85 percent chance of anything. There is no calibration behind it unless the vendor has done calibration work and told you so, which is rare.

It does not compare across time. If your environment changed, today's 70 and last quarter's 70 are not the same claim.

It does not compare across entity types. A service account scoring 80 and a user scoring 80 have been evaluated by different models against different baselines.

It does not indicate severity. Unusual is not the same as harmful. A person doing something rare and entirely legitimate scores identically to a person doing something rare and serious.

Risk accumulation

Many products maintain a running risk value that accumulates over time — several moderate anomalies push an entity above a threshold.

The intent is reasonable: an attacker generates a sequence of individually unremarkable events. The implementation causes two problems.

Accumulation without decay guarantees that every entity eventually reaches the threshold. Decay rates are therefore load-bearing and usually undocumented.

Accumulation of correlated events double counts. One action can generate several features that all deviate, and if they are treated as independent evidence the score inflates on a single fact.

Ask a vendor directly how correlated contributions are handled. The answer is informative about how seriously the model was built.

Use the contributions, not the score

The practical discipline is simple and consistently ignored.

A score is for ordering the queue. Nothing else.

The contributing factors are for triage. "Accessed 14 systems outside normal role, transferred 3.2 GB to an external destination, first activity from this country" is something an analyst can evaluate. "Risk 87" is not.

Never present a score as a finding. In an HR or legal process it will be challenged, and it will not survive the question "what does 87 mean". Present the observations that generated it.

A product that cannot show per-feature contributions with the entity's baseline alongside is not usable for anything that might lead to a consequence for a person.

Thresholds

Choosing an alert threshold is choosing an alert volume, and it should be approached that way.

Set it from review capacity, not from the number. Decide how many entities per day a person can genuinely examine, then set the threshold to produce that many. This sounds crude and it is more honest than picking 80 because it is a round number.

Expect to move it. Thresholds set at deployment are wrong, because the score distribution shifts as the model matures.

Watch the distribution, not the count. A histogram of scores tells you whether the model is discriminating at all. A model producing a smooth unimodal distribution with no tail is not separating anything, however you set the threshold.

Reading the score distribution

More informative than any individual score, and quick to produce.

Plot a histogram of scores across all entities for a week. The shape tells you whether the model is doing anything.

A smooth unimodal distribution with no tail means the model is not separating anything. Every entity looks similar. No threshold will help, and the problem is features or data rather than scoring.

A distribution with a long thin tail is what a working model produces: most entities cluster, a few stand out.

A bimodal distribution usually indicates two populations sharing one model — service accounts and humans, or two regions with different working patterns. They need separate models.

A distribution that shifts week to week without environmental change means instability: retraining, a data source problem, or a feature whose scale is drifting.

Review it monthly. It takes minutes and it catches problems that individual alert review never surfaces.

Common false positives

Score inflation has recurring causes worth checking before treating a high score as behavioural:

Correlated features counting one action several times.

A single extreme feature dominating an otherwise ordinary profile, usually a volume feature affected by duplicate events.

Accumulated risk without decay, where an entity crossed the threshold through the passage of time rather than through anything recent.

Immature baselines producing high deviation because the model has little to compare against.

Data backfill creating an apparent volume spike for every entity at once.

Blind spots and assumptions

That the weights suit your environment. They were set on someone else's data.

That the score is stable. Model updates, retraining and data source changes all shift the distribution, usually without notice.

That higher means worse. It means more unusual, and the correlation between unusual and harmful is much weaker than the interface implies.

That analysts read the contributions. Under queue pressure they read the number. That is a workflow design problem, not an analyst problem, and it is fixed by making contributions the primary display rather than a detail view.