Unsupervised Detection: Limits

Supervised learning needs labelled examples. In security you have almost none: confirmed insider incidents are rare, historical labels are unreliable, and the attacks you most care about are the ones you have not seen.

So UEBA is unsupervised by necessity rather than by design. Understanding what that constraint implies is the difference between realistic expectations and a disappointed deployment.

What unsupervised methods actually do

They model the structure of the data and score how well each observation fits.

Distance-based. How far is this point from its neighbours or from a cluster centre. Intuitive, and it degrades badly in high dimensions where distances converge.

Density-based. How dense is the region this point occupies. Local outlier factor and similar methods. Better behaved than distance in many cases.

Isolation-based. How few random splits are needed to isolate this point. Isolation forests are widely used because they are fast, need little tuning, and handle mixed feature types.

Reconstruction-based. Compress the observation and reconstruct it; large reconstruction error means the model has not seen this shape. Autoencoders. Expressive and opaque.

Probabilistic. Fit a distribution and score likelihood. Interpretable when the distributional assumption holds, which for security telemetry it usually does not.

All of them answer one question: how unusual is this relative to the bulk of the data. None answers whether it is bad.

The contamination assumption

Every unsupervised method assumes the training data is predominantly normal. Most implementations take a contamination parameter — the assumed fraction of anomalies in training.

Two problems follow.

The parameter is a guess. Set it too high and normal behaviour is treated as anomalous, inflating the baseline of what the model considers strange. Too low and genuine anomalies are absorbed as normal.

If the assumption is wrong, the model is wrong in a way nothing reveals. An attacker present throughout the training period is not an outlier; they are part of the distribution. The model has learned their behaviour as ordinary and will never flag it.

This is the same baseline poisoning problem that affects simple statistics, and complexity does not mitigate it.

The dimensionality problem

Anomaly detection degrades as feature count rises. In high-dimensional space, distances between points converge, density becomes uniform, and everything looks roughly equally unusual.

This is not a tuning issue; it is a property of the geometry.

Practical consequences: a dozen well-chosen features usually outperform sixty. Feature selection matters more than algorithm selection. Dimensionality reduction before scoring is frequently worth more than a better model.

The instinct to feed everything available into the model is precisely wrong, and it is what most default configurations do.

Evaluation without labels

The hard part. Without ground truth you cannot compute precision or recall directly.

What you can do:

Analyst adjudication. Have someone triage a sample and record the outcome. Slow, and it produces real labels over time. This is the only honest measure and it requires committing analyst hours.

Injected known behaviour. Generate synthetic activity resembling a known pattern and check the model surfaces it. Establishes sensitivity to what you simulated and says nothing about anything else.

Red team exercises. The most valuable evaluation available. An authorised exercise produces genuine adversarial behaviour with known ground truth. If the model does not surface it, that is decisive information.

Historical replay. Run the model over a period containing a confirmed incident. Rare, and worth doing every time it is possible.

Stability checks. Score distribution over time. A model whose output distribution shifts without a corresponding change in the environment is unstable.

What you cannot do: claim a detection rate. Any vendor quoting one is quoting it against their own simulated data.

Where unsupervised genuinely earns its place

Compromised credentials. The behavioural gap between an attacker and the account owner is large and multidimensional, which is exactly what these methods see.

Service account misuse. Low variance means small deviations are detectable.

Novel patterns. By construction, the only approach that can surface something nobody described.

Where it does not

Known attack patterns. A rule is cheaper and more precise.

Entities with short history. Nothing to model.

Low-and-slow within normal parameters. Invisible by design.

Anything requiring intent. Not a property of the data.

Configuring an unsupervised model sensibly

The defaults are rarely right, and three parameters carry most of the effect.

Contamination rate. The assumed fraction of anomalies in training. Setting it at a typical default of one to five percent asserts that up to one in twenty of your entity-days is anomalous, which is almost certainly too high. Lower values produce fewer, stronger anomalies.

Feature count. Fewer is better. Start with eight to twelve well-chosen features rather than everything available, and add only when you can say what gap the addition fills.

Training window. Long enough to include the cycles you care about, short enough to exclude behaviour that is no longer current. Sixty to ninety days for most populations, longer for service accounts whose behaviour is stable.

Then validate the output shape. Score the training data itself and look at the distribution. If a large fraction of ordinary entity-days score highly, the contamination assumption or the feature set is wrong, and no threshold will fix it.

Common false positives

Unsupervised methods have distinctive failure patterns:

High-dimensional flattening, where too many features make everything roughly equally unusual and the ranking becomes arbitrary.

Contamination set too high, which teaches the model that a slice of normal behaviour is anomalous.

Sparse entities scoring highly because the model has too few observations to characterise them.

Categorical features with high cardinality — hostnames, paths — making every observation unique.

Population mixtures, where service accounts and humans share a model and each looks anomalous relative to the other.

Retraining shifts, where the same behaviour scores differently after an unannounced model update.

Blind spots and assumptions

That the training data is clean. Unverifiable without a hunt.

That anomalies are rare. In some environments a substantial fraction of activity is genuinely unusual, and the method's core assumption fails.

That the score means the same thing over time. Retraining shifts the distribution silently.

That unsupervised means no configuration. Contamination rate, window, feature set and retraining cadence are all choices with large effects, made by someone.