Predictive autoscaling, explained without the buzzwords

Reactive autoscaling waits for a signal to cross a threshold, then reacts. By definition, it's late. The first user to hit the spike pays the latency cost. So does the second, the third, and everyone else who shows up before the new replicas come warm.

Predictive is a different question

Predictive autoscaling doesn't ask 'are we over the threshold?' It asks 'given the last 14 days of traffic shape, what will the next 60 seconds look like?' If the answer is 'busier,' it provisions ahead of the curve. The first user pays nothing.

Why this isn't just bigger thresholds

You can't fake predictive by lowering your reactive threshold. Lower thresholds mean more scale events, more cold starts, more cost — and you still don't beat the curve, you just react sooner. Real prediction needs a model of your traffic, not a tighter dial.

Token budgets, not pod counts

We scale on token budgets — a unit that maps to actual work, not container instances. A 4x spike in requests of trivial shape and a 4x spike of expensive shape are different problems. Pod-count autoscaling treats them the same; budget-based autoscaling treats them honestly.

What this changes for the team

p99 stops being a cliff at every traffic event. Cost stops being a leading indicator of bad capacity planning. And the on-call stops getting paged for 'CPU is high again.' That's the whole product.