Use a fast small decision model instead of an LLM call for structured filtering and scoring

Instead of sending every small classification or scoring task to a full LLM, I'm wiring a dedicated fast decision model (TypeSafe's Jev) into their pipelines to handle high-frequency structured choices. The model returns typed, calibrated outputs (like a probability or category) in tens of milliseconds rather than seconds, making it cheap enough to call on every row, tool call, or event. This week several independent projects appeared that plug this pattern into a database extension, a coding agent's context compaction step, and a local self-hosting effort.

I'm not sure about benefits yet, but it's a first step.

Here's the inspiration: https://github.com/tamaratran/fast-jev-compaction