We spent years optimizing for a single number. Accuracy. Now the best systems track how sure they are about that number. This shift is why I designed the tools in AgentsKit to return structured metadata, not just raw text. It changes how we build alerts, diagnostics, and agents.
The build decision
Replace rigid thresholds with probabilistic reasoning
When you build anomaly detection or tracking systems, stop using hard cutoffs. The Alzheimer's study shows that a probabilistic score with uncertainty intervals outperforms static models because disease progression isn't linear. Similarly, using LLMs for tabular data works because the model synthesizes logic from context rather than just fitting a curve. You need to know the confidence level of your model's output. If you can't measure uncertainty, you are flying blind in production. Keep your data ingestion deterministic, but let your inference layer reason about probability.
- 01
Tracking Alzheimer's progression with Bayesian uncertainty
A new Bayesian framework, DCP, continuously estimates Alzheimer's disease progression from longitudinal DTI scans using a probabilistic disease severity score with uncertainty, outperforming existing methods on ADNI data
Why it matters: Longitudinal tracking requires modeling uncertainty, not just a point estimate.
Source: arxiv.org
- 02
Using LLMs to detect tabular data anomalies
A new LLM-based framework uses in-context learning to detect anomalies in tabular data by synthesizing detection logic from statistical and causal summaries of normal data, outperforming 15 SOTA methods.
Why it matters: LLMs can outperform specialized models by learning detection logic in context.
Source: arxiv.org
- 03
Applying game AI research to complex worlds
Google DeepMind is leveraging 15 years of AI research in games to co-develop innovative AI-driven gameplay with major studios, from classic Atari titles to complex worlds like EVE Online.
Why it matters: Game research is finally graduating to production-grade complex environments.
Source: deepmind.google
Field test
Add confidence scoring to your LLM outputs
- 01Prompt your model to return a JSON object with a confidence field between 0 and 1.
- 02Filter out actions where confidence is below 0.8 before executing code.
- 03Log the low confidence cases for manual review to refine your prompts.
Reader pulse
Your constraint can shape the next issue
What's the last feature you shipped that tracks model confidence? Hit reply with one sentence, I read every response.