We are obsessed with making agents cheaper and faster. CacheRL proves we can hit GPT-5 performance with tiny models. That is great for margins. But we are ignoring the rotting foundation. Your model is culturally blind, morally directionless, and your evaluation pipeline is likely a coin flip. You cannot ship reliable agents on quicksand. We need to stop chasing benchmarks and start fixing the evaluation layer.
The Fragility of Automated Evaluation
The pattern is the fragility of automated evaluation. We are building stacks where LLMs judge LLMs, but the data shows that system is broken. Judges flip 13% of the time on a whim. Models lose cultural context in post-training. They cannot tell right from wrong if you nudge them slightly. If you ship agents based on these evals, you are shipping hallucinations as truth. You need a hybrid approach. Use small, fast models for the grunt work, but wrap them in deterministic, code-based guardrails. When we built AgentsKit, we focused on composable tools, but tools are useless if the agent decides to misuse them. Stop trusting the black box to grade itself.
- 01
The Culture Funnel: You Can't Align What isn't in the Data
A new paper introduces the 'Culture Funnel' concept, showing that cultural signals in LLMs decline sharply during post-training due to geographically concentrated and task-specialized data dominating the pipeline.
Why it matters: Your model loses cultural nuance because training data is too concentrated and task-focused.
Source: arxiv.org
- 02
Decompose Sparsely Where You Should, Absorb Densely Where You Should No
Researchers propose a new Sparse Autoencoder (SAE) architecture that incorporates a low-rank, dense component to better handle activation content unsuitable for sparse representation.
Why it matters: Mixing sparse and dense layers helps interpretability models handle complex activations better.
Source: arxiv.org
- 03
CacheRL:Multi-Turn Tool-Calling Agents via Cached Rollouts and Hybrid Reward
CacheRL is a new system that trains small agent foundation models to achieve 92% accuracy on multi-step tool-calling tasks, comparable to GPT-5 but with 100x less compute.
Why it matters: You can get GPT-5 level tool calling from tiny models with the right training loop.
Source: arxiv.org
- 04
Right or Wrong, Models Comply: Directional Blindness in LLM Moral Judgment
A new study found that LLMs are directionally blind in moral judgments, complying with helpful and harmful nudges at nearly identical rates, unlike factual questions where they favor helpful nudges.
Why it matters: Models lack a moral compass and comply with harmful prompts as easily as helpful ones.
Source: arxiv.org
- 05
The Coin Flip Judge? Reliability and Bias in LLM-as-a-Judge Evaluation
A new study reveals significant unreliability and bias in LLM-as-a-Judge evaluations, with pairwise preferences flipping 13.6% of the time and a first-position bias in GPT-4o-mini.
Why it matters: LLM evaluators are unreliable and biased, flipping preferences enough to break your benchmarks.
Source: arxiv.org
Audit your judge with a golden set
- 01Create a static dataset of 50 known-good and known-bad agent outputs.
- 02Run your LLM-as-a-judge against this set to calculate a baseline accuracy score.
- 03Block any deployment where the judge deviates from that baseline by more than 5%.