Microsoft just published a fascinating post-training experiment that bridges two worlds that rarely touch: database validation and reinforcement learning for agents. The setup is elegant. ThinkingBox-Bench evaluates agents by checking terminal database state against expected outcomes. Each of 507 tasks spins up a real backend, hands the agent tools, lets it work, then runs executable checks. Pass or fail. Binary, verifiable, no vibes.
What Microsoft realized is that this executable verdict isn't just a grader. It's a reward function.
The headline result
They fine-tuned Qwen3.8-27B with GRPO for 50 steps on tasks outside the benchmark, using database correctness as the reward signal. The gains landed almost entirely in the repeatability column:
- Observed 20/20 (tasks that passed every single attempt): 38 → 107 tasks. That's 7.5% → 21.1%. Nearly tripled.
- Pass@1 (average success rate across all attempts): 51.70% → 60.78%. Up 9.08 percentage points.
- Pass@20 (tasks solved at least once): 453 → 446 tasks. Down 7 tasks.
The model didn't get broader. It got sharper. More tasks work every time, fewer paths work occasionally. This is sharpening under RL, and it's consistent with prior work showing that policy gradient methods move probability mass toward high-reward trajectories while low-probability paths vanish.
What repeatable success looks like
Microsoft shows one concrete task: a corporate traveler requests a room downgrade, but company policy requires manager approval first. Before training, the base model changed the booking and issued refunds without waiting for approval in 15 of 20 attempts. After RL fine-tuning, it preserved the booking and payment state in all 20 attempts. In 19 of those, every expected database record matched.
This is the kind of behavior shift that matters for production agents. The model learned to respect a constraint it was frequently violating. The task itself never appeared in training—it's part of the sealed benchmark—but the training pool contained similar multi-step approval workflows.
The training loop: offline policy-relative difficulty
The core challenge in building the training pool was selecting tasks that actually teach something. Most tasks are either already solved (the model succeeds every time) or hopeless (it fails every time). Neither teaches the model anything new.
Microsoft calls this offline policy-relative difficulty. Before training, they ran the base policy once on every candidate task and estimated success probability. They kept only tasks in a difficulty band that the current model could learn from—hard enough to fail sometimes, easy enough to succeed sometimes. That pool stayed fixed throughout the 50-step run.
Difficulty is policy-relative: a task is hard for the current model only until that model learns it.
This is a pragmatic move. You don't want to waste RL steps on tasks the model already nails or tasks so far out of reach that random exploration never finds a reward. The sweet spot is tasks with partial success under the base policy.
The training loop itself is straightforward GRPO. The agent attempts a task, the environment executes it, the database check returns 1 or 0, and that becomes the reward. No human labeling, no LLM-as-judge, no rubric. Just: did the final database state match expectations?
What dependability costs
The tradeoff is visible in the pass@20 column. At-least-once coverage dropped slightly: 453 → 446 tasks. The model now fails to solve 7 tasks it could occasionally solve before. Auto insurance had the largest breadth loss at −7.0 percentage points for pass@20.
This is the classic exploration-exploitation tension. RL sharpens the policy around trajectories that work, but it can prune exploration paths that occasionally succeeded through low-probability action sequences. If you care about coverage—solving as many distinct tasks as possible at least once—this is a cost. If you care about reliability—solving the tasks you attempt with high consistency—this is the right tradeoff.
Where the gains landed
Pass@1 and observed 20/20 rose across all five benchmark domains: retail, auto insurance, travel, neobank, consulting. But pass@20 was mixed. It increased in neobank and consulting, declined in retail, travel, and auto insurance.
The fine-tuned Qwen3.8-27B now sits in an interesting position on the leaderboard. Its pass@1 of 60.78% trails GPT-5.6 Sol at 61.91%, but it passes more tasks on every attempt: 107 versus 82. Claude Sonnet 4.6 scored lower on pass@1 at 59.19%, yet also passed nearly the same number of tasks every time at 102.
Neither metric alone captures the full picture. Pass@1 tells you average success. Observed 20/20 tells you which tasks you can rely on. The ordering changes depending on which you prioritize.
The out-of-distribution test
The 507-task benchmark was never seen during training. The training pool came from a separate set of ThinkingBox-format tasks. So this isn't memorization or overfitting to the test set. The domains, databases, and most MCP tools in the benchmark were not encountered during RL.
That the gains transferred at all is encouraging. It suggests the model learned something about multi-step reasoning with database constraints, not just task-specific shortcuts. The sharpening pattern—more repeatability, slightly less breadth—held across domains the model had never practiced on.
Why database state works as a reward
The reason this setup is compelling is that database correctness is a natural reward boundary for agent tasks. Many real-world agent workflows ultimately modify structured state: CRM records, order tables, ticket systems, booking databases. Checking the terminal state against expectations gives you a binary, deterministic signal with no ambiguity.
No prompt engineering needed. No inter-annotator agreement. No debate about whether the agent's explanation was clear or the tool calls were elegant. Just: is the database in the right state?
This won't generalize to every agent task. Anything involving generation quality, user satisfaction, or open-ended creativity still needs human feedback or LLM-as-judge. But for a large class of operational workflows—data entry, customer service, order management, compliance checks—executable database validation is both available and sufficient.
What's next
Microsoft released ThinkingBox-Training on GitHub, so the training stack is reproducible. The post doesn't specify what's in the training pool or how many tasks, but the offline policy-relative difficulty heuristic is clear enough to implement.
The obvious next steps are scaling the training pool, trying different RL algorithms beyond GRPO, and testing whether the sharpening effect holds for larger models. The leaderboard suggests that even frontier models have a repeatability gap—Claude Opus 5.5 scores 67.16% pass@1 but only 47.53% observed 20/20. If RL can close that gap without sacrificing coverage, it's a meaningful path toward more dependable agents.
The less obvious question is whether this approach works when the database checks are noisier or harder to specify. ThinkingBox tasks have clear expected outcomes: a ticket should be closed, a booking should be modified, a refund should be issued. Many real-world tasks have fuzzier success criteria or require checking multiple tables with complex joins. Scaling the reward function to messier domains is where this either becomes foundational or hits a ceiling.
For now, the result is clean: you can turn database grading into RL rewards, and the model learns to be more consistent. The tradeoff is narrower exploration. Whether that's acceptable depends on whether you're optimizing for coverage or reliability—and for production agents, reliability is usually what matters.