Microsoft Turned Database Grading Into RL Rewards—And It Worked
Microsoft's ThinkingBox converted database correctness checks into RL rewards, tripling repeatable success for agent tasks—but sharpening came at a cost.
#…
Loading…
26 posts
Microsoft's ThinkingBox converted database correctness checks into RL rewards, tripling repeatable success for agent tasks—but sharpening came at a cost.
NVIDIA just dropped a pre-trained model that outperforms supervised ML on relational databases—no feature engineering, no training. It's few-shot, it's fast, and it might finally kill the XGBoost pipeline.
IBM's ALTK-Evolve uses GPT-4.1 to distill agent behavior into reusable guidelines, hitting 67.5% Pass^4 on retail customer service—beating every retail leaderboard entry without weight updates.
Allen AI's new benchmark reveals that models abandon correct decisions at different rates depending on demographic identity—even when choosing the same candidate in both versions.
GPT-6 Astra hits 87% on MultiLangBench and nearly saturates a COBOL migration eval. OpenAI just shipped the first serious public benchmarks for low-resource languages—and the results are wild.
Technology Innovation Institute in Abu Dhabi just shipped a 1.6B-parameter ASR model focused on Emirati Arabic that beats published benchmarks by 2.25 points WER. The numbers tell a story about dialect.
TII's new 7B model scores 84.83% on Emirati-Arabic benchmarks by doing what scale alone can't: targeted dialect adaptation with crawled data, synthetic generation, and cultural grounding.
Hugging Face turned its dataset hub into a universal RL environment registry. No new infra, no sign-ups—just tags. Harbor, Verifiers, OpenEnv, and NeMo Gym environments all land in one filter.
Microsoft's ThinkingBox benchmark runs AI agents 20 times on the same task and grades them on database state, not tool calls. The gap between 'can do it once' and 'does it every time' is huge.
NVIDIA just released Kumo Tabular, an open foundation model that predicts on new tabular data with zero training. It swept four benchmarks and runs entirely on synthetic pretraining data.
H company's new agentic models work across GUIs, code, and APIs in the same system. Built with an Agentic Task Factory and trained on real software workflows, they compete with frontier models at a fraction of the cost.
Liquid AI's new DSpark drafter speeds up LFM2.5-VL by up to 3x on device. But the bigger story is what it reveals about Amdahl's law in VLM inference: when prefill dominates, decode speedups matter less.
Allen AI's new auditing tool reveals that many LLM benchmarks mix multiple capabilities into a single score—and shows which questions actually matter.
Hugging Face's ASR leaderboard just added Hindi and Indian English with metadata on 4,888 speakers across hundreds of districts. Benchmarks decide what gets built—this one might actually be fair.
Researchers introduce three probes that catch speech models reproducing benchmark transcripts—even when the audio contradicts them, words are silenced, or spellings should vary randomly.
IBM's ALTK-Evolve reveals a counterintuitive finding: more agentic memory isn't always better. The right dose depends on the model—and sometimes the cheapest strategy wins.
IBM Research's ALTK-Evolve matches or beats ACE's agentic memory performance at 15–40% of the inference cost. The secret: calibrated delivery instead of injecting the whole playbook every time.
Allen AI's new benchmark reveals that LLMs trained to be helpful assistants struggle with tutoring's core trade-off: knowing when students need support versus when they need productive struggle.
OpenAI's GPT-5.6 Sol jumped from 13.3% to 38.3% on ARC-AGI-3 by keeping reasoning in context and using compaction. The lesson: benchmarks measure more than models—they measure harnesses.
IBM Research just dropped a benchmark that reveals a harsh truth: frontier coding agents achieve less than 10% success migrating real Java apps. The problem isn't code—it's everything else.
ServiceNow built a benchmark proving that deep-research agents leak private info through web queries—and that making them smarter makes it worse. Privacy-aware RL cuts leakage by 70%.
ServiceNow just released a benchmark testing frontier ASR on code-switched speech—and the results reveal which models can actually handle bilingual customers and which fall apart mid-sentence.
ServiceNow's new voice-agent benchmark spans airlines, IT, and healthcare—with joint-generation pipelines, adversarial scenarios, and a coming multilingual expansion.
Frontier models score below 50% on Kubernetes incident response. The new ITBench-AA benchmark from Artificial Analysis and IBM reveals the gap between agent demos and production IT work.
The Open ASR Leaderboard is fighting back against benchmaxxing with a simple but effective strategy: private evaluation datasets that no one can train on.
TII launches QIMMA, a rigorous quality-focused leaderboard for Arabic LLMs that goes beyond translation metrics to measure genuine language understanding and cultural nuance.