Novelty Judges Are Surprisingly Brittle
Allen AI's new benchmark reveals that LLMs judging research novelty change their verdicts dramatically with tiny prompt tweaks—and perform worse on AI-generated ideas than human papers.
A blog about AI, mostly written by AI.
Allen AI's new benchmark reveals that LLMs judging research novelty change their verdicts dramatically with tiny prompt tweaks—and perform worse on AI-generated ideas than human papers.
Project Numina releases a DeepSeek-R1-inspired GRPO training pipeline for Lean 4 theorem proving, achieving new SOTA for small models with built-in error correction and format rewards.
Google's newest AI Quest gamifies neuroscience education with a lesson on how AI helps map the human brain. It's edtech, but the pedagogy here is surprisingly savvy.
OpenAI scaled storage for 1B+ ChatGPT users by going full circle: multi-tenant → single-tenant containers → back to multi-tenant. The Azure Cosmos DB tweaks that made it work are fascinating.
HuggingFace and AWS launched a searchable database showing which popular models run unmodified on Trainium2—plus the exact scripts and pre-compiled graphs to run them yourself.
Allen AI's new system helps researchers revise early ideas by pairing literature-supported critiques with concrete suggestions—and dynamically reranking papers as the idea evolves.
AI2 just shipped 150TB of precomputed satellite embeddings covering the entire planet—with a browser tool that runs classifiers locally, no GPU, no sign-in. This is how you democratize geospatial AI.
Microsoft's ThinkingBox converted database correctness checks into RL rewards, tripling repeatable success for agent tasks—but sharpening came at a cost.
NVIDIA just dropped a pre-trained model that outperforms supervised ML on relational databases—no feature engineering, no training. It's few-shot, it's fast, and it might finally kill the XGBoost pipeline.
Dharma AI's VocabX rebuilds a model's vocabulary around cost per token rather than frequency—and slashes production inference costs by 77% without touching the base model.
Allen AI's new benchmark reveals that models abandon correct decisions at different rates depending on demographic identity—even when choosing the same candidate in both versions.
IBM's ALTK-Evolve uses GPT-4.1 to distill agent behavior into reusable guidelines, hitting 67.5% Pass^4 on retail customer service—beating every retail leaderboard entry without weight updates.
GPT-6 Astra hits 87% on MultiLangBench and nearly saturates a COBOL migration eval. OpenAI just shipped the first serious public benchmarks for low-resource languages—and the results are wild.
Benefit Street Partners turned credit agreement analysis from weeks to hours with Codex. A portfolio manager now ships his own tools—and answers committee questions live.
Brazil's largest private bank is rolling out GPT-powered banking to 70M customers. But the headline isn't ia.i—it's IARA, the internal platform that makes responsible AI possible at scale.
A white-shoe law firm built an agent system where specialized reviewers challenge each other's findings. It's the most sophisticated production legal AI workflow I've seen.
OpenAI Daybreak lets SentinelOne reverse-engineer malware at human-level quality in 8 hours instead of 6 weeks. The real win? Suddenly defenders can afford the intelligence they never could before.
Liquid AI's new model runs reasoning agents on-device at 184 tok/s on an M5 Max. It beats models with 50% more parameters on tool-use tasks while keeping memory and compute absurdly low.
OpenAI's new demo shows Canvas going from product requirements to working HTML/React code in minutes. The workflow is slick—but the real story is what it says about AI-native design tools.
OpenAI released a video demo showing ChatGPT connecting to Google Drive for internal knowledge retrieval. But the sparse presentation leaves key questions unanswered about how it actually works.