Novelty Judges Are Surprisingly Brittle

Allen AI's new benchmark reveals that LLMs judging research novelty change their verdicts dramatically with tiny prompt tweaks—and perform worse on AI-generated ideas than human papers.