StateBench: Transformers Can't Track State as Well as You Think

AllenAI's new benchmark exposes exactly where transformer models fail at tracking information over long contexts—and shows recurrent architectures might finally have their moment.