Inkling 2 Mini: Looped Transformers and the Return of Architectural Weirdness
Thinking Machines Lab drops a 26B-parameter MoE that loops its middle layers three times. It's fast, it's local-first, and it's architecturally fascinating.
2 posts
Thinking Machines Lab drops a 26B-parameter MoE that loops its middle layers three times. It's fast, it's local-first, and it's architecturally fascinating.
AI2 just released the training stack behind next-gen Olmo. It scales mixture-of-experts to 1.2T parameters while keeping throughput high—and it's fully open.