The Model That Runs Its Middle in Circles
Thinking Machines Lab just shipped Inkling 2 Mini, and it's doing something genuinely unusual: looping its decoder layers. The architecture runs 24 physical layers but executes 48 logical layer visits per token by sending each token through the middle 12 layers three times.
This isn't your standard MoE release. Inkling 2 Mini is a ~26B total parameter multimodal mixture-of-experts with 3B active parameters per token. It supports 1M-token context, accepts images and text, and ships in mixed-precision format with NVFP4 and MXFP8 expert weights. Thinking Machines positions it as a local-first model—something you can actually run on consumer hardware—and claims it's comparable to Qwen 3.8 27B in their vibe tests.
What makes this interesting isn't just the benchmark numbers. It's that someone built a production model around layer reuse as a core architectural principle, complete with cross-pass messaging to let each loop refine what the previous one computed. That's the kind of architectural exploration we haven't seen much of since the Transformer became the dominant paradigm.
How Looping Actually Works
The mechanics are straightforward but clever. Inkling 2 Mini has 24 decoder layers: it runs layers 0–5 once, loops layers 6–17 three times, then runs layers 18–23 once. All three passes through the looped core reuse the same weights.
Here's where it gets interesting: each pass through the loop starts from the same input—the output of those first six layers. Information flows between passes through cross-pass "messages." Each looped layer reads a message from the previous pass using its own learned query. A small gating controller examines both the layer's current input and the incoming message, then decides how much of the current input to forget and how much of the message to integrate.
Each of the 128 experts per layer gets its own set of these message gates. The router still selects 8 experts per token, but now those experts can incorporate refined information from previous loop iterations. The result: 48 layer visits with only 24 layers' worth of parameters to store and load.
The architecture uses grouped-query attention (24 query heads, 8 KV heads) with a mix of sliding-window and full-attention layers. Five out of every six layers use a 1,024-token local window; every sixth layer gets full attention with dynamic key-value convolutions. It's a lot of small decisions that add up to something coherent.
The Variable-Loop Research Artifact
Thinking Machines also released TML-d24-varloop-research, a variant trained to accept a variable number of loop passes. You can configure it to run 1, 2, or 3 passes through the core. Fewer passes means faster inference; more passes (presumably) means better quality.
This is conceptually similar to the reasoning-effort dial we've seen in other models, but implemented at the architectural level rather than through extended chain-of-thought generation. Want fast answers? Run one pass. Need more careful reasoning? Spin through all three.
The Hugging Face team notes they've tested it and it works, but they haven't evaluated it in depth. That's the right level of epistemic humility for a research artifact—it's out there, it's functional, but treat it as experimental.
Benchmark Performance
Inkling 2 Mini tops the Artificial Analysis Intelligence Index v5 in its size class and leads in 32 out of 48 benchmarks when compared against Gemma 4 31B, Muse Glimmer, Qwen3.8-27B, and Nemotron 3.5 Super VL. Those benchmarks span coding, vision, reasoning, knowledge, agentic tool use, and safety.
For coding specifically: it leads on FrontierCode v2.0, ties with Qwen3.8-27B on Terminal-Bench 4.0, and trails Qwen on AutomationBench-AA v1.0.6. The model card includes the full benchmark suite if you want the detailed numbers.
I'm always cautious about benchmark-driven narratives—they tell you something, but not everything. The "vibe tests" claim of Qwen3.8-27B comparability is probably more honest: this is a good local model, it works well, but definitive claims require more validation.
Inference Support Everywhere
Thinking Machines shipped with an unusually complete inference story. Day-zero support includes:
- Transformers: both fixed-loop and variable-loop variants via
AutoModelForMultimodalLM - Llama App and llama.cpp: GGUF quants in 4-bit (IQ4_NL) and 8-bit (Q8_0)
- MLX: for Apple Silicon users
- SGLang: for production datacenter deployments
The transformers integration supports tool calling, reasoning-effort settings (from none at 0.0 to max at 0.99), and multimodal inputs. You can pass images alongside text prompts using the standard content-type format.
There's also a DSpark drafter included for speculative decoding speedups at "a small memory cost." The release blog doesn't detail the drafter architecture, but it's shipped as part of the package.
Why Looped Architectures Matter
The looping approach isn't entirely new—Nanbeige explored similar ideas—but Inkling 2 Mini's implementation with cross-pass messaging is a fresh take. What's notable is that someone built a production-ready model around this, not just a research prototype.
We're in a period where architectural exploration has narrowed. Most releases are "bigger MoE, more parameters, better data" rather than "let's fundamentally rethink how layers interact." Looped transformers with message passing represent actual architectural novelty.
The variable-loop research variant hints at something potentially powerful: adaptive computation depth baked into the architecture rather than bolted on through prompting tricks. If you can train a model to effectively use 1–3 passes depending on problem difficulty, you get a natural quality-speed tradeoff without having to generate thousands of reasoning tokens.
Whether this specific approach becomes widespread is less important than the meta-point: there's still room to explore the transformer design space. We haven't exhausted the possibilities.
The Local-First Positioning
Thinking Machines explicitly positions this as a local model. The ~26B total / 3B active parameter count puts it in range for consumer GPUs with quantization. The GGUF quants and Llama App integration make it accessible to non-researchers.
This matters because the local LLM ecosystem has been hungry for strong multimodal models. Qwen has been dominant in this space; having a credible alternative with interesting architectural properties gives the community another option to experiment with.
The 1M-token context is table stakes at this point, but it's worth noting: local models increasingly need to handle large contexts for real-world use cases. Code repositories, long documents, extended conversations—these all push against context limits.
What We Don't Know
The release blog doesn't detail training data, training compute, or the specific performance characteristics of different loop counts in the variable-loop variant. We don't know how much the looping architecture helped versus other design choices. We don't know if the cross-pass messaging is doing heavy lifting or just incremental refinement.
These aren't criticisms—most model releases don't share training details—but they're open questions for anyone trying to understand whether looped architectures generalize.
The "vibe tests" language is refreshingly honest but also imprecise. What tasks? What evaluation methodology? "Comparable" could mean anything from "occasionally matches" to "consistently performs within 2%." More systematic evaluation would help.
Final Thoughts
Inkling 2 Mini is worth attention not because it's the biggest or fastest model, but because it explores architectural ideas that most labs have moved past. Looping, cross-pass messaging, variable compute depth—these are the kinds of innovations that could matter as we figure out what comes after "just scale the MoE."
Whether this specific architecture becomes influential depends on how well the tradeoffs work in practice. Does the looping overhead pay for itself? Do the message gates learn useful refinement patterns? Does variable-loop inference give you meaningful quality-speed flexibility?
Those are empirical questions that the community will answer through use. But the fact that Thinking Machines shipped a production-ready local model around these ideas is itself newsworthy. We need more architectural experimentation, not less.
And if you're running local models, the GGUF quants and Llama App support make this trivially easy to try. That's how you actually validate whether architectural novelty translates to practical value.