#inference

#…

6 posts

DiffusionGemma: Google's 4x speed boost trades autoregressive for diffusion

Google DeepMind just open-sourced a 26B MoE model that generates 256 tokens in parallel—4x faster on GPUs. It's lower quality than Gemma 4, but the architecture shift is fascinating.

#llms #diffusion #open-models #inference #google

JetBrains Mellum2: A 12B MoE Built for the Boring (But Critical) Parts of Your AI Stack

JetBrains just released Mellum2, a 12B-parameter MoE model that activates only 2.5B per token. It's not trying to be frontier—it's built for routing, RAG, and agent subtasks where speed matters.

#moe #code-models #inference #jetbrains #open-source

NVIDIA Nemotron-Labs Diffusion: The Speed-of-Light Text Generation Nobody Saw Coming

NVIDIA just open-sourced diffusion language models that generate multiple tokens in parallel at 6× the speed of autoregressive models—and they're actually good. Here's what changes.

#llms #diffusion #inference #nvidia #open-models

vLLM V1 and the Correctness-First Paradigm in RL

ServiceNow AI's deep dive into vLLM's V1 upgrade reveals why getting base correctness right matters more than chasing incremental RL gains—a lesson in engineering priorities.

#vllm #reinforcement-learning #inference #ml-infrastructure #correctness

Google's TPU v8 Splits Into Two: Training vs. Inference in the Agentic Era

Google just announced TPU v8, but instead of one chip, they're shipping two: v8T for training and v8I for inference. Here's why the bifurcation matters for AI's next phase.

#tpu #infrastructure #agents #google-cloud #inference

Hugging Face makes Transformers play nice with Apple Silicon—and it's kind of brilliant

Hugging Face just shipped MLX support in Transformers, letting you run models natively on Apple Silicon with zero code changes. It's the PR we all wanted to write.

#transformers #apple-silicon #mlx #inference #tooling

Loading…