The Headline
Hugging Face just added Baseten to its Inference Provider ecosystem, giving developers serverless access to a fresh catalog of frontier LLMs—including deepseek-ai/DeepSeek-V4-Flash-0731, moonshotai/Kimi-K3, and zai-org/GLM-5.2—directly from model pages and client SDKs. Baseten joins existing providers like DeepInfra and Scaleway, expanding the breadth of hosted inference available through a single unified interface.
This matters because it removes friction. You can now call a 2.8 trillion parameter model or a 304 billion parameter flash variant with the same OpenAI-compatible SDK you already use, authenticated with your Hugging Face token, and the request routes automatically. No separate signups, no new SDKs to learn, no integration glue code.
The launch supports conversational and text-generation tasks initially, with more modalities promised soon. The full model catalog is available here.
How the Provider Model Works
Hugging Face Inference Providers operate in two modes, and understanding the difference is important for both cost control and privacy.
Custom key mode: You bring your own Baseten API key, set it in your Hugging Face account settings, and requests go directly to Baseten's infrastructure. You're billed on your Baseten account at their standard rates. This gives you full control over usage tracking, billing dashboards, and any provider-specific features.
Routed mode: You authenticate with your Hugging Face token only. The Hub routes the request to Baseten on your behalf, and you're billed through your Hugging Face account—at the same provider API rates, with zero markup. Hugging Face explicitly states they pass through provider costs directly, with no additional fees (though future revenue-sharing agreements with providers are possible).
The routed model is elegant: PRO users get $2 in inference credits monthly that work across all providers, and free users get a small quota to experiment. If you're already paying for HF PRO ($9/month for researchers, more for teams), you've effectively got a sampling budget across the entire provider network.
The SDK Integration Is the Real Win
The headline feature is "it works from the Hub UI," but the real unlock is SDK integration. Both huggingface_hub (Python, version ≥ 1.26.1) and @huggingface/inference (JavaScript) support provider routing natively.
Here's what it looks like in practice with the OpenAI-compatible interface:
from openai import OpenAI
import os
client = OpenAI(
base_url="https://router.huggingface.co/v1",
api_key=os.environ["HF_TOKEN"],
)
completion = client.chat.completions.create(
model="deepseek-ai/DeepSeek-V4-Flash-0731:baseten",
messages=[{"role": "user", "content": "..."}],
)
The model identifier format is {repo_id}:{provider}. That colon-suffix routes the request. If you've used OpenRouter or similar unified inference APIs, this will feel familiar—but here you're authenticated through Hugging Face's ecosystem, not yet another third-party aggregator.
The JavaScript version is identical in structure, just with process.env.HF_TOKEN and async/await syntax.
Agent Harness Compatibility Out of the Box
One detail that's easy to miss: Baseten-hosted models are now accessible through agent frameworks like Pi, OpenCode, Hermes Agents, and OpenClaw without additional integration work. The announcement explicitly calls this out—these harnesses already support Hugging Face Inference Providers, so Baseten models slot in immediately.
This is a quiet but meaningful developer experience win. If you're building with an agentic framework that routes tool calls or manages multi-step reasoning, you can now point it at deepseek-ai/DeepSeek-V4-Flash-0731:baseten and it just works. No custom adapters, no pull requests to add provider support, no waiting for maintainers to merge your OpenAI-compatibility shim.
The full list of compatible harnesses is maintained on the Hub, and the OpenAI-compatible base URL pattern (https://router.huggingface.co/v1) makes this trivial to wire up.
What's in the Model Catalog
Baseten's initial lineup skews toward recent open-weight LLMs that aren't always easy to self-host at scale:
- DeepSeek V4 Flash (
deepseek-ai/DeepSeek-V4-Flash-0731): 304 billion parameters, optimized for speed. DeepSeek's flash variants have consistently punched above their weight on code generation and structured output tasks. - Kimi K3 (
moonshotai/Kimi-K3): 2.8 trillion parameters, image-text-to-text capable. Moonshot AI's models have been notable for long-context performance, and K3's multimodal support makes it interesting for document understanding workflows. - GLM-5.2 (
zai-org/GLM-5.2): 753 billion parameters, text generation. The GLM series from Zhipu AI (zai-org) has strong benchmarks on Chinese and bilingual tasks.
These are not the usual Llama or Mistral suspects. Baseten is positioning itself as the provider for frontier models that are open-weight but operationally complex—the ones where serverless inference is a genuine unlock rather than a convenience.
Support for additional tasks beyond text generation and chat is coming, though no timeline was specified.
Pricing Transparency (and the PRO Incentive)
Hugging Face is being unusually transparent about the economics here. There's no markup on routed requests—you pay Baseten's API rates, and Hugging Face passes through the cost. This is different from many inference aggregators that add 10–30% on top of provider pricing.
The PRO plan ($9/month for individuals) includes $2 in monthly inference credits that work across all providers. That's not a huge budget—maybe a few hundred thousand tokens depending on the model—but it's enough for prototyping and exploration. Free users get a smaller quota.
The incentive structure is clear: Hugging Face wants you on PRO (which also unlocks ZeroGPU, Spaces Dev Mode, and higher rate limits), and they're willing to subsidize multi-provider inference access to make that sticky. For Baseten, this is distribution—they're acquiring developers who might not have discovered them otherwise, and converting them into direct customers if usage scales.
It's a classic platform play, and it's working. The provider ecosystem now spans multiple vendors with different model catalogs, and the unified billing + SDK experience is genuinely better than managing N different API keys.
What This Means for the Inference Market
The Inference Provider model is quietly reshaping how developers think about model access. Instead of "I need to stand up vLLM and figure out KV cache tuning" or "I'll just call OpenAI," there's now a third path: "I'll use the Hugging Face SDK and let provider routing handle it."
This commoditizes infrastructure in a useful way. If Baseten, DeepInfra, and Scaleway are all accessible through the same interface, differentiation shifts to model catalog, latency, reliability, and price—not SDK compatibility or integration overhead.
For Baseten specifically, this is a credibility signal. Being integrated into the Hub alongside established providers like DeepInfra (who launched their provider integration earlier) positions them as a serious player in the serverless inference space. The model catalog—DeepSeek V4, Kimi K3, GLM-5.2—skews toward recent non-Western frontier models, which suggests Baseten is carving out a niche rather than competing head-to-head on Llama and Mistral derivatives.
The open question is whether this erodes margins for inference providers over time. If Hugging Face becomes the primary discovery and routing layer, providers become interchangeable backend compute—and that's a race to the bottom on price. But for now, the pie is growing fast enough that everyone's getting a bigger slice.
Try It
If you're on Hugging Face PRO or willing to burn through the free quota, the setup is trivial:
- Grab your HF token from account settings
- Point your OpenAI client at
https://router.huggingface.co/v1 - Call
deepseek-ai/DeepSeek-V4-Flash-0731:basetenor any other supported model
Hugging Face is collecting feedback in this discussion thread. If you hit rough edges or have strong opinions about which models should be prioritized for provider support, that's the place to surface them.
The unified inference interface is one of those ideas that seems obvious in retrospect but took real coordination to ship. Baseten joining the ecosystem makes it more complete, and the model catalog—especially the non-Western frontier LLMs—is a genuine differentiator. Worth experimenting with if you're building anything that needs access to recent open-weight models without the ops burden.