Hugging Face just quietly shipped one of those features that feels obvious in hindsight but changes how you actually work: hf jobs run now lets you spin up a private, OpenAI-compatible vLLM server with a single command. No Docker wrangling, no Kubernetes manifests, no provisioning headaches. Just point it at a model, pick your GPU flavor, and you're live.
This matters because the gap between "I want to test this model" and "I have an endpoint I can actually query" just collapsed to about two minutes. And unlike managed inference products, you get the full vLLM command line — every flag, every optimization, complete control over tensor parallelism and context windows.
What you actually get
The entire flow is remarkably clean. Install huggingface_hub >= 1.20.0, authenticate once with hf auth login, then run:
hf jobs run --flavor a10g-large --expose 8000 --timeout 2h \
vllm/vllm-openai:latest \
vllm serve Qwen/Qwen3-4B --host 0.0.0.0 --port 8000
That's it. The command prints a URL like https://6a381ca1953ed90bfb947332--8000.hf.jobs where your server is reachable. Wait a couple minutes for weight downloads and startup, and you've got a live endpoint.
The --expose 8000 flag routes the container's port through HF's jobs proxy, which acts as your API gateway. Every request needs your HF token as a bearer, so access is automatically scoped to you and your org. It's gated by design, not public.
The OpenAI compatibility angle
vLLM speaks the OpenAI API natively, which means you can hit it with curl or point the official OpenAI Python client straight at it:
from huggingface_hub import get_token
from openai import OpenAI
client = OpenAI(
base_url="https://<job_id>--8000.hf.jobs/v1",
api_key=get_token(),
)
resp = client.chat.completions.create(
model="Qwen/Qwen3-4B",
messages=[{"role": "user", "content": "Hello!"}],
)
This isn't some wrapper or adapter layer. It's the actual OpenAI SDK talking to vLLM's OpenAI-compatible server. Which means any tool, framework, or agent harness that expects an OpenAI endpoint just works. The post demonstrates hooking it up to Pi, a terminal coding agent, by just dropping the job URL into the config.
Scaling to serious hardware
The same command pattern scales all the way up to multi-GPU setups. For Qwen3.5-122B on 2× H200:
hf jobs run --flavor h200x2 --expose 8000 --timeout 2h \
vllm/vllm-openai:latest \
vllm serve Qwen/Qwen3.5-122B-A10B \
--host 0.0.0.0 --port 8000 --tensor-parallel-size 2 \
--max-model-len 32768 --max-num-seqs 256
The --tensor-parallel-size should match the GPU count in your flavor. The --max-model-len and --max-num-seqs flags are model-specific — Qwen3.5-122B is a hybrid Mamba/attention architecture with a 256K default context that doesn't fit vLLM's default batch settings without capping.
Pricing is per-second for actual usage. An a10g-large runs at $1.50/hour; you can check hf jobs hardware for the full price list. When you're done, hf jobs cancel <job_id> stops billing immediately.
The debugging story is surprisingly good
One detail that elevates this from "useful hack" to "actually practical": you can SSH directly into the running job. Launch with --ssh, register your public key at huggingface.co/settings/keys, then:
hf jobs ssh <job_id>
You're now inside the container where you can run nvidia-smi, tail logs interactively, inspect the vLLM process, or debug startup failures in real time. This matters more than it sounds like it should. Opaque container failures are usually where DIY inference setups turn into multi-hour debugging sessions.
How this compares to Inference Endpoints
Hugging Face already has Inference Endpoints, which is their managed, production-ready serving product. The post explicitly addresses when to use which.
Inference Endpoints give you operational niceties for long-lived services: finer-grained access control (public, protected, or private), scale-to-zero during inactivity, and managed infrastructure. You trade flexibility for convenience.
HF Jobs is docker run for HF infrastructure. You get complete control over the container image, exact vLLM flags, and hardware selection, but you're responsible for the operational details. It's optimized for experiments, evals, batch generation, and tire-kicking before committing to production infrastructure.
The distinction is genuinely useful. Inference Endpoints are what you reach for when you need a durable endpoint. Jobs are what you reach for when you need maximum control right now.
What makes this interesting
The infrastructure play here is subtle but significant. Hugging Face is positioning Jobs as the bridge between "model in the Hub" and "endpoint I can actually use" with minimal friction. One command, pay-per-second billing, SSH access for debugging, and complete control over serving parameters.
This competes less with managed inference platforms (Replicate, Modal, Baseten) and more with the DIY path of spinning up your own GPU instance and fighting with CUDA drivers. It's not trying to be simpler than managed platforms — it's trying to be simpler than doing it yourself while preserving the control that makes doing it yourself attractive.
The OpenAI compatibility is the other critical piece. Every LLM tool built against the OpenAI API (which is most of them at this point) can now point at your self-hosted model with a config change. That's a meaningful unlock for teams that want to own their inference stack but don't want to rewrite their tooling.
The practical implications
For researchers and engineers doing evals, this changes the cost equation. Spinning up a 122B model for a batch run, running your suite, and tearing it down used to mean provisioning infrastructure. Now it's three commands and you pay only for runtime.
For teams prototyping with frontier models, being able to stand up a private endpoint in two minutes makes rapid iteration actually feasible. Test a prompt engineering approach, swap to a different model, compare outputs, all without touching cloud consoles or waiting for managed platform deployments.
And for anyone building agents or tools that need model access, having a stable, OpenAI-compatible endpoint you fully control is valuable. You're not rate-limited by API providers, you can run uncensored models, and you can tune serving parameters (context length, batch size, sampling) to your exact use case.
The command-line interface here matters. This isn't infrastructure-as-code you need to version control. It's infrastructure-as-command you can paste into a README. That's a meaningful reduction in cognitive overhead.
What's still missing
The source post notes that the jobs proxy requires your HF token for every request, which is fine for private use but not if you need finer-grained or public access. For that, you'd need to put a proper gateway in front or use Inference Endpoints instead.
The timeout flag is a safety net to auto-stop billing, but there's no built-in monitoring or alerting. You're responsible for remembering to cancel jobs when you're done. That's reasonable for the target use case (temporary infrastructure) but means this isn't a set-and-forget solution.
And while SSH access is excellent for debugging, the workflow still assumes you're comfortable with container internals and vLLM configuration. The abstraction layer is thin by design, which is the point, but it's not hiding complexity from you.
The bottom line
This is one of those features that doesn't look revolutionary in a feature list but changes your actual workflow. The distance between "I want to test this model" and "I have a queryable endpoint" collapsing to a single command and two minutes of wait time is genuinely useful.
It's not trying to be the simplest possible inference solution. It's trying to be the fastest path to a fully-controlled, OpenAI-compatible endpoint on serious hardware without infrastructure overhead. For evals, prototyping, and batch generation, that's exactly the right trade-off.