Hugging Face just dropped the easiest way to stand up a private vLLM server I've seen. One CLI command, pay-per-second billing, zero server provisioning. If you've ever wrestled with Kubernetes configs or tried to keep a GPU instance alive just long enough to run evals, this feels like cheating.
The entire flow—spin up, query from anywhere, tear down—takes less time than reading the average YAML tutorial. And it's OpenAI-compatible out of the box, so you can swap the endpoint into any client that already speaks that API.
Here's what's interesting, what's not just convenience theater, and when you'd pick this over Inference Endpoints.
The One-Liner
The actual command is hf jobs run with a Docker image, a GPU flavor, and an exposed port:
hf jobs run --flavor a10g-large --expose 8000 -- timeout 2h \
vllm/vllm-openai:latest \
vllm serve Qwen/Qwen3-4B --host 0.0.0.0 --port 8000
You get back a job ID and a URL like https://<job_id>--8000.hf.jobs. Wait a couple minutes for weights to download and vLLM to boot. When the logs show Application startup complete, you're live.
The --expose 8000 flag routes the container's port through HF's jobs proxy, which gates access with your HF token. Every request needs Authorization: Bearer <your_hf_token> in the header. The endpoint isn't public—it's scoped to you (or your org if you're working in a team namespace).
Why This Isn't Just Hosted Docker Run
The immediate value is obvious: you're not provisioning instances, you're not keeping a GPU warm when you don't need it, and you're billed per second. An A10G Large runs at $1.50/hour, so if your eval takes 20 minutes, you pay 50 cents.
But the less obvious piece is that --expose gives you a stable, routable endpoint immediately. No SSH tunneling, no ngrok, no DNS setup. You can hit it from a Jupyter notebook on your laptop, from a CI pipeline, from a Lambda function—anywhere with an HF token.
The jobs proxy is your access control layer. That means you don't have to bolt authentication onto vLLM yourself, which is a relief if you've ever tried to secure a raw inference server. The downside: if you need finer-grained permissions (say, a public endpoint with rate-limiting or org-wide shared access that isn't token-based), you need a real gateway or you should reach for Inference Endpoints instead.
Querying From Anywhere
vLLM speaks the OpenAI API, so you can hit it with curl:
curl https://<job_id>--8000.hf.jobs/v1/chat/completions \
-H "Authorization: Bearer $(hf auth token)" \
-H "Content-Type: application/json" \
-d '{
"model": "Qwen/Qwen3-4B",
"messages": [{"role": "user", "content": "Hello!"}]
}'
Or from Python with the OpenAI client, pointed at the custom base URL:
from huggingface_hub import get_token
from openai import OpenAI
client = OpenAI(
base_url="https://<job_id>--8000.hf.jobs/v1",
api_key=get_token(),
)
resp = client.chat.completions.create(
model="Qwen/Qwen3-4B",
messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)
This is trivial if you've done it before, but the point is there's zero adapter code. Any tool that knows how to talk to OpenAI can talk to this endpoint with a two-line config change.
Scaling to Bigger Models
The same pattern works for much larger models—just pick a beefier --flavor and tell vLLM to shard across GPUs with --tensor-parallel-size.
For example, the 122B Qwen3.5 mixture-of-experts model on 2× H200:
hf jobs run --flavor h200x2 --expose 8000 -- timeout 2h \
vllm/vllm-openai:latest \
vllm serve Qwen/Qwen3.5-122B-A10B \
--host 0.0.0.0 --port 8000 --tensor-parallel-size 2 \
--max-model-len 32768 --max-num-seqs 256
The --tensor-parallel-size should match the GPU count in the flavor (h200x2 → 2, h200x8 → 8). Run hf jobs hardware to see what's available.
The --max-model-len and --max-num-seqs flags are model-specific. Qwen3.5-122B is a hybrid Mamba/attention architecture with a 256K default context, which doesn't fit vLLM's default batch settings in memory. Capping context length and concurrent sequences keeps it within the H200's 141GB per GPU.
If a model OOMs on startup, dialing these two knobs down is the first thing to try. Everything else—the exposed URL, the OpenAI client, the token auth—stays identical.
When You'd Actually Use This
This setup shines for:
- Evals and experiments: Spin up a model, run a benchmark or ablation study, tear it down. Pay only for runtime.
- Batch generation: Stand up an endpoint, fire a few thousand prompts at it, collect outputs, kill the job.
- Prototyping with a specific model: You want to see how Qwen3.5 or Llama 3.3 behaves on your task before committing to a deployment.
It's less suited for:
- Long-lived, production-facing APIs: If you need scale-to-zero, public access with fine-grained auth, or uptime SLAs, you want Inference Endpoints.
- Interactive services with unpredictable traffic: Jobs bill per second whether or not they're serving requests. Inference Endpoints scale to zero when idle.
The official guidance: reach for Jobs when you want maximum control and flexibility (it's just Docker on HF infrastructure). Reach for Inference Endpoints when you want production-ready operational features.
SSH, Gradio UIs, and Coding Agents
A few extensions that show how flexible the primitive is:
SSH Into the Running Server
Launch with --ssh and you can shell straight into the container:
hf jobs run --flavor a10g-large --expose 8000 --timeout 2h --ssh \
vllm/vllm-openai:latest \
vllm serve Qwen/Qwen3-4B --host 0.0.0.0 --port 8000
Then hf jobs ssh <job_id> drops you into a shell where you can run nvidia-smi, tail logs, or inspect the vLLM process. This makes debugging startup failures or memory issues much easier than reading logs from the outside.
Chat UI with Gradio
If you want a chat window instead of curl, point a local Gradio app at the endpoint. The blog post includes a full example that streams both reasoning and answers from Qwen3 into separate UI panels. You run the Gradio script on your laptop; it hits the remote vLLM server over the exposed port.
Coding Agent Backend with Pi
The same endpoint can back a terminal coding agent. Pi is a provider-agnostic agent harness. If you relaunch vLLM with --enable-auto-tool-choice and --tool-call-parser hermes (for Qwen3's tool-calling format), then add the job as a custom provider in Pi's config, you get a Read/Write/Edit/Bash agent running on your own self-hosted model.
This is where the pattern becomes genuinely interesting: the same one-line server command becomes the backend for interactive tools, batch scripts, and autonomous agents. The abstraction is just an OpenAI-compatible HTTP endpoint, so anything that speaks that protocol can plug in.
The Bigger Picture
What makes this feel different from past "easy hosting" pitches is that it's not trying to hide the machinery. You're still picking the Docker image, the exact vLLM flags, the hardware. It's infrastructure-as-code that happens to run on HF's stack instead of yours.
The jobs proxy—authentication via HF token, per-second billing, stable URLs without manual networking—handles the annoying operational bits without forcing you into a framework. You're not locked into a managed service's model selection or configuration constraints.
The line between "run this locally" and "run this on someone else's GPUs" is now thin enough that the decision is mostly about whether you have the hardware sitting around, not whether you want to deal with deployment complexity.
For one-off inference tasks, quick experiments, or eval runs, that's a meaningful shift. If you've ever spun up a GPU instance just to kill it 30 minutes later, you'll recognize the value immediately.