OpenAI just published Part II of their Habitat storage deep-dive, and buried in the infrastructure details is one of the most honest accounts I've seen of how scaling decisions actually play out at hypergrowth companies. Habitat is OpenAI's online storage platform—the layer that lets ChatGPT and other products access data in milliseconds for over 1 billion users across 40 geographic regions. It now serves over 500 petabytes of data.
The headline wins are impressive: 12× faster reads, global caching, reliable multi-tenancy. But the real story is the messy middle—the part where OpenAI's infra team went from multi-tenant to single-tenant containers and then back to multi-tenant, learning hard lessons about noisy neighbors, weak partitions, and capacity planning at each turn.
The classic multi-tenancy trap
Habitat started simple: one shared Azure Cosmos DB container, relying on the platform's stated 10K RU/s (request units per second) limit per logical partition key to provide natural backpressure. This is textbook multi-tenancy—one centralized database team provisions resources, everyone shares the pool, you avoid the over-provisioning waste of giving every product team their own isolated system.
It broke in predictable ways. The problems stemmed from how Azure Cosmos DB actually works under the hood:
Logical partition key collisions
Because Habitat exposed partition key selection directly to clients, different product use cases ended up colliding on common keys like user_id or workspace_id. When multiple features share the same logical partition key, they also share that key's throughput limits. A hot organization or hot key could bottleneck unrelated workloads.
This was the exact problem they'd faced with Postgres before migrating to Habitat—one tenant generating disproportionate load taking out the whole database.
Noisy neighbors at the physical partition layer
Here's where it gets subtle. Azure Cosmos DB enforces limits at the physical partition level, not the logical partition key level. Multiple logical partition keys get colocated on the same physical partition, sharing its resources.
In practice, this meant logical partitions couldn't actually reach their documented 10K RU/s limit—they'd hit physical partition constraints first. A workload operating on data for some niche product feature could disrupt core login and signup flows, purely because they happened to share a physical partition.
Weak partitions from storage growth
The throughput limit for a physical partition is variable, depending on total container throughput divided by partition count. When data volume grows, physical partitions split automatically. More partitions, same total throughput = lower per-partition capacity.
For blob-heavy use cases, this created a failure mode where storage growth alone reduced throughput capacity for active workloads, even with stable request rates.
Phase II: Single-tenancy saves the day (sort of)
The fix seemed obvious: separate containers for different use cases. Each container gets independent physical partitions, eliminating noisy neighbor problems and partition key collisions. Combined with a script to keep provisioned throughput synchronized with partition count, this mostly worked.
But it didn't scale organizationally. Two problems emerged:
Unstable aggregate growth
In a shared container, a new use case ramping from 0 to 1 million QPS gets smoothed out by aggregate baseline traffic. In a dedicated container, that same ramp might represent 1,000× growth over the internal dogfooding baseline.
Product teams would launch features without pre-scaling their containers, and the infra team would end up in incident rooms on launch day scrambling to provision capacity. The multi-tenant system's natural workload mixing benefit—stable aggregate growth that absorbs individual spikes—was gone.
Capacity planning for everyone
Without stable aggregate trends, correctly sizing each dedicated container required intimate product knowledge: when would traffic shift, what's the growth trajectory, what's launching next month?
This pushed capacity planning out to every product team at OpenAI. Some teams did it diligently. Many didn't. The result was a wide variance in provisioning quality and a lot of last-minute fire drills.
Phase III: Multi-tenancy redux, with Azure primitives
So OpenAI worked directly with the Azure team to engineer fixes for the original multi-tenancy problems. They shifted back to shared containers by default, but with three critical improvements:
Hot key throttling
Throttling the hottest keys first prevents any single noisy neighbor from consuming all available throughput on a physical partition. Instead of one hot key having site-wide blast radius, it gets throttled in isolation.
Quality of Service (QoS) request tiering
Offline workloads like bulk inserts or data extracts get assigned lower priority. They're throttled first, protecting higher-priority interactive requests that serve end users. This prevents background jobs from starving production traffic.
Improved serverless mode
Azure Cosmos DB's original serverless mode would've fixed the weak partitions problem—it scales based on storage usage—but had a fatal limitation: 5K RU/s per physical partition, which would've capped RU-heavy use cases.
The Azure team built an improved serverless mode that allows every physical partition to scale up to 10K RU/s and introduced RU-usage-based splits in addition to storage-based splits. This eliminated the need for manual container scaling while preserving the ability to handle high-throughput workloads.
The data movement escape hatch
One more piece completes the picture: Habitat now has built-in data movement capabilities. If a particular use case really does need dedicated isolation—maybe it's a new experimental product with wildly unpredictable traffic, or a compliance-sensitive workload—they can migrate it to a separate container.
But it's an explicit choice now, not the default. Multi-tenancy with strong isolation primitives is the baseline, and single-tenancy is the escape hatch for genuinely exceptional cases.
What makes this interesting
Most infra blog posts present a linear progression: we had problem X, we built solution Y, here are the results. This post documents a loop: multi-tenant → single-tenant → multi-tenant again, with each phase teaching something new.
The lesson isn't "multi-tenancy is always right" or "isolation is always right." It's that the optimal strategy depends on both your technical platform's primitives and your organizational scaling dynamics. Single-tenant containers solved OpenAI's noisy neighbor problems but created a capacity planning coordination nightmare that didn't scale with rapid team growth.
The solution wasn't picking one approach. It was working with Azure to build the platform primitives—hot key throttling, QoS tiering, improved serverless—that made multi-tenancy viable again at billion-user scale.
That's the kind of infra work that doesn't make headlines but quietly powers everything we use. And it's refreshing to see it documented this honestly.