Most AI infrastructure writing focuses on training—clusters of H100s, data pipelines, loss curves. But Ai2's new OlmoEarth Platform post is a refreshing look at the other side: what it actually takes to run foundation models in production at planetary scale.
The OlmoEarth models are Earth observation foundation models pretrained on roughly 10TB of multimodal satellite data. Governments and NGOs are adapting them for deforestation monitoring, food security, and wildfire risk. But shipping a foundation model to partners who lack ML infrastructure teams means building the full stack: fine-tuning workflows, large-scale inference, and everything in between.
Ai2 built that stack. The result is infrastructure that can run inference across continent-scale areas in about a day, processing dozens of terabytes at fractions of a penny per square kilometer. The engineering choices they made—and the constraints they hit—are worth studying if you care about production ML systems.
Why satellite inference is different
Most ML inference is cheap and fast. An LLM processes a few kilobytes of text in milliseconds. A vision model analyzes a smartphone photo in under a second.
Earth observation operates at a completely different scale. A single inference job can move terabytes of data and run for hours. Inputs span multiple spectral bands, sensor types, and timesteps across huge geographic areas. They come from multiple providers using different projections and resolutions. Cloud cover obscures observations. And the output is itself a map, so every prediction must stay pixel-aligned with neighboring areas.
The kicker: prediction jobs often spend more time downloading and preparing imagery than running the model. Data pipelines become the bottleneck, not GPU compute.
Hardware staging: don't waste GPU cycles on I/O
Because data acquisition dominates runtime, you can't just throw everything at GPUs. Ai2 splits each job into three stages matched to different hardware:
- Data acquisition and preprocessing (CPU, high I/O): Fetch, reproject, align, normalize imagery. Write it in formats optimized for fast loading.
- Inference (GPU): Run the model's forward pass. Stream outputs directly to blob storage.
- Postprocessing (CPU): Stitch per-window outputs together, apply masks, export to user-friendly formats like Zarr or GeoTIFF.
This is smart staging. GPUs stay fully utilized on actual inference while multiprocess data loaders feed them continuously. Completed outputs stream to storage without blocking the GPU. The expensive hardware does only what it's uniquely good at.
Fan-out: one request, thousands of workers
OlmoEarth Run, the platform's execution layer, divides each geographic region into partitions sized for individual compute instances. Those partitions subdivide into smaller windows that the models actually process. Because windows are independent, you can parallelize aggressively.
A state-sized area becomes roughly 100 partitions. A continent-scale run becomes thousands. Adjacent partitions overlap slightly; the platform reconciles that overlap during assembly so no seam appears in the final raster.
Ai2 recently generated a wildfire-risk map covering all of North America. At peak, the run used roughly 19,600 CPUs and 994 GPUs in parallel, with network throughput exceeding 168 GB/s. That reduced an estimated 4,737 hours of serial compute to about 30.5 hours of wall-clock time—a 155× speedup.
But fan-out isn't unbounded. More workers push against cloud quotas, so parallelism is a per-run knob. Output resolution trades data volume and compute for detail. Model size trades GPU time for accuracy. Caching raw imagery trades storage for speed across repeated runs. The right setting depends on the task and budget.
Metadata indexing: don't overwhelm public APIs
Given a geographic region and time range, the platform first determines which satellite scenes should feed the model. That means identifying what was captured, where, and when across providers with different catalogs and formats.
Public STAC catalogs and open standards are great. But a large inference job can generate thousands of metadata queries at once—far more than external services like ESA's or Microsoft Planetary Computer's STAC APIs are designed to handle concurrently.
Ai2's solution: maintain their own metadata index, updated as new imagery is published. For datasets hosted through AWS Open Data, they receive an SNS notification for each new scene. When a provider doesn't offer a change stream, they poll the upstream index every few minutes.
This means their requests to external services follow the steady pace of new publications rather than sudden bursts from large inference jobs. Each index entry stores scene metadata plus pointers to every location where the underlying pixels are available. At runtime, the platform selects the best source and performs windowed reads against cloud-optimized formats like COG or Zarr, retrieving only the bytes needed rather than downloading entire scenes.
Best practices from data providers
Ai2 calls out three characteristics that made providers easiest to work with:
- Queue-based notifications when new imagery becomes available
- Storage on major cloud platforms without bespoke rate limits or availability bottlenecks
- Cloud-optimized formats that support ranged reads
If you're publishing Earth observation data (or any large-scale scientific data), this is your checklist.
Failure handling: expect chaos at scale
At planetary scale, failures are routine. A provider may be slow or briefly unavailable. Metadata may indicate imagery exists even when a required band is missing. Cloud cover may leave too few usable observations. Tasks crash.
The OlmoEarth Platform is designed to recover automatically. For each task within a stage and partition, it dynamically provisions a VM running their runner Docker container. The runner retrieves task parameters, executes the work, returns the result, and shuts down.
Because every task is reentrant and idempotent, intermittent failures can be handled safely by rerunning it. The platform responds with task tracking, automatic retries, fallback to alternate providers when available, and clear distinctions between retryable and fatal errors. A separate monitoring process detects stalled or stopped runners and restarts their tasks.
This is unglamorous infrastructure work, but it's the difference between a research demo and production software that partners rely on every day.
What's next: embeddings and agents
Ai2's roadmap is shaped by partner needs. A few highlights:
Automated model runs: Schedule inference jobs in advance or trigger them whenever the imagery index registers a new scene over an area of interest.
Change detection and alerts: Notify users when monitored landscapes change, so events like deforestation or flooding surface as alerts rather than rasters someone has to manually inspect.
Agentic tools and interfaces: Lower the barrier to using geospatial models. Agents can help with data curation, feature engineering, and identifying ways to improve fine-tuned models. The goal is letting users at any technical level do work that previously required an experienced ML researcher.
Embeddings: Develop a dedicated embedding model and precompute embeddings at global scale. For many tasks, running inference against those embeddings could replace a full forward pass over raw imagery, making workloads substantially faster and cheaper. Fine-tuning and direct inference will remain important for maximum performance, but embeddings could open up a much broader range of efficient applications.
Why this matters
Foundation models are only useful if you can actually run them. The gap between releasing an open model and delivering impact is infrastructure: cost-effective inference at the right time and place, monitoring, turning raw outputs into actionable insights, and verifying those outputs drive the outcomes partners want.
Ai2 has spent more than a decade operating platforms like Skylight and EarthRanger that users around the world rely on every day. That experience shows in the OlmoEarth Platform's design choices.
The post is worth reading in full if you're building production ML systems at scale. The challenges they hit—hardware staging, metadata indexing, failure recovery, fan-out limits—are not geospatial-specific. They're what happens when you take any foundation model from research to production at meaningful scale.
And the best practices they call out for data providers? Those apply broadly. Queue-based notifications, cloud-native storage, ranged reads—this is infrastructure 101 for any scientific data ecosystem.
We spend a lot of time talking about training runs and benchmark scores. It's refreshing to see detailed writing about what it takes to actually ship.