The XGBoost Pipeline Is Finally Vulnerable
For years, production ML on relational data has meant one thing: XGBoost on hand-engineered features. You hire a data scientist, they spend three weeks writing SQL to aggregate "average order value in the last 30 days" and 47 other features, you tune hyperparameters, and you ship. Rinse and repeat for every new prediction task.
NVIDIA's Kumo Relational is the first foundation model that credibly threatens that workflow. It's pre-trained once on synthetic relational data, then makes predictions on real databases in a single forward pass—no task-specific training, no feature engineering, no flattening tables into rows. And it beats gradient-boosted trees on expert-engineered features by nearly 4 AUROC points on average across RelBench V1 classification tasks.
On the SAP SALT enterprise benchmark, it surpasses AutoGluon ensembles by 8% in mean reciprocal rank out of the box, and by 16% after fine-tuning. These aren't academic curiosities—these are the kinds of margins that make you reconsider your production stack.
What Makes This Different From Every Other Tabular Model
Most foundation models for structured data work on flat tables. That's fine if your data lives in a CSV, but real enterprise data lives in dozens of tables connected by foreign keys: users join to orders join to products join to inventory events. The standard move is to flatten everything into one wide table through manual aggregation, which is both lossy and labor-intensive.
Kumo Relational operates directly on the relational structure. You connect your database, write a predictive query in their Predictive Query Language (PQL), and the model gathers up to 10,000 historical examples as context. Each example is an entity-centric subgraph at a point in time, paired with its ground-truth target—"this user churned" or "this item sold 142 units."
The architecture does three things that matter:
- Within-table reasoning: Column attention relates values within columns, row attention relates features within rows. The model learns which columns and rows are signal versus noise before propagating anything.
- Cross-table reasoning: Graph attention over primary-key/foreign-key relationships, interleaved with cross-sample attention across context examples. This is where it learns to link a user's profile to their transaction history.
- Temporal replay: For time-based predictions, Kumo automatically samples subgraphs up to an anchor timestamp and computes targets from events after that timestamp. No manual data leakage prevention.
Because the cross-table stage works on row embeddings rather than raw cell values, the design avoids quadratic blowup and scales to databases with 500B+ rows.
The Pre-Training Story: Why Synthetic Data Matters Here
Foundation models need diverse training data, but diverse relational databases are rarely public. NVIDIA solved this with synthetic generation based on PluRel (Kothapalli et al., ICML 2026)—members of the Kumo team co-authored that paper.
PluRel generates full relational databases in three stages: schema definition, foreign-key links that cluster realistically, and column values from Structural Causal Models with temporal trends and cross-table dependencies. The result is effectively unlimited pre-training data that reflects real-world relational structure.
The key insight: pre-training loss follows a power law as synthetic databases are added, and gains transfer to real databases. So Kumo was pre-trained on entirely synthetic data, starting with single tables and progressing to increasingly complex relational schemas.
This is important because it sidesteps the data access problem that has blocked relational foundation models for years. You don't need to license enterprise databases to train—you need a good simulator.
Benchmark Performance: Where It Wins and Where It Doesn't
NVIDIA evaluated Kumo on 41 tasks across 15 databases from RelBench V1 and V2, SAP SALT, and 4DBInfer. None were seen during pre-training. The results are striking:
- Beats gradient-boosted trees on expert features by ~4 AUROC points (RelBench V1 classification)
- Beats XGBoost on engineered features by 7 MRR points out of the box on SAP SALT, 13 points after fine-tuning
- Outperforms recent tabular foundation models by about 25% on SAP SALT
- Sits on the accuracy-efficiency Pareto front on the RelArena-α leaderboard
That last point deserves emphasis. The highest-scoring entry on RelArena-α reaches Elo 1,878 but takes about 7,300 seconds median inference time per 1,000 samples. Kumo hits Elo 1,842 at 17 seconds—roughly 430× faster at similar accuracy.
The model also shows robustness under degraded conditions: it performs well with as little as 0.2% of training data as context, stays stable when up to 75% of foreign-key links are removed, and barely degrades when many noisy columns are added.
Where It Struggles
Kumo works best on data with meaningful relationships and temporal dynamics. On tasks with very large training sets where all 10,000+ examples are high-quality positive samples, task-specific supervised models may still win. Fine-tuning is designed for exactly this case.
Predictive signals must also be reachable within a limited number of hops from the target entity. If your signal is buried six joins away, start with a focused query over a few relevant tables and expand the graph as needed.
Why This Matters Beyond Benchmarks
The foundation model shift in language and vision happened because three things aligned: better accuracy than task-specific models, radically simpler production deployment, and new capabilities (like agents) that weren't possible before.
Kumo checks all three boxes for relational data:
Accuracy today: The RelBench and SAP SALT numbers make a compelling case for replacing production XGBoost pipelines. When a pre-trained model beats expert-engineered features by 4-13 points, you have to take it seriously.
Deployment velocity: One model replaces separate pipelines for individual tasks. A new prediction becomes a query instead of a three-week project. ML teams can finally take on use cases that traditional ML made too laborious to build.
Agentic predictions: LLMs let agents read, write, and reason. Structured data foundation models let them anticipate what happens next—which customer will churn, which SKU will stockout—and act on it. Kumo's PQL interface is designed to be called by agents, not just data scientists.
That last point is the real unlock. Most agent demos today are read-only or write simple records. Agents that can predict future states change the game for workflow automation.
The Distribution Question
NVIDIA is shipping this through two channels: an open-source structured-data-models library that's GPU-native, and as an NVIDIA NIM (their managed inference service). The dual distribution is smart—open source for developer adoption, NIM for enterprise customers who want managed deployment.
The paper is public, and documentation is already live. No waitlist, no private beta. That's a confidence move.
The real test is whether enterprises trust a foundation model over pipelines they've tuned for years. Feature engineering feels artisanal and controllable. A pre-trained model feels like a black box. But when the black box is 4 AUROC points better and ships in days instead of weeks, the calculus shifts.
What Comes Next
If Kumo's numbers hold up in production, we're looking at the same transition that already happened in NLP: task-specific pipelines displaced by pre-trained models that adapt through prompting (or here, context examples). The question is how fast it happens.
XGBoost isn't going away tomorrow—it's embedded in too many production systems, and some teams will always prefer the control of hand-tuned features. But the marginal new prediction task? That's where Kumo wins. And once a few high-profile migrations succeed, the floodgates open.
The other thing to watch: whether other players rush to match this. Foundation models for relational data have been a known gap for years, but NVIDIA just demonstrated it's solvable with synthetic pre-training and the right architecture. Expect competition.
For now, though, Kumo is the first foundation model to credibly threaten the XGBoost-on-engineered-features pipeline. That's not incremental—that's a genuine paradigm shift for structured data ML.