A Foundation Model for Spreadsheets

A while ago I wrote that choosing a foundation model is a lot like choosing a car: people care about what the product does for them, not about the engine, so pick the model that is good enough and keep the architecture ready for the next one [10]. That post was about language models. A Machine Learning Street Talk episode with Frank Hutter makes the case that the same logic now reaches the most common data in any enterprise, the table. Hutter, who works on TabPFN at Prior Labs and co-authored the AdamW optimizer [8], describes TabPFN as a model where “we learn this entire algorithm that is executed in a forward pass” [1]. If that holds up, the per-dataset routine of encoding, picking a model and running a hyperparameter search becomes something you check instead of something you assume. This post looks at how it works, what the published numbers say, where the limits are, and what a small CPU run on four public tables showed.

Why tables resisted deep learning

Text and images have a property that tables lack. A word means roughly the same thing in every document, and pixels next to each other are related in almost every photo. That is why one huge corpus can teach a model something that transfers.

Tables don’t work that way. The Nature paper puts it plainly: “in tabular datasets the same value can mean fundamentally different things” [2]. A column of numbers in a drug-discovery table and a column in a materials-science table share nothing. Each table also brings its own mess: mixed types, missing values, outliers, columns that carry no signal. Hutter makes the same point in the interview: rows from a medical table tell you nothing about an insurance table, so there is nothing to transfer at the level of rows [1].

So the field settled on one model per dataset, usually a boosted-tree ensemble with a tuning loop around it. The Nature paper notes that gradient-boosted decision trees “have dominated tabular data for the past 20 years” [2]. They are fast, tolerate messy inputs, and good enough that most deep-learning attempts on tables didn’t generalize to new datasets.

Learning the algorithm instead of the model

TabPFN moves the work to a different place. What can transfer between tables, the argument goes, is patterns: how features interact, what distributions look like, how causes produce effects [1]. A model that has seen enough of those patterns could recognise them in a table it has never seen.

There is no large corpus of real tables to learn those patterns from, so Prior Labs generates one. Each synthetic dataset comes from a randomly sampled structural causal model: a random graph of causes and effects, with small neural networks, decision-tree rules, categorical discretisation and noise on its edges [2]. Post-processing adds warping, binning and missing values. The Nature model was trained on more than 100 million such synthetic tasks [2].

The transformer is then trained on one task, over and over: given the labelled rows of a synthetic table and some unlabelled ones, predict the missing labels. Because each training example is a whole dataset, what the model learns is a fitting procedure. The paper frames this as approximate Bayesian inference: the network learns to approximate the posterior predictive distribution for the prior defined by the synthetic data [2].

At prediction time you hand it your real table. The training rows with their labels and the test rows go in together as context, and the predictions come out of forward passes, by default a small four-way ensemble for classification [2]. Nothing is trained on your data. This is in-context learning, the same mechanism large language models use when you put examples in a prompt, applied to rows and columns.

Why not hand the table to a language model directly? Hutter’s arithmetic in the interview is a good answer: a table with a million rows and a thousand columns is a billion cells, at three or four tokens per number that is billions of tokens of context, and a model that reads rows in sequence doesn’t know that swapping two rows gives the same dataset [1]. Language models help around the model, with feature engineering that needs world knowledge. The statistics over many rows are what a tabular model is built for.

Two design details matter for practitioners. First, the architecture gives each cell its own representation and alternates attention across a row and down a column, a design meant to make the order of rows and features irrelevant [2]. Second, test rows attend only to training rows, never to each other [2]. That keeps the familiar contract of a scikit-learn model: a prediction for one row doesn’t change because of what else is in the batch. It also lets the work on the training rows be cached and reused for later predictions, which the package offers as an option (fit_mode="fit_with_cache"), not as the default [5].

What the Nature paper measured

The 2025 Nature paper describes TabPFN v2 and is careful about scope. On 29 classification and 28 regression datasets with up to 10,000 rows and 500 features from two established benchmark suites, TabPFN with default settings “outperforms an ensemble of the strongest baselines tuned for 4 h”, taking 2.8 seconds on average to fit and predict for classification, with TabPFN on a consumer GPU [2]. The authors compared against random forest, XGBoost, CatBoost, LightGBM, linear models, SVMs and MLPs, and against AutoGluon, a stacked AutoML ensemble [2]. In the per-dataset view, CatBoost still won some datasets; TabPFN won most [2].

The same paper also states its limits:

• “the inference speed of TabPFN may be slower than highly optimized approaches such as CatBoost” — for a 10,000-row, 10-column dataset, predicting one sample took 0.2 seconds versus 0.0002 seconds for default CatBoost [2]
• memory grows linearly with the number of cells (rows times features), compute quadratically in rows and in features [2]
• the evaluation stopped at 10,000 samples and 500 features, and “for larger datasets and highly non-smooth regression datasets, approaches such as CatBoost, XGB or AutoGluon are likely to outperform TabPFN” [2]

For explanations, the paper shows SHAP feature attributions, the same post-hoc tool people use for boosted trees [2]. It also returns a full predictive distribution rather than a single number, including bimodal ones; to approximate the same with CatBoost, the authors had to train several quantile models [2].

An independent paper written for statisticians reaches a similar view from outside the company: an out-of-the-box TabPFN sometimes beat specialised methods on tasks such as prediction under covariate shift and treatment-effect estimation, which the authors partly explain by its ability to adapt to both parametric and non-parametric structure [7].

Where the model family is now

The paper is from early 2025, and the model has moved on. TabPFN-3.5, released in September 2026, has recommended limits of one million training rows and 20,000 columns, with up to 6,000 columns recommended for better results, and the two limits trade off against each other [4]. The open-source package now turns text and date columns into features itself, and a single checkpoint serves classification and regression [3] [14].

Its technical report claims first place on seven tabular benchmarks, including TabArena, a living benchmark for tabular methods [3]. That is a vendor report, so read it as such. What I find more useful are the places where it says the model does not lead yet. On BeyondArena, a benchmark built around grouped and temporal splits, very large tables and text-heavy data, “tuned and ensembled MLPs retain the highest performance on grouped, temporal, and large datasets”, although the report says the gap has narrowed [3]. The report also notes that TabPFN-3.5 runs slower than its predecessor on large training sets because the model is wider, and that a separate Fast variant trades a little accuracy for speed [3].

So my summary for late 2026 is this. On TabArena, the report shows tabular foundation models at the top of the leaderboard, and the strongest default one ahead of AutoML systems that tune and ensemble models for hours [3]. On data where the test set comes from a new group or a later time period, which is a lot of real enterprise data, a well-tuned conventional model is still a serious contender.

A small test on four public tables

To get a feel for it, a short script compared TabPFN v2, the open model from the Nature paper, with XGBoost on four public datasets of at most 1,000 rows: scikit-learn’s breast-cancer data, and German Credit, Pima diabetes and blood-transfusion from OpenML. Each dataset got five stratified 80/20 splits (seeds 0 to 4). XGBoost ran once with library defaults and once with a 30-configuration random search using 3-fold cross-validation, scored by AUC, over tree count (50 to 600), depth (2 to 8), learning rate (0.01 to 0.3), row and column subsampling, minimum child weight and L2 regularisation. Categorical columns were ordinal-encoded for both models, and TabPFN was told which columns were categorical. Times are wall-clock for fit plus predict, including the search for tuned XGBoost. Everything ran on an 8-vCPU machine without a GPU, with tabpfn 9.2.0, XGBoost 3.4.1 and scikit-learn 1.9.1.

Mean test ROC AUC over five splits. Higher is better. AUC measures ranking only; calibration was not checked.

TabPFN had the higher test AUC on 17 of the 20 splits against the tuned XGBoost, and on 19 of 20 against default XGBoost. The margins against tuned XGBoost are small. Taking TabPFN’s AUC minus tuned XGBoost’s on each split, the mean and standard deviation of the five differences are 0.0035 ± 0.0035 on breast-cancer, 0.0081 ± 0.0199 on credit-g, 0.0078 ± 0.0071 on diabetes and 0.0245 ± 0.0037 on blood-transfusion. On breast-cancer and credit-g the mean gap is no larger than its spread, so those two are effectively a draw. The timing is the part worth noting: on CPU, TabPFN took a median of about 5 seconds per split, roughly what the small XGBoost search needed, while default XGBoost finished in 0.03 seconds. The comparison script, the per-split results and the run log are in [18].

This is four well-known datasets and a light tuning budget, so it proves very little on its own. Well-known matters here. TabPFN never trains on real data, but its authors chose its settings on a separate set of real development datasets [2], and all four tables above are staples of tabular benchmarking. The test that matters is your own table. It does match the paper’s direction: without any tuning, the foundation model landed where a tuned boosted-tree model lands, and above an untuned one.

The code is the familiar scikit-learn shape, which is part of the appeal [5]:

from tabpfn import TabPFNClassifier
from tabpfn.constants import ModelVersion

clf = TabPFNClassifier.create_default_for_version(ModelVersion.V2, device="cpu")
clf.fit(X_train, y_train)          # stores the context, no gradient steps
proba = clf.predict_proba(X_test)  # forward passes over train + test rows

The package README adds two tips that run against habit: don’t one-hot encode or scale the inputs, and predict in batches, because each predict call recomputes the training-set context [5].

Licenses for code and weights

The code in the tabpfn package is Apache 2.0. The weights are licensed separately [5]. TabPFN v2 weights use the Prior Labs License 1.1, which its text describes as Apache 2.0 with one added paragraph on attribution [15]. That paragraph asks anyone who distributes the weights, or a product that contains them, to include the license and display “Built with PriorLabs-TabPFN”, and to put “TabPFN” at the start of the name of any model built from them that is made available. Internal benchmarking and testing without external communication needs no attribution. The rest is Apache 2.0, which does not restrict commercial use, and that is why the test above uses these weights. The TabPFN-2.5, 2.6, 3 and 3.5 weights are under non-commercial licenses [5] [4].

Prior Labs’ license FAQ is specific about what that means. Internal testing and benchmarking on your own data is allowed. Production use, client projects, using the outputs to support business decisions, and competitive benchmarking for procurement need a commercial license or an API agreement, and fine-tuned derivatives inherit the same restriction [4]. If you plan to evaluate TabPFN-3.5 for a real decision, involve whoever handles software licensing before you start, not after the results look good.

For context on the company: SAP completed its acquisition of Prior Labs in July 2026, and Prior Labs continues to operate as an independent entity [9].

What this changes for model choice

The argument from my earlier posts still holds, and it extends to tables. A system is more than its model [11], and the model that is just good enough, inside an architecture that can take the next one, is usually the right pick [10]. What changes is the cost of the first baseline. Getting a strong tabular baseline used to mean an afternoon of feature encoding and a tuning job. With a tabular foundation model, it’s a fit and a predict.

That moves the data scientist’s attention to the questions no model answers for them. Hutter’s list in the interview is a good one: where the data comes from, whether it will drift, and whether there are fairness or causality problems in the framing [1]. I argued something similar about the role of data scientists last year [12]. Less time on hyperparameters means more time on whether the prediction is the right one to make.

Here is how I would use it today:

• small or medium table, independent rows, a prototype or internal analysis: start with a tabular foundation model as the baseline, then decide whether a bespoke model earns its keep
• grouped or time-ordered data, very large tables, or hard latency limits per prediction: keep the tuned boosted-tree or MLP pipeline in the comparison, because the published results say it can still win there
• anything heading to production: check the license of the exact weights first

If you’re running this on AWS

Prior Labs documents an Amazon SageMaker path through AWS Marketplace [6]. The TabPFN-3.5-Plus listing is free under the non-commercial TabPFN-3.5 License, with no software charge [16]. As with any SageMaker model package from AWS Marketplace, it runs in your own AWS account and you pay for the infrastructure it uses [17]. The model package declares only GPU instance types, such as ml.g5.xlarge, and the guide asks for an inference AMI with NVIDIA driver 535 or later, set through InferenceAmiVersion on the production variant [6].

For larger tables, the guide recommends deploying with asynchronous inference [6]. SageMaker Asynchronous Inference queues requests with payloads up to 1 GB and processing times up to one hour, and can scale the endpoint to zero when idle [13]. The tabpfn-client SDK wraps the endpoint behind the same fit/predict interface as the local package [6].

I haven’t deployed the Marketplace listing for this post. The test above ran the open v2 weights locally on CPU, which is the quickest way to see whether the approach fits your data before you think about endpoints.

What is your default baseline for a 5,000-row table today, and would you swap it?

Sources

• [1] Machine Learning Street Talk, “The AI That Replaces Hours of Model Tuning – Frank Hutter” (YouTube, 2026) · interview on TabPFN, in-context learning and the data-scientist role.

• [2] Hollmann et al., “Accurate predictions on small data with a tabular foundation model,” Nature 637, 319–326 (2025) · TabPFN v2: method, synthetic causal prior, benchmarks, limitations.

• [3] Prior Labs, “TabPFN-3.5: Technical Report,” arXiv:2609.17895 (2026) · current model family, benchmark results, where tuned models still lead.

• [4] Prior Labs documentation: Models · row and column limits per version, model license FAQ.

• [5] PriorLabs/TabPFN on GitHub · installation, CPU limits, usage tips, code and weight licenses.

• [6] Prior Labs documentation: Amazon SageMaker · Marketplace listings, GPU requirement, async inference.

• [7] Zhang, Tan, Tian and Li, “TabPFN: One Model to Rule Them All?,” arXiv:2505.20003 · a statistics view of TabPFN as approximate Bayesian inference.

• [8] Loshchilov and Hutter, “Decoupled Weight Decay Regularization” (AdamW), arXiv:1711.05101 · the AdamW paper.

• [9] SAP News, “SAP Completes Acquisition of Prior Labs” (17 July 2026) · acquisition closed; Prior Labs remains an independent entity.

• [10] How Do You Choose the Foundation Model for Your Generative AI App — Like Your Car? · my earlier post on picking a good-enough model.

• [11] LLMs for the rescue?! Or are we actually building Compound AI Systems? · my earlier post on systems versus models.

• [12] Controversial take: Data Scientists are becoming extinct · my earlier post on how the data-science role is changing.

• [13] Amazon SageMaker AI Developer Guide: Asynchronous inference · payloads up to 1 GB, processing up to one hour, scale to zero.

• [14] Prior Labs documentation: TabPFN-3.5 changelog · package 9.0.0 with built-in text and datetime preprocessing, single multitask checkpoint, integrations.

• [15] Prior Labs License, Version 1.1 (TabPFN v2 classifier weights, Hugging Face) · license text, including Paragraph 10 on attribution.

• [16] AWS Marketplace: TabPFN-3.5-Plus · listing, free of charge under the non-commercial TabPFN-3.5 License, infrastructure costs extra.

• [17] AWS Marketplace Seller Guide: Machine learning product pricing · buyers run SageMaker products in their own AWS account and pay infrastructure plus software price.

• [18] tabular-foundation-models, schristoph-blog-samples on GitHub: compare.py, results.csv, run-log.txt