TabPFN

This article relies excessively on references to primary sources. Please improve this article by adding secondary or tertiary sources. Find sources: "TabPFN" – news · newspapers · books · scholar · JSTOR (July 2025) (Learn how and when to remove this message)

TabPFN (Tabular Prior-data Fitted Network) is a machine learning model for tabular datasets proposed in 2022. It uses a transformer architecture. It is intended for supervised classification and regression analysis on tabular datasets, particularly focusing on small- to medium-sized datasets. The latest version, TabPFN-3.5, was released in September 2026. It uses a single model for both classification and regression and is recommended by its developers for datasets with up to one million rows and 6,000 features.

History

TabPFN was first introduced in a 2022 pre-print and presented at ICLR 2023. TabPFN v2 was published in 2025 in Nature by Hollmann and co-authors. The source code is published on GitHub under a modified Apache License and on PyPi. With version 9.0.0 of the package in September 2026, the code was relicensed under the standard Apache License 2.0, removing the additional attribution requirement. Model weights are licensed separately. The TabPFN-2 weights remain under the modified Apache license, while the weights of TabPFN-2.5 and later versions are released under non-commercial licenses. The TabPFN-3.5 weight license permits research, evaluation and internal benchmarking. It prohibits commercial or production use of the model and its outputs, and offering the model as a hosted service, without a separate commercial license. Writing for ICLR blogs, McCarter states that the model has attracted attention due to its performance on small dataset benchmarks. TabPFN v2.5 was released on November 6, 2025. TabPFN-3 was released on May 12, 2026. TabPFN-3.5 was released on September 15, 2026.

Prior Labs, founded in 2024, aims to commercialize TabPFN.

As of April 2026, the open-source TabPFN repository had more than 6,000 stars on GitHub.

Overview and pre-training

TabPFN supports classification, regression and generative tasks. It leverages "Prior-Data Fitted Networks" models to model tabular data. By using a transformer pre-trained on synthetic tabular datasets, TabPFN avoids benchmark contamination and costs of curating real-world data.

TabPFN v2 was pre-trained on approximately 130 million such datasets. Synthetic datasets are generated using causal models or Bayesian neural networks; this can include simulating missing values, imbalanced data, and noise. Random inputs are passed through these models to generate outputs, with a bias towards simpler causal structures. During pre-training, TabPFN predicts the masked target values of new data points given training data points and their known targets, effectively learning a generic learning algorithm that is executed by running a neural network forward pass. The new dataset is then processed in a single forward pass without retraining. The model's transformer encoder processes features and labels by alternating attention across rows and columns. TabPFN v2 handles numerical and categorical features, missing values, and supports tasks like regression and synthetic data generation, while TabPFN-2.5 scales this approach to datasets with up to 50,000 rows and 2,000 features. Adaptations such as TabPFN-Wide increase the maximum number of features that can be used beyond 30,000. TabPFN-3 introduced a redesigned architecture with row-compression, an attention-based many-class decoder, native missing-value handling, and inference optimizations such as row chunking and a reduced key-value cache, with benchmark-validated regimes of up to 1 million rows with 200 features, 100,000 rows with 2,000 features, or 1,000 rows with 20,000 features. TabPFN-3.5 replaced the separate classification and regression checkpoints of earlier versions with a single multitask model. It doubled the width of the in-context transformer from 512 to 1,024 dimensions, which raised the parameter count to about 220 million, compared with 53 and 58 million for the TabPFN-3 classifier and regressor. The model added Fourier-feature encodings of cell values and features derived from each column's empirical distribution function. It dropped earlier preprocessing steps such as quantile transformations, robust scaling and singular value decomposition features. The synthetic pre-training data was adjusted to contain more datasets with high-cardinality categorical variables, many features, and grouped structure. The developers recommend TabPFN-3.5 for up to one million rows and 6,000 features, and state that up to 20,000 features can be used with more estimators. According to the report, the doubled model width makes TabPFN-3.5 up to twice as slow as TabPFN-3 on large training sets. A smaller variant, TabPFN-3.5-Fast, with about 84 million parameters, was released as an alpha version and is reported to run up to three times faster than TabPFN-3. Two further variants are available only through Prior Labs' commercial API and enterprise deployments: TabPFN-3.5-Plus, which adds proprietary text processing, and TabPFN-3.5-Thinking, which uses additional computation at inference time. Version 9.0.0 of the open-source package made TabPFN-3.5 the default model and added optional preprocessing of text and datetime columns.

Since TabPFN is pre-trained, in contrast to other deep learning methods, it does not require costly hyperparameter optimization.

Research

TabPFN is the subject of on-going research. Applications for TabPFN have been investigated for domains such as chemoproteomics, insurance risk classification, and metagenomics. In clinical research, TabPFN was used in a study on the early detection of pancreatic cancer from blood samples, where it was combined with metabolomic data and reported high diagnostic performance. As explainability remains critical for deploying these models in sensitive areas, there is ongoing research to enhance transparency and interpretability.

Applications

TabPFN has been used in industrial and biomedical contexts. Hitachi Ltd. has been reported to use the model for predictive maintenance in rail networks, with its use described as helping to identify track issues earlier and reduce manual inspections.

In the biomedical domain, Oxford Cancer Analytics has used TabPFN in the analysis of proteomic data in lung disease research.

A 2025 ML Contests report noted that the winners of DrivenData's PREPARE challenge used TabPFN to generate features for gradient-boosted decision tree models.

Limitations

TabPFN has been criticized for its "one large neural network is all you need" approach to modeling problems. Further, a 2025 study found that the performance of TabPFN v2 was limited on high-dimensional and large-scale datasets. Adaptations that handle these cases exist.

Scaling Mode

In late November 2025, Prior Labs introduced ‘‘Scaling Mode’’, an operating mode for TabPFN designed to remove the fixed upper bound on training set size. Earlier versions of TabPFN had been optimized and validated primarily for datasets of up to 100,000 rows, whereas Scaling Mode was reported to extend support to substantially larger datasets, with benchmarked experiments on datasets containing up to 10 million rows.

According to Prior Labs, Scaling Mode preserves the existing TabPFN architecture, including its alternating row-attention and column-attention design, as well as the same feature-count limits as prior releases. Inference remains based on a single forward pass rather than dataset-specific gradient-descent training, while scalability is described as being constrained primarily by available compute and memory resources.

Prior Labs reported benchmark results on an internal collection of datasets ranging from 1 million to 10 million rows across industry and scientific applications. In these benchmarks, Scaling Mode was compared with CatBoost, XGBoost, LightGBM, and TabPFN 2.5 using 50,000-row subsampling. The company stated that predictive performance improved monotonically with increasing training set size and that no diminishing returns from scaling were observed within the tested range.

Prior Labs also announced the release through company and executive social media channels.

TabPFN-3 later incorporated related scaling improvements, including row chunking and a reduced key-value cache, into the model architecture and inference pipeline.

See also

• XGBoost
• LightGBM
• CatBoost

References

External links

• TabPFN on GitHub