The AI Scaling Wall is the proposition that improvements in generative artificial intelligence (AI), driven by greater training data, computing power, and model size, are reaching diminishing returns. The idea proposes an imminent limit to the AI scaling laws, a term OpenAI coined in 2020 after Baidu researchers showed predictable performance gains from increasing resources such as training data or computing power.
In November 2024, reports indicated that OpenAI's GPT-4.5 model fell short of expectations, while Google's Gemini and Anthropic's Claude 3.5 Opus also faced delays or disappointing results. Ilya Sutskever said that gains from scaling up pre-training had plateaued, and Gary Marcus argued that models had reached diminishing returns. Sam Altman, Jensen Huang, and Dario Amodei disputed this.
Proposed causes include the exhaustion of human-made training data, which Epoch AI predicted could occur by 2028, and limits on computing power and energy. Synthetic data offers a partial remedy, but a Nature study warned of model collapse from its indiscriminate use. Amodei said training runs could cost $100 billion in coming years. Companies have turned to test-time compute, used in OpenAI's o1 model, which Huang described as a second scaling law. The improvement seen in GPT-5 was widely seen as underwhelming and fell below expectations. One 2025 analysis estimated that each tenfold rise in compute yields only one to two percentage points of MMLU-Pro accuracy. Analysts noted that United States export controls and safety policy were built on assumptions of continued growth in model size.