Timeline of machine learning

This page is a timeline of machine learning. Major discoveries, achievements, milestones, and other major events in machine learning are included.

Overview

DecadeSummary
pre-
1950		Statistical methods are discovered and refined.
1950sPioneering machine learning research is conducted using simple algorithms.
1960sBayesian methods are introduced for probabilistic inference in machine learning.
1970s'AI winter' caused by pessimism about machine learning effectiveness.
1980sRediscovery of backpropagation causes a resurgence in machine learning research.
1990sWork on Machine learning shifts from a knowledge-driven approach to a data-driven approach. Scientists begin creating programs for computers to analyze large amounts of data and draw conclusions – or "learn" – from the results. Support-vector machines (SVMs) and recurrent neural networks (RNNs) become popular. The fields of computational complexity via neural networks and super-Turing computation started.
2000sSupport-Vector Clustering and other kernel methods and unsupervised machine learning methods become widespread.
2010sDeep learning becomes feasible, which leads to machine learning becoming integral to many widely used software services and applications. Deep learning spurs huge advances in vision and text processing.
2020s Generative AI leads to revolutionary models, creating a proliferation of foundation models, both proprietary and open source, notably enabling products such as advanced chatbots and text to image models. Machine learning and AI enter the wider public consciousness. The commercial potential of AI based on machine learning causes large increases in the valuations of companies linked to AI.

Timeline

Year Event type Caption Event
1763 Publication Foundations of Bayesian inference Thomas Bayes's An Essay towards solving a Problem in the Doctrine of Chances is published posthumously, communicated by Richard Price. It addresses inverse probability and provides a foundation for Bayes' theorem.
1805 Publication Least squares Adrien-Marie Legendre publishes the least squares method in Nouvelles méthodes pour la détermination des orbites des comètes, presenting a method for estimating quantities from observations.
1812 Publication Analytical theory of probability Pierre-Simon Laplace publishes Théorie analytique des probabilités, developing methods in probability theory, including the calculation of inverse probabilities associated with Bayesian inference.
1843 Conceptual proposal Symbolic computation Ada Lovelace publishes her translation of Luigi Federico Menabrea's account of the Analytical Engine, accompanied by additional notes. She discusses the possibility of using the machine to manipulate symbols and compose music, extending its proposed applications beyond numerical calculation.
1913 Research Markov chains and text analysis Andrey Markov applies a probabilistic model of dependent events to sequences of vowels and consonants in Alexander Pushkin's Eugene Onegin, demonstrating an application of Markov chains to text analysis.
1943 Model introduction Mathematical model of neural activity Warren McCulloch and Walter Pitts publish a mathematical model of neural activity using networks of simplified, all-or-none neurons, relating their operation to mathematical logic.
1950 Conceptual proposal Learning machines In Computing Machinery and Intelligence, Alan Turing discusses machines that learn through training and proposes developing a machine that could be educated in a manner analogous to a child.
1951 System development SNARC Marvin Minsky develops SNARC, an early neural-network machine that uses reinforcement to modify its behavior.
1952 System development Samuel's checkers program Arthur Samuel develops a checkers program for the IBM 701. He subsequently adds learning mechanisms to the program.
1957 Model introduction Perceptron Frank Rosenblatt describes the perceptron, a model for learning to recognize patterns, in a report produced at the Cornell Aeronautical Laboratory.
1959 Publication Machine learning applied to checkers Arthur Samuel publishes "Some Studies in Machine Learning Using the Game of Checkers", describing methods that allow a checkers program to improve its play through experience.
1963 Publication Matchbox game learning Donald Michie publishes an analysis of a game-learning model implemented using matchboxes and beads. The system, known as MENACE, learns to play tic-tac-toe through trial and error.
1967 Theoretical result Nearest-neighbor classification Thomas M. Cover and Peter E. Hart publish a theoretical analysis of nearest-neighbor classification, including a bound relating its asymptotic error rate to the optimal classification error.
1969 Publication Limitations of perceptrons Marvin Minsky and Seymour Papert publish Perceptrons, examining the mathematical properties and limitations of the perceptron models considered in the book.
1970 Algorithm development Reverse-mode automatic differentiation Seppo Linnainmaa describes a general reverse-mode method for automatic differentiation in his master's thesis. Reverse-mode differentiation provides the computational basis for backpropagation in neural networks.
1976 Research Transfer learning in neural networks Stevo Bozinovski and Ante Fulgosi investigate transfer learning in neural networks, examining how training on one task can affect learning on another.
1979 Demonstration Stanford Cart The Stanford Cart navigates a room containing chairs without human intervention, using computer vision to identify obstacles and plan its movement.
1979 Model introduction Neocognitron Kunihiko Fukushima proposes the neocognitron, a multilayer neural network designed to recognize patterns despite changes in their position. An English-language account is published in 1980.
1981 Publication Adaptive pattern classification Stevo Bozinovski publishes Teaching Space: A Representation Concept for Adaptive Pattern Classification, a technical report examining teaching and representation in adaptive pattern classification.
1982 Model introduction Hopfield networks John Hopfield describes a recurrent neural network that can function as a content-addressable memory. The model becomes known as the Hopfield network.
1982 Research Crossbar Adaptive Array Stevo Bozinovski reports a self-learning system using secondary reinforcement. The Crossbar Adaptive Array model is applied to tasks involving delayed reinforcement, including path learning and inverted pendulum control.
1986 Publication Explanation-based learning Gerald DeJong and Raymond Mooney publish an account of explanation-based learning, in which a system uses domain knowledge to explain an example and generalize from that explanation.
1986 System development NETtalk Terrence Sejnowski and Charles Rosenberg describe NETtalk, a neural network trained to convert written English into representations of its pronunciation.
1986 Research Backpropagation and learned representations David Rumelhart, Geoffrey Hinton, and Ronald J. Williams demonstrate how backpropagation can train multilayer neural networks to learn internal representations.
1989 Theoretical result Universal approximation Kurt Hornik, Maxwell Stinchcombe, and Halbert White establish universal approximation results for multilayer feedforward neural networks, showing that networks with sufficient hidden units can approximate broad classes of functions under specified conditions.
1989 Algorithm introduction Q-learning Christopher Watkins introduces Q-learning in his doctoral thesis, Learning from Delayed Rewards, as a method of reinforcement learning.
1989 Commercial software Evolver Axcelis releases Evolver, a commercial software package applying genetic algorithms to optimization on personal computers.[citation needed]
1992 Demonstration TD-Gammon Version 2.0 of Gerald Tesauro's TD-Gammon is demonstrated at the World Cup of Backgammon. The program uses a neural network trained through self-play and temporal-difference learning, and is evaluated against leading human players.
1995 Algorithm introduction Random decision forests Tin Kam Ho publishes "Random decision forests", describing an ensemble method that combines multiple decision tree classifiers.
1995 Algorithm development Support-vector networks Corinna Cortes and Vladimir Vapnik publish "Support-vector networks", describing support-vector classification for data that are not completely separable.
1997 Competition result Deep Blue defeats Kasparov IBM's Deep Blue defeats reigning world chess champion Garry Kasparov in a six-game match.
1997 Model introduction Long short-term memory Sepp Hochreiter and Jürgen Schmidhuber introduce long short-term memory (LSTM), a recurrent neural-network architecture designed to learn dependencies over extended time intervals.
1998 Dataset publication MNIST Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner describe the MNIST database in their work on document recognition. Derived from NIST handwritten-digit datasets, it contains 60,000 training images and 10,000 test images.
2002 Software development Torch Ronan Collobert, Samy Bengio, and Johnny Mariéthoz describe Torch, a modular software library for machine learning.
2006 Competition launch Netflix Prize Netflix launches the Netflix Prize, offering a prize for reducing the root-mean-square error of its movie-rating predictions by at least 10% relative to its existing recommendation system.
2009 Dataset publication ImageNet Jia Deng and colleagues introduce ImageNet, a large image database organized according to the WordNet hierarchy and intended to support research in visual recognition.
2010 Platform launch Kaggle Kaggle is founded as a platform for data-science and machine-learning competitions.
2011 Competition result Watson wins at Jeopardy! IBM's Watson defeats Ken Jennings and Brad Rutter in a televised Jeopardy! competition, using a question-answering system that combines language processing, information retrieval, and evidence evaluation.
2012 Research Unsupervised visual feature learning Quoc Le and colleagues report that a large neural network trained on unlabeled images sampled from YouTube videos learns features responsive to objects including human faces and cat faces.
2012 Competition result AlexNet Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton describe AlexNet, a deep convolutional neural network. An ensemble of their networks wins the 2012 ImageNet Large Scale Visual Recognition Challenge, achieving a top-five test error of 15.3%, compared with 26.2% for the second-place entry.
2013 Model introduction Word2vec Tomáš Mikolov and colleagues introduce efficient methods for learning word embeddings, including the continuous bag-of-words and skip-gram architectures associated with word2vec.
2014 Research DeepFace Researchers describe DeepFace, a neural-network system for face verification. They report an accuracy of 97.35% on the Labeled Faces in the Wild benchmark.
2014 System presentation Sibyl Tushar Chandra presents Sibyl, an internal Google system for large-scale machine learning, in a keynote at the International Conference on Dependable Systems and Networks.
2016 Publication AlphaGo David Silver and colleagues publish an account of AlphaGo, which combines deep neural networks with Monte Carlo tree search. The paper reports the system's five-game victory over professional Go player Fan Hui.
2017 Model introduction Transformer Ashish Vaswani and colleagues introduce the transformer in "Attention Is All You Need". The architecture uses attention mechanisms without recurrent or convolutional layers in its sequence-transduction model.
2018 Competition result AlphaFold at CASP13 DeepMind's AlphaFold achieves the highest performance among participating methods in the free-modelling assessment at CASP13, using deep learning to support protein structure prediction.
2020 Competition result AlphaFold 2 at CASP14 AlphaFold 2 achieves substantially higher protein-structure prediction accuracy than other participating methods at CASP14. A detailed account of the system is published in 2021.
2022 Public release ChatGPT On 30 November, OpenAI releases ChatGPT as a research preview. The conversational system is based on a model from the GPT-3.5 series and is trained using human feedback.
2023 Model introduction LLaMA Meta introduces LLaMA, a family of language models ranging from 7 billion to 65 billion parameters, and makes the models available to the research community under its initial access arrangements.
2023 Model introduction GPT-4 OpenAI introduces GPT-4, describing it as a multimodal model capable of accepting text and image inputs and generating text outputs.
2024 Model introduction AlphaFold 3 Researchers at Google DeepMind and Isomorphic Labs introduce AlphaFold 3, a model for predicting the structures of biomolecular complexes containing proteins, nucleic acids, small molecules, ions, and modified residues.

See also

References

Citations

  1. ↑ Solomonoff, R.J. (June 1964). "A formal theory of inductive inference. Part II". Information and Control. 7 (2): 224–254. doi:10.1016/S0019-9958(64)90131-7.
  2. ↑ Marr 2016.
  3. ↑ Siegelmann, H.T.; Sontag, E.D. (February 1995). "On the Computational Power of Neural Nets". Journal of Computer and System Sciences. 50 (1): 132–150. doi:10.1006/jcss.1995.1013.
  4. ↑ Siegelmann, Hava (1995). "Computation Beyond the Turing Limit". Journal of Computer and System Sciences. 238 (28): 632–637. Bibcode:1995Sci...268..545S. doi:10.1126/science.268.5210.545. PMID 17756722. S2CID 17495161.
  5. ↑ Ben-Hur, Asa; Horn, David; Siegelmann, Hava; Vapnik, Vladimir (2001). "Support vector clustering". Journal of Machine Learning Research. 2: 51–86.
  6. ↑ Hofmann, Thomas; Schölkopf, Bernhard; Smola, Alexander J. (2008). "Kernel methods in machine learning". The Annals of Statistics. 36 (3): 1171–1220. arXiv:math/0701907. doi:10.1214/009053607000000677. JSTOR 25464664.
  7. ↑ Bennett, James; Lanning, Stan (2007). "The netflix prize" (PDF). Proceedings of KDD Cup and Workshop 2007.
  8. ↑ Bayes, Thomas (1763). "An Essay towards Solving a Problem in the Doctrine of Chances". Philosophical Transactions of the Royal Society of London. 53: 370–418. doi:10.1098/rstl.1763.0053.
  9. ↑ Legendre, Adrien-Marie (1805). Nouvelles méthodes pour la détermination des orbites des comètes (in French). Paris: Firmin Didot.
  10. ↑ "Adrien-Marie Legendre: Royal Astronomical Society obituary". MacTutor History of Mathematics Archive. University of St Andrews.
  11. ↑ O'Connor, J. J.; Robertson, E. F. "Pierre-Simon Laplace". MacTutor History of Mathematics Archive. University of St Andrews.
  12. ↑ Zwolak, Justyna (2023-03-22). "Ada Lovelace: The World's First Computer Programmer Who Predicted Artificial Intelligence". Taking Measure. National Institute of Standards and Technology.
  13. ↑ Markov, A. A. (2006) [1913]. "An Example of Statistical Investigation of the Text Eugene Onegin Concerning the Connection of Samples in Chains". Science in Context. 19 (4): 591–600. doi:10.1017/S0269889706001074.
  14. ↑ McCulloch, Warren S.; Pitts, Walter (1943). "A logical calculus of the ideas immanent in nervous activity". Bulletin of Mathematical Biophysics. 5 (4): 115–133. Bibcode:1943BMaB....5..115M. doi:10.1007/BF02478259.
  15. ↑ Turing, A. M. (1950). "Computing Machinery and Intelligence". Mind. 59 (236): 433–460. doi:10.1093/mind/LIX.236.433.
  16. ↑ "Marvin Minsky". IEEE Computer Society. 13 April 2018.
  17. ↑ Copeland, B. Jack (2000). "The Turing Test" (PDF).
  18. ↑ Rosenblatt, Frank (1957). The Perceptron, a Perceiving and Recognizing Automaton (Project Para) (Report). Cornell Aeronautical Laboratory.
  19. ↑ Samuel, Arthur L. (1959). "Some Studies in Machine Learning Using the Game of Checkers". IBM Journal of Research and Development. 3 (3): 210–229. doi:10.1147/rd.33.0210.
  20. ↑ Michie, Donald (1963). "Experiments on the Mechanization of Game-Learning Part I. Characterization of the Model and its parameters". The Computer Journal. 6 (3): 232–236. doi:10.1093/comjnl/6.3.232.
  21. ↑ Cover, Thomas M.; Hart, Peter E. (1967). "Nearest neighbor pattern classification" (PDF). IEEE Transactions on Information Theory. 13 (1): 21–27. Bibcode:1967ITIT...13...21C. doi:10.1109/TIT.1967.1053964.
  22. ↑ Minsky, Marvin; Papert, Seymour (1969). Perceptrons: An Introduction to Computational Geometry. MIT Press.
  23. ↑ Griewank, Andreas (2012). "Who invented the reverse mode of differentiation?". Optimization Stories. Documenta Mathematica, Extra Volume ISMP. Vol. 6. pp. 389–400. doi:10.4171/DMS/6/38. ISBN 978-3-936609-58-5.
  24. ↑ Bozinovski, Stevo (2020). "Reminder of the First Paper on Transfer Learning in Neural Networks, 1976". Informatica. 44 (3): 291–302. doi:10.31449/inf.v44i3.2828.
  25. ↑ "Stanford's robotics legacy". Stanford Report. Stanford University. January 2019.
  26. ↑ Fukushima, Kunihiko (2020-01-24). "Neocognitron: Deep Convolutional Neural Network". Institute of Electronics, Information and Communication Engineers.
  27. ↑ Fukushima, Kunihiko (1980). "Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position". Biological Cybernetics. 36 (4): 193–202. doi:10.1007/BF00344251. PMID 7370364.
  28. ↑ Bozinovski, Stevo (1981). Teaching Space: A Representation Concept for Adaptive Pattern Classification (PDF) (Report). University of Massachusetts Amherst.
  29. ↑ Hopfield, John J. (1982). "Neural networks and physical systems with emergent collective computational abilities". Proceedings of the National Academy of Sciences. 79 (8): 2554–2558. Bibcode:1982PNAS...79.2554H. doi:10.1073/pnas.79.8.2554. PMC 346238. PMID 6953413.
  30. ↑ Bozinovski, Stevo (1982). "A self-learning system using secondary reinforcement". In Trappl, Robert (ed.). Cybernetics and Systems Research. North-Holland. pp. 397–402. ISBN 9780444864888.
  31. ↑ Bozinovski, Stevo (2014). "Modeling Mechanisms of Cognition-emotion Interaction in Artificial Neural Networks, since 1981". Procedia Computer Science. 41: 255–263. doi:10.1016/j.procs.2014.11.111.
  32. ↑ DeJong, Gerald; Mooney, Raymond (1986). "Explanation-Based Learning: An Alternative View" (PDF). Machine Learning. 1 (2): 145–176. doi:10.1023/A:1022898111663.
  33. ↑ Sejnowski, Terrence J.; Rosenberg, Charles R. (1986). NETtalk: A parallel network that learns to read aloud (PDF) (Report). Johns Hopkins University.
  34. ↑ Rumelhart, David E.; Hinton, Geoffrey E.; Williams, Ronald J. (1986). "Learning representations by back-propagating errors". Nature. 323 (6088): 533–536. Bibcode:1986Natur.323..533R. doi:10.1038/323533a0.
  35. ↑ Hornik, Kurt; Stinchcombe, Maxwell; White, Halbert (1989). "Multilayer feedforward networks are universal approximators". Neural Networks. 2 (5): 359–366. doi:10.1016/0893-6080(89)90020-8.
  36. ↑ Watkins, Christopher John Cornish Hellaby (1989). Learning from Delayed Rewards (PhD). University of Cambridge.
  37. ↑ Tesauro, Gerald (1995). "Temporal Difference Learning and TD-Gammon". Communications of the ACM. 38 (3): 58–68. doi:10.1145/203330.203343.
  38. ↑ Ho, Tin Kam (1995). "Random decision forests". Proceedings of the Third International Conference on Document Analysis and Recognition. Vol. 1. IEEE. pp. 278–282. doi:10.1109/ICDAR.1995.598994.
  39. ↑ Cortes, Corinna; Vapnik, Vladimir (1995). "Support-vector networks". Machine Learning. 20 (3): 273–297. doi:10.1007/BF00994018.
  40. ↑ "Deep Blue". IBM. 10 November 2025.
  41. ↑ Hochreiter, Sepp; Schmidhuber, Jürgen (1997). "Long short-term memory". Neural Computation. 9 (8): 1735–1780. doi:10.1162/neco.1997.9.8.1735. PMID 9377276.
  42. ↑ LeCun, Yann; Bottou, Léon; Bengio, Yoshua; Haffner, Patrick (1998). "Gradient-based learning applied to document recognition". Proceedings of the IEEE. 86 (11): 2278–2324. Bibcode:1998IEEEP..86.2278L. doi:10.1109/5.726791.
  43. ↑ Collobert, Ronan; Bengio, Samy; Mariéthoz, Johnny (2002). Torch: A modular machine learning software library (Report). Idiap Research Institute.
  44. ↑ Bennett, James; Lanning, Stan (2007). "The Netflix Prize" (PDF). Proceedings of the KDD Cup and Workshop 2007.
  45. ↑ Deng, Jia; Dong, Wei; Socher, Richard; Li, Li-Jia; Li, Kai; Li, Fei-Fei (2009). "ImageNet: A large-scale hierarchical image database" (PDF). 2009 IEEE Conference on Computer Vision and Pattern Recognition. pp. 248–255. doi:10.1109/CVPR.2009.5206848.
  46. ↑ Li, Fei-Fei (2017-03-08). "Welcome Kaggle to Google Cloud".
  47. ↑ "Watson and Jeopardy!". IBM. 10 November 2025.
  48. ↑ Le, Quoc V.; Ranzato, Marc'Aurelio; Monga, Rajat; Devin, Matthieu; Chen, Kai; Corrado, Greg S.; Dean, Jeff; Ng, Andrew Y. (2012). "Building high-level features using large scale unsupervised learning". Proceedings of the 29th International Conference on Machine Learning. arXiv:1112.6209.
  49. ↑ Krizhevsky, Alex; Sutskever, Ilya; Hinton, Geoffrey E. (2012). "ImageNet Classification with Deep Convolutional Neural Networks". Advances in Neural Information Processing Systems. Vol. 25. pp. 1097–1105.
  50. ↑ Mikolov, Tomas; Chen, Kai; Corrado, Greg; Dean, Jeffrey (2013). "Efficient Estimation of Word Representations in Vector Space". International Conference on Learning Representations, Workshop Track. arXiv:1301.3781.
  51. ↑ Taigman, Yaniv; Yang, Ming; Ranzato, Marc'Aurelio; Wolf, Lior (2014). "DeepFace: Closing the Gap to Human-Level Performance in Face Verification". 2014 IEEE Conference on Computer Vision and Pattern Recognition. pp. 1701–1708. doi:10.1109/CVPR.2014.220.
  52. ↑ Chandra, Tushar (2014). "Sibyl: A system for large scale machine learning at Google". DSN 2014.
  53. ↑ Silver, David; et al. (2016). "Mastering the game of Go with deep neural networks and tree search". Nature. 529 (7587): 484–489. Bibcode:2016Natur.529..484S. doi:10.1038/nature16961. PMID 26819042.
  54. ↑ Vaswani, Ashish; Shazeer, Noam; Parmar, Niki; Uszkoreit, Jakob; Jones, Llion; Gomez, Aidan N.; Kaiser, Łukasz; Polosukhin, Illia (2017). "Attention Is All You Need". Advances in Neural Information Processing Systems. Vol. 30. arXiv:1706.03762.
  55. ↑ Senior, Andrew W.; et al. (2020). "Improved protein structure prediction using potentials from deep learning". Nature. 577 (7792): 706–710. Bibcode:2020Natur.577..706S. doi:10.1038/s41586-019-1923-7. PMID 31942072.
  56. ↑ Jumper, John; et al. (2021). "Highly accurate protein structure prediction with AlphaFold". Nature. 596 (7873): 583–589. Bibcode:2021Natur.596..583J. doi:10.1038/s41586-021-03819-2. PMC 8371605. PMID 34265844.
  57. ↑ OpenAI (2022-11-30). "Introducing ChatGPT".
  58. ↑ Touvron, Hugo; et al. (2023). "LLaMA: Open and Efficient Foundation Language Models". arXiv:2302.13971 [cs.CL].
  59. ↑ OpenAI (2023). "GPT-4 Technical Report". arXiv:2303.08774 [cs.CL].
  60. ↑ Abramson, Josh; et al. (2024). "Accurate structure prediction of biomolecular interactions with AlphaFold 3". Nature. 630 (8016): 493–500. Bibcode:2024Natur.630..493A. doi:10.1038/s41586-024-07487-w. PMC 11168924. PMID 38718835.

Works cited