Technology & Medicine › Internet, genomes & AI
Deep learning (modern AI)
2012 (AlexNet); roots 1986 onward· Invention or discovery· Communication & computing
Well documentedSolid records, physical evidence or scientific consensus.
Computer systems that learn skills from huge numbers of examples using many-layered artificial 'neural networks'.
See it on the Technology & Medicine map →How it works
Deep learning trains neural networks, stacked layers of simple math units, by showing them millions of examples and nudging internal numbers to reduce errors, a method called backpropagation, popularized in 1986. The idea long underperformed for lack of data and computing power. In 2012 AlexNet, built by Geoffrey Hinton's Toronto group and trained on two Nvidia graphics chips, won an image-recognition contest with a 15% error rate versus 26% for the runner-up.
Why it mattered
It made computers good at seeing, hearing and translating, and became the basis of nearly all modern AI, from face unlock and voice assistants to chatbots.
Think of it like
Like training a new leasing agent by showing thousands of past deals rather than writing a rulebook: judgment emerges from examples.
How we know
The 1986 Nature backpropagation paper; AlexNet's 2012 results on the public ImageNet benchmark; the 2018 Turing Award to Hinton, LeCun and Bengio; Hinton's 2024 Nobel Prize in Physics.
Still unknown or debated
Why huge networks generalize so well, and what exactly they learn inside, is still poorly understood; credit for early ideas is disputed, notably by Jürgen Schmidhuber.
Inside this (3)
Connections (7)
▸Microprocessor (Intel 4004) evolved into the GPU chips driving Deep learning (modern AI)Well documented
The connection
Microprocessors put a computer's logic on one chip and grew millions of times more complex under Moore's law. Graphics chips (GPUs), designed for video games, pack thousands of simple processors doing math in parallel, exactly what neural networks need. Around 2009–12 researchers found GPUs sped training dozens of times; Nvidia's chips now power most AI.
When and how
Intel's 4004, 1971; Nvidia's GeForce 256, marketed as the first GPU, 1999, and its CUDA programming toolkit, 2006; AlexNet trained on two Nvidia gaming cards, 2012.
How we know
Papers such as Stanford's 2009 GPU study and AlexNet (2012) report the speedups; Nvidia's data-center revenue shows the scale today.
Debates and open questions
Whether specialized AI chips, like Google's TPUs, will overtake GPUs, and whether chip supply and electricity can keep pace with AI's appetite, is debated.
▸World Wide Web supplied the training data for Deep learning (modern AI)Mostly agreed
The connection
Deep learning needs enormous numbers of labeled examples, and the web supplied them. ImageNet gathered about 14 million web photos, labeled by online workers through Amazon Mechanical Turk; AlexNet's 2012 breakthrough was trained on it. Later language models learned from billions of web pages collected by crawls such as Common Crawl.
When and how
ImageNet built at Princeton and Stanford under Fei-Fei Li, 2007–09; Common Crawl, a nonprofit web archive, from 2008; used by AI labs worldwide.
How sure are we?
The ImageNet, AlexNet and GPT-3 papers state their data sources; GPT-3's paper lists filtered Common Crawl as the bulk of its training text.
Debates and open questions
Whether training AI on copyrighted web content is legal fair use is being fought in court, including suits by authors and The New York Times from 2023.
▸Deep learning (modern AI) powered AlphaGo beats Lee SedolWell documented
The connection
AlphaGo used two deep neural networks: a 'policy' network suggesting promising moves and a 'value' network judging who was winning. Trained first on about 30 million positions from expert games, then by playing itself, they guided a search through future moves, replacing the hand-written rules that had kept Go programs at amateur level.
When and how
DeepMind in London, 2014–16, led by Demis Hassabis and David Silver; matches against European champion Fan Hui (October 2015) and Lee Sedol in Seoul (March 2016).
How we know
DeepMind's January 2016 Nature paper details the networks and training; the match records are public.
Debates and open questions
Its successor AlphaGo Zero (2017) learned without any human games and played stronger, raising questions about how much human knowledge actually helped.
▸Deep learning (modern AI) led to Large language modelsWell documented
The connection
Large language models are deep learning applied to text at enormous scale: the same layered neural networks, adjusted by backpropagation, run on clusters of graphics chips. Researchers found that bigger models trained on more data improved predictably ('scaling laws', 2020), which drove the jump to models with hundreds of billions of internal settings.
When and how
Word-meaning models such as Google's word2vec (2013); OpenAI's GPT (2018) and GPT-3 (2020); scaling-law studies at OpenAI (2020) and DeepMind (2022).
How we know
Model papers describe the deep learning methods used; scaling-law papers chart performance rising with model size and data.
Debates and open questions
Whether scaling alone can keep improving models, or new ideas are needed, is debated as high-quality text data runs short.
▸Deep learning (modern AI) led to AlphaFold protein structure predictionWell documented
The connection
AlphaFold 2 is a deep neural network that takes a protein's amino-acid sequence, plus related sequences from other species, and outputs the 3D position of every atom. It used attention, the mechanism behind transformers, to relate distant parts of the chain, and learned from known structures much as image models learn from labeled photos.
When and how
DeepMind in London; AlphaFold 1 won the CASP13 contest in 2018, and AlphaFold 2, led by John Jumper, dominated CASP14 in November 2020.
How we know
DeepMind's 2021 Nature paper and open code document the design; independent teams reproduced it (OpenFold) or built similar systems (RoseTTAFold).
Debates and open questions
Whether AlphaFold learned the real physics of folding or clever pattern matching is debated; it reveals little about how proteins fold step by step.
▸Deep learning (modern AI) was reshaped in 2017 by the Transformer architectureWell documented
The connection
By 2016, deep learning for language relied on recurrent networks that read text word by word, slowly and forgetfully over long passages. The 2017 transformer kept deep learning's core methods but replaced that step-by-step reading with attention across a whole passage at once, training far faster on graphics chips. It soon spread to images, speech and biology.
When and how
Google Brain and Google Research in Mountain View, California, 2017; the paper's eight authors included Ashish Vaswani, Noam Shazeer and Jakob Uszkoreit.
How we know
The 2017 paper showed better translation at a fraction of the training cost; within a few years transformers topped most AI benchmarks.
Debates and open questions
Whether transformers are the final architecture or will be overtaken by more efficient designs, such as state-space models, is open.
▸Deep learning (modern AI) has found new antibiotic candidates against Antibiotic resistanceMostly agreed
The connection
Deep-learning models trained on how known molecules affect bacteria can screen millions of untested compounds in days, work that would take labs years. In 2020 an MIT team found halicin this way, which killed many resistant bacteria in lab tests; later work found abaucin (2023) and a new class of compounds against MRSA.
When and how
MIT and the Broad Institute, Cambridge, Massachusetts (James Collins, Regina Barzilay), 2020–25; McMaster University, Canada (abaucin, 2023).
How sure are we?
Peer-reviewed papers in Cell and Nature report lab and mouse results; none of these compounds has yet completed human trials.
Debates and open questions
Whether AI-found compounds will survive costly human trials, and whether drug companies will pay to develop antibiotics that earn little, are open questions.