Psychology › Learning & behavior › Reinforcement and punishment
Variable reward schedules
1957 (Ferster & Skinner)· Finding or experiment
Well replicatedStrong, repeated evidence.
Rewards that arrive unpredictably, after a varying number of actions, produce the most persistent behavior.
See it on the Psychology map →What it says
In Schedules of Reinforcement (1957), Charles Ferster and B. F. Skinner reported years of pigeon data on reward timing. Fixed schedules, such as a reward every tenth peck or every minute, produce pauses after each reward. Variable-ratio schedules, a reward after an unpredictable number of responses, produce fast, steady responding that is very hard to extinguish, because a dry spell looks like normal bad luck. Slot machines pay this way; so, loosely, do social media feeds, loot boxes and email.
Why it matters
It explains why slot machines and app notifications are so compelling and why intermittent rewards build hard-to-break habits. Investors refreshing prices for the occasional win feel a similar pull.
Think of it like
Like prospecting for deals: most calls go nowhere, but the occasional big one keeps you dialing.
The evidence
Status: well replicated. Ferster and Skinner's book holds hundreds of response records; the patterns repeat across many species, and gambling research confirms the pull of random wins.
Open questions
How directly pigeon schedules explain people: human gambling also involves near-misses, illusions of control and losses disguised as wins, and app 'addiction' remains contested.
Connections (3)
▸Variable reward schedules shape the power of Reinforcement and punishmentWell replicated
The connection
The same reward can build weak or very stubborn behavior depending on its schedule. Rewarding every response teaches fastest but fades fastest once the reward stops. Intermittent, unpredictable rewards make behavior far more persistent, the 'partial reinforcement extinction effect'. Timing, not just size, sets the strength of reinforcement.
Who showed it and when
Skinner stumbled on intermittent reward at Harvard in the early 1930s, by his account while stretching a short supply of food pellets; he and Charles Ferster catalogued schedules in 1957.
The evidence
Thousands of pigeon and rat experiments, plus human studies; Lloyd Humphreys demonstrated the extinction effect in people in 1939.
Open questions
Why unpredictability makes behavior so persistent: theories cite frustration, difficulty noticing that rewards have stopped, and dopamine responses to uncertainty, without full agreement.
▸Variable reward schedules fuel Gambling disorderWell replicated
The connection
Slot machines pay on a variable-ratio schedule: wins come after an unpredictable number of bets. Skinner showed this produces the fastest, most persistent responding, because any losing streak might end on the next try. Near-misses and small 'wins' that return less than the bet add to the pull.
Who showed it and when
Ferster and Skinner's pigeon studies (1957); applied to modern gambling by researchers such as anthropologist Natasha Dow Schüll, who studied Las Vegas slot machines (Addiction by Design, 2012).
The evidence
Lab studies show variable rewards resist extinction, and studies of gamblers show near-misses prolong play and activate reward circuits.
Open questions
Variable rewards pull on almost everyone; why only about 1 percent develop a disorder points to genes, stress and other vulnerabilities.
▸Variable reward schedules are built into the apps behind Social media and teen mental healthMixed evidence
The connection
Skinner found that rewards delivered unpredictably, a variable schedule, produce the most persistent behavior; slot machines work this way. Feeds, likes and notifications arrive the same way: refresh, and sometimes something good appears. Critics, including former tech insiders, say apps were designed like this to maximize time spent, which may hit teens hardest since their reward systems are especially sensitive.
Who showed it and when
Ferster and Skinner's 1957 pigeon work at Harvard; the ideas spread through Silicon Valley via B. J. Fogg's Stanford lab and Nir Eyal's 2014 book Hooked; insiders spoke out from 2017.
How solid is it?
Insider testimony, such as Facebook's first president Sean Parker in 2017, describes deliberate use of intermittent rewards; evidence that this harms teens is mostly correlational.
Open questions
Whether 'addictive design' produces true addiction or just heavy habitual use; 'social media addiction' is not an official diagnosis in the DSM-5.