Book summary “The evolution of cooperation”: summary

Introduction

You often face situations where pursuing your own immediate interests harms everyone involved. When individuals act as self-seeking egoists, mutual trust breaks down and leaves you worse off than if you had cooperated.

Cooperation can emerge and endure among egoists without central authority through the power of reciprocity and the shadow of the future.

Robert M. Axelrod explores this phenomenon using the iterated Prisoner's Dilemma, computer tournaments, historical instances like trench warfare, and biological systems. You will learn how simple behavioral rules outcompete complex strategies and how you can foster cooperation in daily life.

• How can cooperation emerge in a world of unconditional defection?
• What strategy wins repeated encounters against diverse opponents?
• How do biological systems sustain cooperation without foresight?
• What practical steps can you take to promote trust and reciprocity?

1. The Problem of Cooperation

Cooperation often fails to emerge because individual self-interest leads both parties to defect, resulting in an outcome worse than mutual cooperation.

When you pursue your immediate self-interest without a central authority, you face a structure where defection yields a higher payoff regardless of what the other player does. This creates a dilemma where rational choices lead to mutual punishment instead of mutual reward.

Two industrial nations erect trade barriers against each other's exports; both would benefit from free trade, but each has an incentive to retain barriers, leading to a worse outcome for both.

The Prisoner's Dilemma Framework

The Prisoner's Dilemma is mathematically defined by a specific ranking of payoffs and the inability of players to escape the trap by taking turns exploiting each other. The payoffs are ranked from best to worst as temptation to defect, reward for mutual cooperation, punishment for mutual defection, and sucker's payoff.

Furthermore, the reward for mutual cooperation must be greater than the average of the temptation and the sucker's payoff to prevent alternating exploitation. The standard values consist of:

• 5 points for the temptation to defect;
• 3 points for the reward of mutual cooperation;
• 1 point for the punishment for mutual defection;
• 0 points for the sucker's payoff.

The Shadow of the Future

The possibility of future interaction casts a shadow back upon the present, making cooperation viable when players care about subsequent payoffs. Because future payoffs are valued less than current ones due to time discounting and the risk that interactions may end, the discount parameter measures the weight of the next move relative to the current move.

You can represent this weight as $w = 1/2$, representing each move being half as important as the previous move. The sum of an infinite series of cumulative payoffs is expressed as $1/(1 - w)$. A 90 percent value of the previous move is represented as $w = .9$, yielding a cumulative value of ten points for a string of ones.

If the future is important enough, there is no single best strategy that works independently of what the other player is doing. The optimal choice depends entirely on the strategy employed by the opponent. For instance, always defecting is best against an uncooperative opponent, whereas cooperating is better against a strategy that rewards cooperation.

2. The Success of TIT FOR TAT

In computer tournaments featuring complex programs submitted by experts, the simplest strategy, TIT FOR TAT, won consecutive victories.

To study effective choice in the iterated Prisoner’s Dilemma, a computer tournament was organized where entries submitted decision rules as computer programs to play against each other. The tournament was structured as a round robin where each program was paired with every other entry, its own twin, and a random program called RANDOM. Each game consisted of exactly 200 moves with a payoff matrix of 3 points for mutual cooperation, 1 point for mutual defection, 5 points for defecting while the other cooperated, and 0 points for cooperating while the other defected.

In the first round, 14 entries from psychology, economics, political science, mathematics, and sociology were submitted and run five times for a total of 120,000 moves and 240,000 choices.

Tournament Design and Results

Anatol Rapoport submitted TIT FOR TAT, which starts with cooperation and thereafter simply mimics the other player's previous move. It won both the first round of fourteen entries and the second round of sixty-two entries.

Professor Anatol Rapoport of the University of Toronto submitted TIT FOR TAT, which averaged 504 points per game and won first place. None of the more complex programs submitted by game theorists and experts were able to outperform this simplest possible entry.

A single property distinguished high-scoring entries from low-scoring entries in the tournament: the property of being nice, meaning never being the first to defect. All eight top-ranking entries were nice, whereas none of the lower-ranking entries were. Nice rules received tournament averages between 472 and 504 points, while the best non-nice entry received only 401 points. Nice rules achieved high scores by cooperating with each other until virtually the end of the game.

In the second round with 62 entries from 6 countries, TIT FOR TAT won again despite all contestants knowing its previous success and the analytical lessons of round one. The second round eliminated endgame effects by ending games probabilistically. Contestants ranged from a ten-year-old computer hobbyist to professors across multiple disciplines. Of the top 15 rules, all but one were nice, showing a strong correlation between niceness and success.

John Maynard Smith submitted TIT FOR TWO TATS, Stanley Quayle and Leslie Downing submitted REVISED DOWNING, and Craig Feathers submitted TRANQUILIZER, but none could surpass pure TIT FOR TAT.

Four Pillars of Success

Analyzing choice requires looking three levels deep: direct effects, indirect effects of punishment, and tertiary effects where retaliation repeats and amplifies one's own exploitative choice. Rules like FRIEDMAN used permanent retaliation and scored lowest among nice rules, whereas TIT FOR TAT forgave after a single punishment.

When JOSS defected 10 percent of the time after cooperation against TIT FOR TAT, a single defection created an echoing string of alternating defections on every subsequent move, reducing both players to 1 point per turn.

By setting the number of offspring proportional to a strategy's tournament score, the simulation models survival of the fittest through learning, imitation, and selection. Exploitative non-nice rules like HARRINGTON initially grow by preying on easygoing rules, but eventually become extinct as their support base erodes.

By the 1,000th generation of the simulated ecological tournament, bottom-tier rules disappeared and TIT FOR TAT was the most successful rule, growing at the fastest rate.

The robust success of TIT FOR TAT across diverse environments stems from a combination of four distinct behavioral properties: niceness, retaliation, forgiveness, and clarity.

Niceness prevents it from getting into unnecessary trouble. Retaliation discourages the other side from persisting in defection. Forgiveness helps restore mutual cooperation after punishment. Clarity makes it intelligible to the other player, eliciting long-term cooperation because its nonexploitability is easy to recognize and appreciate.

3. The Chronology of Cooperation

The evolution of cooperation proceeds in three distinct stages from initial foothold to complete stability.

Cooperation starts from small clusters of reciprocating individuals interacting with each other. It thrives in a variegated environment of diverse strategies. Once fully established, it finally resists invasion by less cooperative strategies.

John Maynard Smith's evolutionary approach considers a population using a single native strategy and a single mutant newcomer. If the newcomer receives a higher expected payoff interacting with natives than natives receive among themselves, the mutant strategy invades. Only collectively stable strategies can sustain themselves in the long run against any possible mutation.

TIT FOR TAT is collectively stable if the future interactions are important enough relative to the immediate payoff parameters. Because TIT FOR TAT has a single-move memory, the most effective challengers are alternating defect-cooperate sequences or unconditional defection. If the discount parameter representing the probability of future interaction is sufficiently high, these challengers cannot outperform TIT FOR TAT, rendering it collectively stable.

If payoffs are T=5, R=3, P=1, and S=0, TIT FOR TAT is collectively stable when the next move is at least two-thirds as important as the current move. A population of unconditional defectors can be easily invaded by a small cluster of mutually cooperating strategies. Unconditional defection is always collectively stable against single individual invaders. However, if cooperative newcomers enter in a small cluster where they interact with each other often enough, their high mutual scores offset the initial exploitation by meanies, allowing cooperation to take root.

A firm switching to TIT FOR TAT needs only 5 percent of its interactions to be with another such firm to outperform a population of meanies. For a cooperative nice strategy to be collectively stable, it must be provocable by the very first defection of an opponent. Nice rules never initiate defection, which maximizes mutual cooperation rewards. To prevent exploitation by single-defector strategies, a nice rule must possess a finite chance of responding with a defection of its own upon the opponent's first transgression.

Trench Warfare Application

Small military units in stationary sectors during World War I faced an iterated Prisoner's Dilemma that favored mutual restraint. While high commands sought attrition and treated the war as a zero-sum game, front-line battalions facing each other for extended periods preferred mutual restraint over mutual punishment. This transformed a one-move kill-or-be-killed game into an iterated game supporting reciprocal cooperation across a 500-mile line in France and Belgium.

British and German soldiers separated by 100 to 400 yards of no-man's-land in quiet sectors exercised restraint regarding ration wagons and developed tacit truces. A typical battalion consisting of 1,000 men maintained the live-and-let-live system through retaliatory demonstrations, rotation socialization, and artillery protection. Troops maintained cooperation by showing they could retaliate effectively if provoked, often using two-for-one or three-for-one responses.

Every eight days a battalion changed places, and outgoing units socialized incoming replacements through briefings. Infantrymen cultivated peaceful relations with artillery observers to prevent unwanted escalation. A German sniper demonstrated prowess by cutting a hole in a cottage wall to show he could hit targets if necessary.

The live-and-let-live system was ultimately destroyed by the high command's introduction of monitored raids. Army headquarters wanted to foster an offensive spirit and could not easily monitor quiet restraint, but they could monitor active raids involving 10 to 200 men. The British High Command initiated raids to show their French allies they were harassing the enemy and to boost troop morale.

Sustained cooperative exchanges in trench warfare spontaneously generated moral ethics and stylized combat rituals. Extended mutual restraint transformed player preferences, causing soldiers to feel moral regret for accidental violations and a strong sense of revenge when trust was broken. Simultaneously, routine perfunctory firing acts served as physical rituals that symbolized fellow-feeling for the enemy while feigning aggression for commanding officers. A 7 o'clock evening gun fired so regularly by the British allowed soldiers to set watches by it, and a Saxon soldier shouted an apology across the parapet after a salvo landed, blaming the Prussian artillery rather than his own unit.

4. The Evolution of Biological Systems

Cooperation can evolve in biological systems without foresight or central authority through genetics and the survival of the fittest.

Science 211 published the foundational biology paper on March 27, 1981, pages 1390-96. William D. Hamilton and Robert Axelrod demonstrated that cooperation based on reciprocity accounts for behavioral patterns across biological systems ranging from bacteria to birds. Individuals eliciting beneficial responses leave more surviving offspring.

Mechanisms Without Foresight

Cooperation can evolve in biological systems without any foresight or conscious intent on the part of the organisms. By applying evolutionary game theory, researchers show that strategies like reciprocity can emerge through natural selection and survival of the fittest, even in organisms without brains like bacteria or viruses.

Cooperation can start in a completely noncooperative world through genetic kinship or spatial clustering. When a population is dominated by unconditional defection, genetic kinship allows altruism toward related individuals, while clustering allows a small group of cooperative mutant strategies to provide enough mutual interactions to survive and grow.

Territoriality and Recognition

Cooperation occurs among organisms such as bacteria, birds, and animals in territoriality, mating, and disease without requiring conscious thought or rational planning. Lower organisms rely on continuous contact or fixed meeting places to sustain cooperation when they lack individual recognition skills.

Higher organisms use complex individual recognition, such as facial recognition enabled by specific brain regions, to retaliate against defectors. Lower organisms substitute this by maintaining permanent physical association or using fixed territories.

Male territorial birds use songs to recognize neighbors and show aggressive reactions to unfamiliar male songs.

5. How to Choose Effectively

Individual participants in a Prisoner's Dilemma can optimize their outcomes by following four simple behavioral rules.

In a durable iterated Prisoner's Dilemma, four simple behavioral suggestions maximize your long-term score. You should apply these principles whenever you interact with other egoists:

• Don't be envious.
• Don't be the first to defect.
• Reciprocate both cooperation and defection.
• Don't be too clever.

Strategic Advice for Participants

Comparing your score to the other player's score breeds self-destructive envy in non-zero-sum interactions. Because most real-world situations are not zero-sum, trying to surpass the other player's score inevitably leads to unnecessary defection, retaliation, and mutual punishment. Students playing the Prisoner’s Dilemma in class for money inevitably compared scores, leading to retaliatory defections and worse overall outcomes.

Never being the first to defect is the single best predictor of a strategy's success in repeated interactions. In the computer tournaments, all top-ranking rules in the first round and almost all in the second round were nice, refusing to defect first.

Avoiding Envy and Complexity

A successful strategy must promptly reciprocate both cooperation and defection to avoid exploitation and escalation. Extracting more than one defection for each defection risks unending escalation, while excessive forgiveness invites exploitation by aggressive rules. TIT FOR TWO TATS forgives isolated defections by returning one-for-two, which caused it to fail in the second round against easily exploitative rules.

Overly sophisticated or complex strategies fail because they ignore how the other player reacts and adapts to their behavior. Unlike weather or a zero-sum chess opponent, an iterated Prisoner's Dilemma opponent responds to your actions. Complex maximizing rules make wrong inferences, and incomprehensible probabilistic strategies appear random and unresponsive.

6. How to Promote Cooperation

You can promote cooperation by enlarging the shadow of the future, altering payoff structures, teaching reciprocity, and improving recognition abilities.

Enlarging the Shadow of the Future

Cooperation can be promoted by enlarging the shadow of the future, making the next move more important relative to the current one. This can be achieved in several specific ways:

• Make the interactions more durable
• Make the interactions more frequent
• Break down issues or treaties into small pieces and stages

Henry Kissinger arranged for the Israeli disengagement from the Sinai after the 1973 war in stages coordinated with Egyptian moves.

Teaching Reciprocity and Verification

Teaching strategies based on reciprocity is superior to unconditional cooperation because it helps police the community and prevents exploitation. Feuds between families in Albania and the Middle East go on for decades as one injury is repaid by another. Slightly forgiving strategies, such as returning nine-tenths of a tit for a tat, prevent endless feuds.

Governments and central authorities often change payoffs by passing laws and enforcing penalties for cheating, tax evasion, or contract breaking. Governments enforce tax compliance by threatening evaders with jail time, making defection less attractive.

Improving the ability to recognize past players and verify their prior actions expands the scope of sustainable cooperation. Agreements to ban nuclear test explosions were prevented by the technical difficulty of distinguishing explosions from earthquakes.

7. The Social Structure of Cooperation

Social structures including observable labels, reputations, government regulation, and territorial systems shape the diffusion of cooperative strategies.

Labels allow players to make inferences about strangers before interacting. A town with eighty Greens and twenty Blues demonstrates how groups expect mean behavior and create self-confirming stereotypes. Minority groups interact with the majority more frequently and suffer lower average scores. Observable traits also establish status hierarchies where individuals act like bullies toward those below them and meekly toward those above them.

A player's reputation consists of others' beliefs about their strategy, built by observing past interactions. Britain's decision to take back the Falkland Islands enhanced its reputation for being provocable in response to an invasion. John McNaughton's memo on U.S. aims in Vietnam outlined specific percentages for strategic goals:

• 70 percent to avoid a humiliating U.S. defeat to our reputation as a guarantor;
• 20 percent to keep SVN and adjacent territory from Chinese hands;
• 10 percent to permit the people of SVN to enjoy a better, freer way of life.

Labels and Status Hierarchies

Governments operate in an iterated Prisoner's Dilemma with citizens or corporations regarding standards. Environmental regulatory agencies interact with regulated companies regarding industrial pollution standards and compliance. Regulators adopt an enforcement mode utilizing flexible enforcement when companies comply voluntarily and coercive enforcement when they evade.

Ordinary business transactions rely on anticipation of mutually rewarding future relations rather than central authority. A large packaging manufacturer found two-thirds of orders lacked legally binding contracts in its records.

• Get the other party on the telephone to deal with problems when they arise.
• Avoid reading legalistic contract clauses at each other if you intend to do business again.

Territorial Diffusion and Regulation

In territorial systems, players interact with their four neighbors (one north, one east, one south, one west) and imitate successful strategies. Simulations using sixty-three different decision rules from the second round of the computer tournament were assigned to territories of 14 cells high and 18 cells wide. The evolution took eleven to twenty-four generations for the simulation process to stop evolving.

Rapid turnover of leadership creates short-term incentives to defect, requiring accountability. Executives are often transferred from one position to another approximately every two years.

• Keep track of an executive's success in a position as well as the state in which the position was left to the next occupant.
• Hold political parties accountable for the acts of their elected members across election cycles.

Preventing undesirable collusion involves manipulating structural conditions. The U.S. government selecting two companies with shared specialties for competitive development contracts makes tacit collusion too easy to achieve.

• Reduce company specialization or compensate for its effects to decrease the frequency of interactions between specific pairs of firms.
• Detect and respond to potential violations of arms control agreements as early as possible.

10 Key Ideas

1. Cooperation fails when individual self-interest leads both parties to defect, resulting in an outcome worse than mutual cooperation.
2. The simplest computer tournament strategy, TIT FOR TAT, won consecutive victories by starting with cooperation and simply mimicking the opponent's previous move.
3. The robust success of TIT FOR TAT stems from a combination of four distinct behavioral properties: niceness, retaliation, forgiveness, and clarity.
4. TIT FOR TAT is collectively stable against any possible mutation when future interactions are important enough relative to immediate payoffs.
5. Cooperative newcomers can take root and invade an uncooperative population if they enter in a small cluster where they interact with each other often enough.
6. Small military units in stationary sectors during World War I transformed a one-move kill-or-be-killed game into an iterated game supporting reciprocal cooperation across a 500-mile line.
7. Cooperation can evolve in biological systems without foresight or conscious intent through genetics, natural selection, and the survival of the fittest.
8. Participants in an iterated Prisoner's Dilemma can optimize their outcomes by following four simple rules: don't be envious, don't be the first to defect, reciprocate both cooperation and defection, and don't be too clever.
9. You can promote cooperation by enlarging the shadow of the future, making interactions more durable or frequent and breaking down issues into small pieces.
10. Teaching strategies based on reciprocity helps police the community, prevents exploitation, and avoids endless feuds when paired with slight forgiveness.