Evolutionary Game Theory
From static equilibria to stochastic dynamics in finite populations
A Brief History
The application of strategic thinking to biology has a rich intellectual lineage, drawing from economics, mathematics, and ecology. Where classical game theory asks what a rational agent should do, evolutionary game theory asks what natural selection will favour. The shift is profound: players need not be conscious strategists — they can be bacteria, viruses, or plants — and "rationality" is replaced by differential reproduction.
Games and Payoff Matrices
Frequency-dependent fitness introduces a strategic aspect to evolution. Evolutionary game theory is the study of biological systems with frequency-dependent fitness. Consider a population with two strategies, $A$ and $B$. Individuals interact pairwise, and the outcome of each interaction is encoded in a payoff matrix:
$$ \begin{pmatrix} a_1 & a_0 \\ b_1 & b_0 \end{pmatrix} $$The entries give payoffs to the row player. For instance, an $A$-player meeting another $A$-player receives payoff $a_1$, while meeting a $B$-player yields $a_0$.
The Hawk–Dove Game
A classic example introduced by Maynard Smith and Price (1973) models animal conflict over resources. Hawks escalate fights; Doves share peacefully. Let the resource value be $V$ and the cost of fighting be $C$, with $C > V$:
| Hawk | Dove | |
|---|---|---|
| Hawk | $\frac{V-C}{2}$ | $V$ |
| Dove | $0$ | $\frac{V}{2}$ |
Since $C > V$, mutual escalation is costly — Hawks meeting Hawks get a negative expected payoff. A population of all Hawks can be invaded by Doves (who avoid injury), and a population of all Doves can be invaded by Hawks (who take the whole resource). The result is a stable coexistence of both strategies — precisely the coexistence regime of the general $2 \times 2$ theory described below.
The Prisoner's Dilemma
The problem of cooperation is captured by another classic game. With strategies Cooperate ($C$) and Defect ($D$), benefits $b$ and costs $c$:
| C | D | |
|---|---|---|
| C | $b - c$ | $-c$ |
| D | $b$ | $0$ |
Here, $D$ always does better regardless of the opponent's strategy — defection is a dominant strategy. Yet mutual cooperation ($b-c$ each) beats mutual defection ($0$ each). This tension between individual and collective optimality lies at the heart of the evolution of cooperation.
Nash Equilibrium and ESS
In classical game theory, a Nash equilibrium is a strategy profile from which no player can improve their payoff by unilaterally switching strategies. In evolutionary game theory, we ask a related but distinct question: can a resident strategy resist invasion by a rare mutant?
Strategy $A$ is a Nash equilibrium if $a_1 \geq b_1$, i.e. no alternative strategy does better against $A$ than $A$ does against itself. It is a strict Nash equilibrium if $a_1 > b_1$.
Strategy $A$ is an ESS if either of the following holds:
(i) $a_1 > b_1$ (strict Nash), or
(ii) $a_1 = b_1$ and $a_0 > b_0$ (Bishop–Cannings condition).
The first condition says $A$ does better against itself than any invader does against $A$. The second is a tiebreaker: if $B$ does equally well against $A$, then $A$ must do better against $B$ than $B$ does against itself.
The relationship between these concepts forms a strict hierarchy: every strict Nash equilibrium is an ESS, every ESS is a Nash equilibrium, but the converses do not always hold. An ESS is a refinement of Nash — it adds the requirement of evolutionary robustness.
Replicator Dynamics
The ESS tells us whether a state is stable once reached, but how does a population get there? For this we need dynamics. The replicator equation, introduced by Taylor and Jonker (1978) and refined by Schuster and Sigmund (1983), is the fundamental dynamical equation of evolutionary game theory.
Let $x$ be the frequency of strategy $A$ in an infinitely large, well-mixed population. Strategy fitnesses are:
$$ \begin{aligned} f_A &= a_1 x + a_0(1 - x) \\ f_B &= b_1 x + b_0(1 - x) \end{aligned} $$The replicator equation for two strategies reduces to a single ODE:
$$\dot{x} = x(1-x)(f_A - f_B)$$
where $\dot{x} = dx/dt$. The term $x(1-x)$ ensures that extinct strategies stay extinct, and the factor $(f_A - f_B)$ drives the direction of selection: whichever strategy is fitter increases in frequency.
Setting $\dot{x} = 0$ gives three equilibria: $x=0$ (all $B$), $x=1$ (all $A$), and an interior equilibrium $x^* = \frac{b_0 - a_0}{a_1 - a_0 - b_1 + b_0}$ when the fitnesses cross. Depending on the payoff values, four qualitative outcomes are possible:
Dominance ($a_1 > b_1$ and $a_0 > b_0$): $A$ is always fitter; the
population converges to all-$A$.
Coexistence ($a_1 < b_1$ and $a_0 > b_0$): each strategy has an advantage
when rare, leading to a stable internal equilibrium.
Bi-stability ($a_1 > b_1$ and $a_0 < b_0$): the common strategy has the
advantage; the outcome depends on initial conditions.
Neutrality ($a_1 = b_1$ and $a_0 = b_0$): both strategies are equivalent.
Explore: Replicator Dynamics
Finite Populations: The Moran Process
The replicator equation assumes an infinitely large population. Real populations are finite, and randomness — genetic drift — becomes important. The Moran process is the standard model for evolutionary dynamics in a well-mixed finite population.
Consider a population of $N$ individuals, $i$ playing strategy $A$ and $N-i$ playing $B$. At each time step:
1. An individual is selected for reproduction proportional to fitness.
2. It produces one identical offspring.
3. A randomly chosen individual (possibly the parent) is removed to keep the population size constant.
The fitnesses of the two types are: $$f_A = 1 - w + w\pi_A, \quad f_B = 1 - w + w\pi_B$$ where $w \in [0,1]$ is the intensity of selection. When $w=0$, the process is neutral (pure drift); when $w=1$, fitness equals payoff.
In the Moran process, the transition probabilities from state $i$ (number of $A$ individuals) are:
$$ T_i^+ = \frac{i \, f_A}{i \, f_A + (N-i) f_B} \cdot \frac{N-i}{N}, \qquad T_i^- = \frac{(N-i) f_B}{i \, f_A + (N-i) f_B} \cdot \frac{i}{N} $$where $T_i^+$ is the probability that the number of $A$-players increases by one, and $T_i^-$ the probability it decreases by one.
Fixation Probability
A central quantity is the fixation probability $\phi_i$ — the probability that starting from $i$ copies of $A$, the entire population eventually becomes all-$A$. This is obtained by solving the recursion with absorbing boundaries at $i=0$ (extinction) and $i=N$ (fixation):
$$ \phi_i = \frac{1 + \displaystyle\sum_{k=1}^{i-1} \prod_{m=1}^{k} \frac{T_m^-}{T_m^+}}{1 + \displaystyle\sum_{k=1}^{N-1} \prod_{m=1}^{k} \frac{T_m^-}{T_m^+}} $$For a single mutant $A$ in a population of $B$ players ($i=1$), we get $\phi_1$, the probability of fixation of a single mutant. Under neutral drift ($w=0$), this simplifies to $\phi_1 = 1/N$ — each individual has an equal chance of taking over.
A mutant strategy is said to be favoured by selection if $\phi_1 > 1/N$, i.e. its fixation probability exceeds that of a neutral mutant. This provides a clean criterion for when natural selection promotes the spread of a new type.
Explore: Moran Process
Fixation Times
Beyond whether a mutant fixates, we often want to know how long it takes. The conditional fixation time $\tau_1^N$ is the expected number of generations for a single mutant $A$ to take over the population, given that it does fixate.
For a neutral process ($w=0$), the unconditional mean absorption time starting from state $i=1$ scales as $N^2$, reflecting the slowness of pure random drift. More precisely:
$$ \tau_1^N = \sum_{k=1}^{N-1}\sum_{l=1}^{k}\frac{\phi_l^N}{T_l^+}\prod_{m=l+1}^{k}\frac{T_m^-}{T_m^+} $$A remarkable result from Altrock, Gokhale and Traulsen (2010) is the phenomenon of stochastic slowdown: even when type $A$ has a fitness advantage (selection bias $\beta > 0$), the conditional mean fixation time can increase with $\beta$ in a certain parameter range. In other words, a fitter mutant can take longer to fix than a neutral one. This counterintuitive effect arises from the asymmetry between the birth and death transition rates in the Moran process.
Under weak selection ($N\beta \ll 1$), the conditional fixation time for a single advantageous mutant can exceed that of a neutral mutant. The fitness advantage increases the fixation probability but simultaneously reshapes the distribution of paths in a way that lengthens the expected time to absorption. The maximal relative increase in conditional fixation time (compared to neutral) is bounded by a constant that does not grow with population size $N$.
Multiple Strategies
The two-strategy framework generalises naturally. With $n$ strategies competing in pairwise interactions, the payoff matrix becomes $n \times n$:
$$ \mathbf{A} = \begin{pmatrix} a_{1,1} & a_{1,2} & \cdots & a_{1,n} \\ a_{2,1} & a_{2,2} & \cdots & a_{2,n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n,1} & a_{n,2} & \cdots & a_{n,n} \end{pmatrix} $$The fitness of strategy $i$ is $f_i(\mathbf{x}) = \sum_{j=1}^n a_{i,j} x_j$, and the replicator equation generalises to:
$$ \dot{x}_i = x_i \left[ f_i(\mathbf{x}) - \bar{f} \right], \quad i = 1, 2, \ldots, n $$where $\bar{f} = \sum_{i=1}^n x_i f_i(\mathbf{x})$ is the average population fitness. The dynamics now unfold on an $(n-1)$-dimensional simplex, the space of all valid frequency vectors. For three strategies this is a triangle; for four, a tetrahedron.
A biological example of the richness of multi-strategy games comes from strains of Escherichia coli competing via colicin production. Three strains — a killer ($K$), a resistant strain ($R$), and a sensitive strain ($S$) — form a rock-paper-scissors cycle: $K$ beats $S$, $S$ beats $R$ (by growing faster), and $R$ beats $K$ (by resisting the toxin without the cost of producing it). Such intransitive competition can maintain biodiversity indefinitely in the deterministic limit.
Exercises
Conceptual Questions
- Explain the concept of an evolutionarily stable strategy (ESS) and why it is different from a Nash equilibrium in classical game theory. Can a strategy be stable if it is not a best response to itself?
- In the Hawk–Dove game, derive the ESS frequency of hawks and explain why a mixed strategy (both types present) can be stable even though playing pure Hawk or pure Dove is not.
- What is the replicator equation and how does it link payoffs in a game to fitness and allele frequency change? Why is it called "replicator" dynamics?
- How does finite population size affect evolutionary stability? Why might a strategy that is ESS in infinite populations become unstable in small, finite populations?
- Describe an example of rock-paper-scissors (intransitive) dynamics in biology and explain why such cycles prevent competitive exclusion. How would spatial structure affect cycling?
Computer Problems
- Hawk-Dove Game and ESS. Implement the Hawk-Dove payoff matrix with $V = 4$ (resource value), $C = 6$ (cost of conflict). Compute the payoff to Hawk and Dove as a function of frequency $p$ (fraction Hawks). Use the replicator equation $\dot{p} = p(1-p)(w_H - w_D)$ and integrate to show convergence to the ESS frequency $p^* = V/C = 2/3$.
- Stable and Unstable Equilibria in 2x2 Games. For an arbitrary $2 \times 2$ game with payoff matrix, implement the replicator equation and classify equilibria (fixed points). Use eigenvalue analysis to determine stability, and show how bifurcations occur as payoffs change. Plot phase portrait for several parameter regimes.
- Finite Population Effects on ESS. Implement the Moran process where individuals reproduce proportional to fitness and one dies randomly. Compare evolutionary outcomes in finite populations ($N = 100, 1000$) to the infinite-population ESS prediction. Show stochastic drift away from ESS and compute fixation probabilities.
- Rock-Paper-Scissors Cycles. Implement a three-strategy game where Rock beats Scissors, Scissors beats Paper, Paper beats Rock. Use the replicator equation with payoff $\pi_{ij}$ (payoff when playing strategy $i$ against $j$). Show that the interior fixed point is a centre (neutral cycles) and plot the limit cycles in frequency space.
- Coevolutionary Dynamics in Multi-Strategy Games. Implement a replicator-like dynamic where strategies are determined by two genes (each with two alleles), allowing four phenotypes with possibly different payoffs. Show how linkage and genetic architecture affect the path to ESS and whether polymorphisms are maintained.
References
- Maynard Smith, J. & Price, G. R. (1973). The Logic of Animal Conflict. Nature, 246, 15–18.
- Nash, J. (1950). Equilibrium Points in N-Person Games. Proc. Natl. Acad. Sci. USA, 36, 48–49.
- Taylor, P. D. & Jonker, L. B. (1978). Evolutionary stable strategies and game dynamics. Math. Biosci., 40, 145–156.
- Hofbauer, J. & Sigmund, K. (1998). Evolutionary Games and Population Dynamics. Cambridge University Press.
- Nowak, M. A. & Sigmund, K. (2004). Evolutionary dynamics of biological games. Science, 303, 793–799.
- Nowak, M. A. (2006). Evolutionary Dynamics: Exploring the Equations of Life. Harvard University Press.
- Nowak, M. A., Sasaki, A., Taylor, C. & Fudenberg, D. (2004). Emergence of cooperation and evolutionary stability in finite populations. Nature, 428, 646–650.
- Traulsen, A., Claussen, J. C. & Hauert, C. (2005). Coevolutionary dynamics: from finite to infinite populations. Phys. Rev. Lett., 95, 238701.
- Altrock, P. M., Gokhale, C. S. & Traulsen, A. (2010). Stochastic slowdown in evolutionary processes. Phys. Rev. E, 82, 011925.
- Gokhale, C. S. & Traulsen, A. (2010). Evolutionary games in the multiverse. Proc. Natl. Acad. Sci. USA, 107, 5500–5504.
- Gokhale, C. S. (2011). Evolutionary dynamics on multi-dimensional fitness landscapes. PhD thesis, Kiel University.
- Schuster, P. & Sigmund, K. (1983). Replicator dynamics. J. Theor. Biol., 100, 533–538.
- von Neumann, J. & Morgenstern, O. (1944). Theory of Games and Economic Behavior. Princeton University Press.
- Fisher, R. A. (1930). The Genetical Theory of Natural Selection. Clarendon Press.
- Zeeman, E. C. (1980). Population dynamics from game theory. Global Theory of Dynamical Systems, Springer.