Skip to main content
The law of large numbers says that the larger the sample drawn from a population, the closer the sample mean comes to the mean μ\mu of the whole population (OpenStax).

What happens to the proportion of heads when you toss a coin many times?

Toss a fair coin: heads and tails each have probability 0.5. Score heads as 1 and tails as 0, and the expected value of one toss is 1×0.5+0×0.5=0.51 \times 0.5 + 0 \times 0.5 = 0.5, while the average of nn tosses is the proportion of heads. The chart below simulates 100 runs of 100,000 tosses each and draws each run’s proportion of heads after every toss, with the number of tosses on a logarithmic axis. The shaded band around 0.5 is 0.01 wide on each side.
Over the first tosses the lines fan out, because the proportion jumps with every head or tail. The more tosses, the more the lines pull in toward 0.5 and settle inside the band. The probability that a run lies inside the band 0.5 ± 0.01 is 7.96% after 100 tosses, 45.20% after 1,000, 95.34% after 10,000 and nearly 100% after 100,000, computed exactly from the binomial distribution. In the chart, 8, 45, 95 and then 100 of the 100 runs lie inside the band at those four points.

What does the law of large numbers say, exactly?

Stated rigorously, it comes in two forms (Pishro-Nik) under one condition: X1,X2,…,XnX_1, X_2, \dots, X_n are independent, identically distributed (i.i.d.) random variables with a finite expected value EXi=μ<∞E X_i = \mu < \infty. For the coin, XiX_i is the result of toss ii (1 or 0) and μ=0.5\mu = 0.5. Let MnM_n be the mean of the first nn variables: Mn=(X1+X2+⋯+Xn)/nM_n = (X_1 + X_2 + \dots + X_n) / n Weak law. For every ϵ>0\epsilon > 0: lim⁡n→∞P(∣Mn−μ∣≥ϵ)=0\lim_{n \to \infty} P(|M_n - \mu| \geq \epsilon) = 0 ϵ\epsilon is a miss of a given size. The probability that the mean misses μ\mu by ϵ\epsilon or more goes to 0 as nn grows. That is what you see looking down a vertical line at one number of tosses: the share of runs outside the band μ±ϵ\mu \pm \epsilon shrinks to 0. With ϵ=0.01\epsilon = 0.01, the probability of missing by ϵ\epsilon or more is 92.04% after 100 tosses, 54.80% after 1,000 and 4.66% after 10,000. Strong law (section 7.2.7). MnM_n converges to μ\mu almost surely: P(lim⁡n→∞Mn=μ)=1P\left(\lim_{n \to \infty} M_n = \mu\right) = 1 The probability that the whole sequence of means M1,M2,M3,…M_1, M_2, M_3, \dots of one run goes to μ\mu, from start to end, is 1. That is what you see looking along a single line: with probability 1, a run eventually enters any narrow band around 0.5 and never leaves it. The weak law is about the probability at each nn; the strong law is about the whole path of one run. According to Pishro-Nik, the two forms differ mainly in theory.

References

Sources: OpenStax, Introductory Statistics 2e, section 7.3 and Pishro-Nik, sections 7.1.1 and 7.2.7.

Descriptive statistics

The mean, median, standard deviation and quantiles of a data set.

Expected value

The average profit or loss per bet, and why a positive expected value still leaves some people in the red.
Last modified on October 16, 2026