What happens to the proportion of heads when you toss a coin many times?
Toss a fair coin: heads and tails each have probability 0.5. Score heads as 1 and tails as 0, and the expected value of one toss is , while the average of tosses is the proportion of heads. The chart below simulates 100 runs of 100,000 tosses each and draws each run’s proportion of heads after every toss, with the number of tosses on a logarithmic axis. The shaded band around 0.5 is 0.01 wide on each side.- Simulation
- Python code
What does the law of large numbers say, exactly?
Stated rigorously, it comes in two forms (Pishro-Nik) under one condition: are independent, identically distributed (i.i.d.) random variables with a finite expected value . For the coin, is the result of toss (1 or 0) and . Let be the mean of the first variables: Weak law. For every : is a miss of a given size. The probability that the mean misses by or more goes to 0 as grows. That is what you see looking down a vertical line at one number of tosses: the share of runs outside the band shrinks to 0. With , the probability of missing by or more is 92.04% after 100 tosses, 54.80% after 1,000 and 4.66% after 10,000. Strong law (section 7.2.7). converges to almost surely: The probability that the whole sequence of means of one run goes to , from start to end, is 1. That is what you see looking along a single line: with probability 1, a run eventually enters any narrow band around 0.5 and never leaves it. The weak law is about the probability at each ; the strong law is about the whole path of one run. According to Pishro-Nik, the two forms differ mainly in theory.References
Sources: OpenStax, Introductory Statistics 2e, section 7.3 and Pishro-Nik, sections 7.1.1 and 7.2.7.Read next
Descriptive statistics
The mean, median, standard deviation and quantiles of a data set.
Expected value
The average profit or loss per bet, and why a positive expected value still leaves some people in the red.

