Suppose I tell you the axioms of elementary probability and suppose further that I tell you that I have a coin that, on tossing, will show Heads with a probability of . Can you tell anything at all abou the coin?

You might be tempted to say something like “if I toss the coin times I’d probably expect to see heads times”. But this is a loaded response that seems out of scope of the simple axioms we started with. I say this because , as it has been stated, says nothing whatsover about a fraction of tosses coming up heads.

In other words, is there any reality to the probability of heads ? As it turns out there is and is the content of Bernoulli’s Law of Large Numbers. Let’s see how this laws helps assign an intuitive reality for .

Let’s first codify our intuition that the fraction of heads in tosses has something to do with . Construct the following sample space for tosses.

Define its probabilities as generated by the tossing of coins with success probability .

Create a random variable to capture the number of heads.

Our intuition says that after observing tosses we might expect to be close to . In analysis, we would expect the following to hold for sufficiently large .

But this can’t work for arbitrary because we are requiring that this be true for all , which is not possible because if , then the probability of getting all heads or all tails is still non-zero which prevents us from getting arbitrarily close to for any .

However, note that as increases the probability of getting all heads or all tails become small. So, we can instead define closeness to by investivgating the following probability.

Using Chebyshev’s inequality we saw here, we can provide an upper bound

Hence, we see that as the probability that the fraction deviates from more than tends to zero. This says that indeed has a realistic interpretation as given by the fraction of heads observed. I have used the following fact in the above derivation.