## What’s Real About Probability? (11/365)

Suppose I tell you the axioms of elementary probability and suppose further that I tell you that I have a coin that, on tossing, will show Heads with a probability of $p$. Can you tell anything at all abou the coin?

You might be tempted to say something like “if I toss the coin $n$ times I’d probably expect to see heads $np$ times”. But this is a loaded response that seems out of scope of the simple axioms we started with. I say this because $p$, as it has been stated, says nothing whatsover about a fraction of tosses coming up heads.

In other words, is there any reality to the probability of heads $p$? As it turns out there is and is the content of Bernoulli’s Law of Large Numbers. Let’s see how this laws helps assign an intuitive reality for $p$.

Let’s first codify our intuition that the fraction of heads in $n$ tosses has something to do with $p$. Construct the following sample space for $n$ tosses. $\displaystyle \Omega_n = \{ \omega = \omega_1, \dots, \omega_N : \omega_i \in \{ 0,1 \} \}$

Define its probabilities as generated by the tossing of $n$ coins with success probability $p$. $\displaystyle P(\omega) = \prod_i P(\omega_i) = p^n$

Create a random variable to capture the number of heads. $\displaystyle S_n(\omega) = \sum_{i=1}^n \omega_i$

Our intuition says that after observing $n$ tosses $\omega$ we might expect $\frac{S(\omega)}{n}$ to be close to $p$. In analysis, we would expect the following to hold for sufficiently large $n$. $\displaystyle \left| \frac{S(\omega)}{n} - p \right| \le \epsilon \text{ for all } \omega \text{ and } \epsilon > 0$

But this can’t work for arbitrary $\epsilon$ because we are requiring that this be true for all $\omega$, which is not possible because if $0 < p < 1$, then the probability of getting all heads or all tails is still non-zero which prevents us from getting arbitrarily close to $p$ for any $\omega$.

However, note that as $n$ increases the probability of getting all heads or all tails become small. So, we can instead define closeness to $p$ by investivgating the following probability. $\displaystyle P \left( \left| \frac{S_n}{n} - p \right| \ge \epsilon \right)$

Using Chebyshev’s inequality we saw here, we can provide an upper bound $\displaystyle P \left( \left| \frac{S_n}{n} - p \right| \ge \epsilon \right) \\ \le \frac{1}{\epsilon^2} E\left( \frac{S_n}{n} - p \right)^2 \\ \le \frac{VS_n}{n^2 \epsilon^2} \\ = \frac{npq}{n^2 \epsilon^2} \\ \le \frac{1}{4 n \epsilon^2}$

Hence, we see that as $n \rightarrow \infty$ the probability that the fraction deviates from $p$ more than $\epsilon$ tends to zero. This says that $p$ indeed has a realistic interpretation as given by the fraction of heads observed. I have used the following fact in the above derivation. $\displaystyle E\frac{S(\omega)}{n} \\ = E(\frac{\omega_1 + \dots + \omega_n}{n}) \\ = E\frac{\omega_1}{n} + \dots + E\frac{\omega_n}{n} \\ = p$

This entry was posted in Uncategorized and tagged , . Bookmark the permalink.