What’s Real About Probability? (11/365)

Suppose I tell you the axioms of elementary probability and suppose further that I tell you that I have a coin that, on tossing, will show Heads with a probability of p. Can you tell anything at all abou the coin?

You might be tempted to say something like “if I toss the coin n times I’d probably expect to see heads np times”. But this is a loaded response that seems out of scope of the simple axioms we started with. I say this because p, as it has been stated, says nothing whatsover about a fraction of tosses coming up heads.

In other words, is there any reality to the probability of heads p? As it turns out there is and is the content of Bernoulli’s Law of Large Numbers. Let’s see how this laws helps assign an intuitive reality for p.

Let’s first codify our intuition that the fraction of heads in n tosses has something to do with p. Construct the following sample space for n tosses.

\displaystyle  \Omega_n = \{ \omega = \omega_1, \dots, \omega_N : \omega_i \in \{ 0,1 \} \}

Define its probabilities as generated by the tossing of n coins with success probability p.

\displaystyle  P(\omega) = \prod_i P(\omega_i) = p^n

Create a random variable to capture the number of heads.

\displaystyle  S_n(\omega) = \sum_{i=1}^n \omega_i

Our intuition says that after observing n tosses \omega we might expect \frac{S(\omega)}{n} to be close to p. In analysis, we would expect the following to hold for sufficiently large n.

\displaystyle  \left| \frac{S(\omega)}{n} - p \right| \le \epsilon \text{ for all } \omega \text{ and } \epsilon > 0

But this can’t work for arbitrary \epsilon because we are requiring that this be true for all \omega, which is not possible because if 0 < p < 1, then the probability of getting all heads or all tails is still non-zero which prevents us from getting arbitrarily close to p for any \omega.

However, note that as n increases the probability of getting all heads or all tails become small. So, we can instead define closeness to p by investivgating the following probability.

\displaystyle  P \left( \left| \frac{S_n}{n} - p \right| \ge \epsilon \right)

Using Chebyshev’s inequality we saw here, we can provide an upper bound

\displaystyle  P \left( \left| \frac{S_n}{n} - p \right| \ge \epsilon \right) \\ \le \frac{1}{\epsilon^2} E\left( \frac{S_n}{n} - p \right)^2 \\ \le \frac{VS_n}{n^2 \epsilon^2} \\ = \frac{npq}{n^2 \epsilon^2} \\ \le \frac{1}{4 n \epsilon^2}

Hence, we see that as n \rightarrow \infty the probability that the fraction deviates from p more than \epsilon tends to zero. This says that p indeed has a realistic interpretation as given by the fraction of heads observed. I have used the following fact in the above derivation.

\displaystyle  E\frac{S(\omega)}{n} \\ = E(\frac{\omega_1 + \dots + \omega_n}{n}) \\ = E\frac{\omega_1}{n} + \dots + E\frac{\omega_n}{n} \\ = p

This entry was posted in Uncategorized and tagged , . Bookmark the permalink.

1 Response to What’s Real About Probability? (11/365)

  1. Pingback: Chebyshev’s Inequality For Bounding From Below? (12/365) | Latent observations

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s