Correlation – I (6/365)

Let me follow up on the theme – talked about here – of distributing an operator over another. To recap, we saw two ways of distributing an expectation

\displaystyle E(\theta + \phi) = E(\theta) + E(\phi) \\ (E \theta \phi)^2 = E \theta^2 E \phi^2 \\ E\theta \phi = E\theta E\phi \text{ if } \theta, \phi \text{ are independent}

Can we say anything like this about variance? Consider the variance of the sum of random variables (recall that V\theta = E(\theta - E\theta)^2).

\displaystyle V(\theta + \phi) = V\theta + V\phi + 2E(\theta - E\theta)(\phi - E\phi)

Looks like we have an extra term. Before trying to see what this is, can we think of a situation when this term could be 0 and thus give us V(\theta + \phi) = V\theta + V\phi? The answer is yes because if \theta and \phi are independent, then 2E(\theta - E\theta)(\phi - E\phi) = 2E\theta\phi - 2E\theta E\phi = 2E\theta \phi - 2E\theta \phi = 0.

In general, though, this extra term will be non-zero. How can we make sense of this extra term? If you go back to the Cauchy-Bunyakovskii inequality we looked at, we can write this term as

\displaystyle (E(\theta - E\theta)(\phi - E\phi))^2 \le E(\theta - E\theta)^2 E(\phi - E\phi)^2 \\ (E(\theta - E\theta)(\phi - E\phi))^2 \le V\theta V\phi

So it looks as if we can relate the extra term to the variance of the random variables. More specifically, we see that

\displaystyle | \rho(\theta,\phi) | = | \frac{E(\theta - E\theta)(\phi - E\phi)}{\sqrt{V\theta V\phi}} | \le 1

And this gets to be called the correlation coefficient. Why is this interesting you ask? First, note that

\displaystyle \rho(\theta,\phi) = 0 \implies V(\theta + \phi) = V\theta + V\phi

You’ve often heard how a lack of correlation does not imply independence of random variables. Well, the above identity is the conclusive proof. Because

\displaystyle \text{if } E(\phi - E\phi)(\theta - E\theta) = E\theta\phi - E\theta E\phi = 0 \\ \text{then } E\theta\phi = E\theta E\phi \text{ does not imply } \theta,\phi \text{ independent}

Will furnish with an example of when this happens. It seems tricky to come up with an example.


This entry was posted in Uncategorized and tagged , . Bookmark the permalink.

1 Response to Correlation – I (6/365)

  1. Pingback: Correlation – III, Linear Dependence (8/365) | Latent observations

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s