Sometimes expectations and random variables appear in so many ways that I find it a little confusing at times. The book I am following has a neat diagram that helps quite a bit.
Start with a distribution for sample space and a random variable . The basic expectation tells us the value is likely to take on average.
Note that induces a decomposition of as follows
Instead of we could be given a distribution with respect to an event . The expectation over this is the value is likely to take on average conditioned on the event (i.e. restricted to).
Let’s suppose that we have an event and its conditional probabilities where is a decomposition of . We can write this as a random variable that takes on the value whenever .
Now that we have this random variable, we can once again take its expectation which in this case gives the probability of on average conditioned on an event from . This has a special name and is called the total probability.
One More Generalization
Suppose now that we have a random variable inducing the decomposition where . We also have conditional probabilities . We can certainly take the following expectation from before
which is the expectation of conditioned on . We now do this for the entire decomposition to arrive at this random variable
which takes on a conditional expectation of at each . We can now generalize the total probability formula such that it gives the the expectation of on average conditioned on an event from .
The lesson here is to always translate the things we do with probabilities to expectations.
Pingback: Conditional Probability – Problem (19/365) | Latent observations