Introduction to Joint Random Variables

Consider 2 random variables over a joint sample space, $X$ and $Y$ .

Note that techniques in this entire section generalize to $n$ variable functions, though for simplicity we will only cover the 2-variable case.

Discrete Case

In the discrete case, if $X$ and $Y$ are discrete random variables, then the joint probability distribution of $X$ and $Y$ , denoting the probability of the value pair $(x \in X, y \in Y)$ occurring, is given as

f (x, y) = P (X = x, Y = y)

In the discrete case of 2 random variables, we often find it useful to represent $f (x, y)$ as a table of values, where rows denote one random variable, and columns denote the other.

The discrete joint pdf has the following properties:

$f (x, y) \geq 0$
$\sum_{x} \sum_{y} f (x, y) = 1$

Example: Joint Probability Distributions (Discrete Case)

There are 3 (different) math books, 2 history, 4 physics.

Before, we could only define a single random variable $X$ for one type of book specifically. Now, using joint distributions, we can define multiple random variables at once.

Let $X$ be the total math books chosen, and $Y$ be the total history books chosen, out of 2 books total. We can use $f (x, y)$ to find the probability of multiple random variables occurring at once.
$P (X = 1, Y = 1) = \frac{( 1 3 ) ( 1 2 )}{( 2 9 )} = \frac{1}{6}$

Furthermore, given $f (x, y)$ , the joint cumulative probability distribution of $X$ and $Y$ is given as

F (x, y) = P (X \leq x, Y \leq y) = s \leq x \sum t \leq y \sum f (s, t)

Where $f (s, t)$ is the pdf of the joint distribution.

The discrete joint cdf has the following properties:

$F (- \infty, - \infty) = 0$
$F (\infty, \infty) = 1$
If $a < b$ , and $c < d$ , then $F (a, c) \leq F (b, d)$ .

Continuous Case

In the continuous case, if $X$ and $Y$ are continuous random variables, then the joint probability density function $f (x, y)$ over $X$ and $Y$ is given as the function satisfying

P (X, Y) \in R = \int \int_{R} f (x, y) d A

Where $R$ is a region on the $x y$ -plane.

The continuous joint pdf has the following properties:

$f (x, y) \geq 0$ for all $x, y \in R$ .
$\int_{- \infty}^{\infty} \int_{- \infty}^{\infty} f (x, y) d y d x = 1$

Furthermore, given $f (x, y)$ , we can find the joint cumulative probability distribution of $X$ and $Y$ as

F (x, y) = P (X \leq x, Y \leq y) = \int_{- \infty}^{y} \int_{- \infty}^{x} f (s, t) d s d t

Similar to the continuous one-variable case, we can differentiate the cdf to find our pdf, and integrate our pdf to find our cdf. Note that order of differentiation and integration do not matter.

f (x, y) = \frac{\partial ^{2}}{\partial x \partial y} F (x, y) = F_{y x} = \frac{\partial ^{2}}{\partial x \partial y} F (x, y) = F_{x y}

Consider the following examples.

Example: Joint Continuous Distributions

Given the joint cdf
$F (x, y) = {(1 - e^{- x}) (1 - e^{- y}) 0 x, y > 0 else$
Find $P (1 < X < 3, 1 < Y < 2)$ .

To find this, we first find $f (x, y)$ by taking the partial derivative of the cdf twice.
$f (x, y) = F_{x y} (x, y) = e^{- (x + y)} = F_{y x}$ $\to P (1 < X < 3, 1 < Y < 2) = \int_{1}^{2} \int_{1}^{3} e^{- (x + y)} d x d y = 0.074$

Example: Joint Probability Distribution (2)

In a study on workers, the hours $X$ spent using their phones, and the hours $Y$ spent on their job is approximated by the joint pdf
$f (x, y) = x y e^{- (x + y)} x, y > 0$
What is the probability a person spends at least twice as much time on their phone than doing their job?

To find this, we want to find $P (X \geq 2 Y)$ .
$P (X \geq 2 Y) = \int_{0}^{\infty} \int_{0}^{X /2} x y e^{- (x + y)} d y d x$

Example: Joint Continuous Distributions - PDF to CDF

$f (x, y) = {x + y 0 0 < x < 1, 0 < y < 1 else$
Given the above joint continuous pdf, find the cdf of $X$ and $Y$ .

By virtue of multiple variables, we’ll have to address numerous cases to find $F (x, y)$ .

If $x, y \leq 0$ , then $F (x, y) = 0$ .

If $x, y \geq 1$ , then $F (x, y) = 1$ .

If $0 < x, y < 1$ , $F (x, y) = \int_{0}^{y} \int_{0}^{x} (s + t) d s d t = \frac{1}{2} x y (x + y)$

If $x > 1, 0 < y < 1$ , $F (x, y) = \int_{0}^{y} \int_{0}^{1} (s + t) d s d t = \frac{1}{2} (y) (y + 1)$

If $y > 1$ , $0 < x < 1$ , $F (x, y) = \frac{1}{2} (x) (x + 1)$

Putting these cases together, we find cdf
$F (x, y) = ⎩ ⎨ ⎧ 0 \frac{1}{2} x y (x + y) \frac{1}{2} y (y + 1) \frac{1}{2} x (x + 1) 1 x, y \leq 0 0 < x, y < 1 0 < y < 1, x > 1 0 < x < 1, y > 1 1 \leq x, y$

Conditional Distributions and Independence

If $f (x, y)$ is the joint pdf of a random variable, then the marginal distribution of $X$ , denoting the total probability of $x$ occurring over all cases $y$ , is given as

g_{X} (x) = {\sum_{y} f (x, y) \int_{- \infty}^{\infty} f (x, y) d y Discrete Case Continuous Case

The marginal distribution of $Y$ follows similarly.

Note that in a marginal distribution, we sum / integrate over the other variable.

Now, given joint pdf $f (x, y)$ and marginal distribution of $Y$ $g_{Y} (y)$ , we can find the conditional distribution of $X$ given $Y = y$ as

f (x ∣ y) = \frac{f ( x , y )}{g _{Y} ( y )} y_{Y} (y) \neq = 0

This definition is analogous to the definition of conditional probability in the one-variable case.

Similarly, the cumulative conditional distribution of $X$ given $Y = y$ is given as

F (x ∣ y) = P (X \leq x ∣ Y = y) = {\sum_{a \leq x} f (a ∣ y) \int_{- \infty}^{x} f (t ∣ y) d t Discrete Case Continuous Case

Note that for the cumulative conditional distribution, $y$ is fixed.

Example: Joint Conditional Distributions

Let the joint pdf of $X$ and $Y$ be given as
$f (x, y) = {\frac{2}{3} (x + 2 y) 0 0 < x, y < 1 else$
Find $P (X \leq 1/2∣ Y = 1/2)$ .

First, we find the marginal pdf as
$g_{Y} (y) = \int_{- \infty}^{\infty} f (x, y) d x = \int_{0}^{1} f (x, y) d x = \frac{1}{3} (1 + 4 y)$
and use this to find $f (x ∣ y)$ as
$f (x ∣ y) \to f (x ∣ \frac{1}{2}) = \frac{f ( x , y )}{g _{Y} ( y )} = \frac{\frac{2}{3} ( x + 2 y )}{\frac{1}{3} ( 1 + 4 y )} = \frac{2 x + 2}{3}$
Finally, we use this to find our answer as
$P (X \leq 1/2∣ Y = 1/2) = \int_{- \infty}^{x} f (t ∣ \frac{1}{2}) d t = \int_{0}^{1/2} \frac{2 x + 2}{3} d x = \frac{5}{12}$

Let $X_{1}, X_{2}, \dots X_{n}$ be random variables, and let $g_{i} (X)$ be the marginal distribution of random variable $X_{i}$ .

We say that random variables $X_{1}, X_{2}, \dots X_{n}$ are independent if and only if for all $(x_{1}, \dots, x_{n})$ that $X_{1}, \dots, X_{n}$ take on, we have that

f (x_{1}, \dots, x_{n}) = g_{1} (x_{1}) g_{2} (x_{2}) \dots g_{n} (x_{n})

In other words, the joint probability distribution of all of the variables equals the product of their marginal probability distributions.

It naturally follows from this that if $X$ and $Y$ are independent, then

f (x ∣ y) = \frac{f ( x , y )}{g _{Y} ( y )} = \frac{g _{X} ( x ) g _{Y} ( y )}{g _{Y} ( y )} = g_{X} (x)

This is the same as our one-variable case! It tells us that the conditioning of $Y$ does not affect our probability of $X$ , if they are independent.

Consider the following examples for independence.

Example: Independence on Joint Distributions

Given the joint pdf of $X, Y$
$f (x, y) = {12 x y (1 - y) 0 0 < x, y < 1 else$
We find that $X$ and $Y$ are independent, as $g_{X} (x) \cdot g_{Y} (y) = f (x, y)$ .
$g_{X} (x) g_{Y} (y) g_{X} (x) g_{Y} (y) = \int_{0}^{1} f (x, y) d y = 2 x = \int_{0}^{1} f (x, y) d x = 6 (1 - y) y = 12 x y (1 - y) = f (x, y)$

Knowing independence can be highly useful, and simplify many computations. For example, it gives us an easy shortcut for evaluating combinations of random variables!

Theorem: Summation of Independent Random Variables

Let $X, Y$ be independent random variables with pmf/pdfs given as $f_{X} (x), f_{Y} (y)$ . Define $Z = X + Y$ .

Then, for the continuous case,
$f_{Z} (z) = \int_{- \infty}^{\infty} f_{X} (x) f_{Y} (z - x) d x$
Called the convolution of the two functions.

Example: Summation of Independent Random Variables

Let $X \sim Binom (n, p)$ and $Y \sim Binom (m, p)$ . Furthermore, let $X, Y$ be independent and assume
$f_{X} (t) = f_{Y} (t) = {e^{- t} 0 t > 0 else$
What is the pdf of random variable $Z = X + Y$ ?

We can find this using the convolution of $X$ and $Y$ .
$f_{Z} (z) = \int_{- \infty}^{\infty} f_{X} (x) f_{Y} (z - x) d x = \int_{0}^{z} e^{- x} e^{- (z - x)} d x = z e^{- z}$
Note that we can only do this because our random variables are independent of one another.

Transformations of Joint Distributions

Recall in the one-variable case, that for random variable $X$ with pdf $f_{X} (x)$ , and random variable $Y = g (X)$ , we can find the pdf of $Y$ as

f_{Y} (y) = f_{X} (g^{- 1} (y)) \cdot \frac{d}{d y} (g^{- 1} (y))

Let’s generalize this fact to multiple variables. Given joint pdf $f_{X, Y} (x, y)$ and random variables $u = h_{1} (x, y)$ and $v = h_{2} (x, y)$ , how do we find the joint pdf of $u$ and $v$ ?

Assuming that the partial derivatives exist, and that $h_{1} (x, y)$ and $h_{2} (x, y)$ are one-to-one, we can perform a change in variables to find our pdf (similar to multivariable calculus). This gives us the following theorem:

Theorem: Transforming Joint Distributions

Let $X, Y$ be random variables with joint pdf $f_{X, Y} (x, y)$ and let there be random variables
$U = h_{1} (X, Y) V = h_{2} (X, Y)$
Then, the joint pdf of $U$ and $V$ , $f_{U, V} (u, v)$ , is given as
$f_{U, V} (u, v) = f_{X, Y} (x (u, v), y (u, v)) J (u, v)$
Where the Jacobian, $J$ (provided $J \neq = 0$ ) is given as
$J = det [\partial x / \partial u \partial y / \partial u \partial x / \partial v \partial y / \partial v]$

Note: One can show that with $U = h_{1} (X, Y), V = h_{2} (X, Y)$ ,
$J^{- 1} = \frac{1}{det [ \partial h _{1} / \partial x \partial h _{2} / \partial x \partial h _{1} / \partial y \partial h _{2} / \partial y ]}$
Which can be helpful if $J^{- 1}$ evaluates to a constant.

Consider the following examples.

Example: Transforming Joint Distributions

Let the joint pdf of random variables $X, Y$ be given as
$f (x, y) = {e^{- (x + y)} 0 x, y > 0 else$
Find the joint pdf of random variables $U, V$ where
$U = X + Y, V = \frac{X}{X + Y}$
First, we solve for $x, y$ in terms of $u, v$ .
$x = uv y = u - uv$
We can use this perform a change of variables using our theorem.
$J (u, v) = det [\partial x / \partial u \partial y / \partial u \partial x / \partial v \partial y / \partial v] = v 1 - v u - u = - u$ $f_{U, V} (u, v) = f_{X, Y} (uv, u - uv) ∣ - u ∣ = u e^{- u} 0 < u, 0 < v < 1$

Expectation and Variance

Let $X, Y$ be random variables. If $f (x, y)$ is their pdf, then the expected value of some combination of $X$ and $Y$ , given as $g (x, y)$ is given as

E (g (x, y)) = {\sum_{x} \sum_{y} g (x, y) \cdot f (x, y) \int_{- \infty}^{\infty} \int_{- \infty}^{\infty} g (x, y) \cdot f (x, y) d y d x Discrete Case Continuous Case

Provided the values exist.

Theorem: Linearity of Expectation

Let $c_{1}, \dots c_{n}$ be constants, and $X_{1}, \dots X_{n}$ be random variables. Then,
$E (i = 1 \sum n c_{i} g_{i} (x_{1}, \dots, x_{n})) = i = 1 \sum n c_{i} E (g_{i} (x_{1}, \dots, x_{n}))$
In particular, we can apply this theorem for $X$ and $Y$ on $c_{1}$ and $c_{2}$ , to find
$E (X + Y) = E (X) + E (Y)$

Example: Expectation

Let $X$ and $Y$ be random variables with pdf $f (x, y)$ given as
$f (x, y) = {\frac{2}{7} (x + 2 u) 0 0 < x < 1, 1 < y < 2 else$
We can find the expectation of some combination of $X$ and $Y$ , say, $g (x, y) = X / Y^{3}$ .
$E (\frac{X}{Y ^{3}}) = \int_{1}^{2} \int_{0}^{1} (\frac{X}{Y ^{3}}) \frac{2}{7} (x + 2 y) d x d y = \frac{10}{56}$

Recall that if $X, Y$ are independent, then $f_{X, Y} (x, y) = f_{X} (x) f_{Y} (y)$ . Supposing that $X, Y$ are independent, we find that

E (X Y) = \int_{- \infty}^{\infty} \int_{- \infty}^{\infty} x y f_{X, Y} (x, y) = \int_{- \infty}^{\infty} \int_{- \infty}^{\infty} x y f_{X} (x) f_{Y} (y) d y d x = \int_{- \infty}^{\infty} x f_{X} (x) d x \int_{- \infty}^{\infty} y f_{Y} (y) d y = E (X) \cdot E (Y)

This is formalized in the following theorem.

Theorem: Independence and Expectation

If $X, Y$ are independent, then
$E (X Y) = E (X) \cdot E (Y)$

Note that this does not, however, imply the converse. In other words, $E (X Y) = E (X) \cdot E (Y)$ does not necessarily mean $X$ and $Y$ are independent.

When $X$ and $Y$ are not independent, we can measure their relationship with one another. One measure that may help in doing this is the covariance, denoted $Cov (X, Y)$ .

C o v (X, Y) = E ((X - E (X)) (Y - E (Y))) = E ((X - μ_{x}) (Y - μ_{y}))

$C o v (X, Y)$ is seen as a measure of the linear relationship of $X$ and $Y$ .

The covariance tells us the direction of the relationship between $X$ and $Y$ .

If covariance is negative, then $X$ and $Y$ have a negative relationship (as $X$ increases, $Y$ decreases)
If covariance is positive, then $X$ and $Y$ have a positive relationship (as $X$ increases, $Y$ increases)

Theorem: Covariance

$C o v (X, Y) = E (X Y) - E (X) E (Y)$

Proof

$E ((X - E (X)) (Y - E (Y))) = E (X Y - E (X) Y - X E (Y) + E (X) E (Y)) = E (X Y) - E (X) E (Y) - E (X) E (Y) + E (X) E (Y) = E (X Y) - E (X) E (Y)$

It follows from this theorem that if $X$ and $Y$ are independent, then $C o v (X, Y) = 0$ (they are uncorrelated). Other properties of our covariance are given below:

$C o v (X, X) = V (X)$
$C o v (X, Y) = C o v (Y, X)$
$C o v (k X, Y) = k C o v (X, Y)$

Example: Covariance

Let the pdf of $X, Y$ be given as
$f (x, y) = {20 x > 0, y > 0, x + y < 1 else$
Find $C o v (X, Y)$ .

We find
$E (X) = \int_{0}^{1} \int_{0}^{1 - y} x \cdot 2 d x d y = \int_{0}^{1} (1 - y)^{2} d y = \frac{1}{3} E (Y) = \int_{0}^{1} \int_{0}^{1 - y} y \cdot 2 d x d y = \int_{0}^{1} 2 y (1 - y) d y = \frac{1}{3} E (X Y) = \int_{0}^{1} \int_{0}^{1 - y} x y \cdot 2 d x d y = \int_{0}^{1} y (1 - y)^{2} d y = \frac{1}{12} C o v (X Y) = E (X Y) - E (X) E (Y) = \frac{1}{12} - \frac{1}{3} \cdot \frac{1}{3} = - 1/36$

Note that using the definition of covariance, we can derive expressions for the covariance of sums of random variables. This is given in the following theorem.

Theorem: Covariance of Sums

Let $X_{1}, \dots, X_{n}$ be random variables, and assume
$Y_{1} = i = 1 \sum n a_{i} X_{i} Y_{2} = i = 1 \sum n b_{i} X_{i}$
Then,
$C o v (Y_{1}, Y_{2}) = i = 1 \sum n a_{i} b_{i} V (X_{i}) + 2 i < j \sum a_{i} b_{j} C o v (X_{i}, X_{j})$

Corollary

If $Y = X_{1} + X_{2} + \dots + X_{n}$ , then
$C o v (Y, Y) = V (Y) = i = 1 \sum n V (X_{i}) + 2 i < j \sum C o v (X_{i}, X_{j})$
In particular, if $X_{1}, \dots, X_{n}$ are independent, then $V (Y)$ is the sum of the variances of $X_{i}$ !
$V (Y) = i = 1 \sum n V (X_{i})$

Example: Variance of Sums

Let the joint pdf of $X, Y$ be given as
$f (x, y) = {\frac{1}{3} (x + y) 0 0 < x < 1, 0 < y < 2 else$
Find $V (Z)$ , where $Z = 3 X + 4 Y - 5$ .

We can apply our theorem for $Z = 3 X + 4 Y - 5$ . Note that because the variance of a constant is 0, we see that
$C o v (X, 5) = C o v (Y, 5) = 0$
We can now calculate
$V (3 X + 4 Y) = V (3 X) + V (4 Y) + 2 C o v (X, Y) = 9 V (X) + 16 V (X) + 2 (3) (4) C o v (X, Y)$

We can further use covariance to define correlation coefficient of $X$ and $Y$ as

P_{X, Y} = \frac{C o v ( X , Y )}{V ( X ) V ( Y )}

measuring the “power” of the relationship - or in other words, how related $X$ and $Y$ are.

Properties of the correlation coefficient are given below.

$- 1 \leq P_{X, Y} \leq 1$
$P_{X, Y} = \pm 1$ if and only if $Y = α X + β$ , for some $α, β \in R$ .

Conditional Expectation

Let $X$ and $Y$ be random variables, and let $g (X)$ be a function on random variable $X$ .

Then, the conditional expectation of $g (X)$ given $Y = y$ is defined as

E (g (x) ∣ y) = {\sum_{x} g (x) f (x ∣ y) \int_{- \infty}^{\infty} g (x) f (x ∣ y) d x Discrete Case Continuous Case

In particular, if $g (x)$ is given as $x^{i}$ , then we define the conditional $i^{t h}$ moment as $E (x^{i} ∣ y)$ . We can use this to define the following:

E (X ∣ Y) V (X ∣ Y) = E (X^{2} ∣ Y) - E (X ∣ Y)^{2} Conditional Mean Conditional Variance

We also have the following properties:

Theorem: Conditional Expectation and Expectation / Variance

The following two equalities are true.
$E (X^{i}) = E (E (X^{i} ∣ Y)) V (X) = V (E (X ∣ Y)) + E (V (X ∣ Y))$
We prove them below.

Proof (1)

First, define function $h (Y)$ as
$h (Y) = E (X^{i} ∣ Y = y) = \int_{- \infty}^{\infty} x^{i} f (x ∣ y) d x = \int_{- \infty}^{\infty} x^{i} \frac{f ( x , y )}{g _{Y} ( y )} d x$
We want to use this to show that $E (X^{i}) = E (h (Y))$ .
$E (h (Y)) = \int_{- \infty}^{\infty} h (y) g_{Y} (y) d y = \int_{- \infty}^{\infty} [\int_{- \infty}^{\infty} x^{i} \frac{f ( x , y )}{g _{Y} ( y )} d x] g_{Y} (y) d y = \int_{- \infty}^{\infty} [\int_{- \infty}^{\infty} x^{i} f (x, y) d x] d y$
Here, we apply the definition of conditional probability (multivariate case)
$f (y ∣ x) = \frac{f ( x , y )}{g _{X} ( x )}$ $\int_{- \infty}^{\infty} [\int_{- \infty}^{\infty} x^{i} f (x, y) d x] d y = \int_{- \infty}^{\infty} \int_{- \infty}^{\infty} x^{i} g_{X} (x) f (y ∣ x) d x d y = \int_{- \infty}^{\infty} \int_{- \infty}^{\infty} x^{i} g_{X} (x) f (y ∣ x) d y d x = \int_{- \infty}^{\infty} x^{i} g_{X} (x) [\int_{- \infty}^{\infty} f (y ∣ x) d y] d x = \int_{- \infty}^{\infty} x^{i} g_{X} (x) (1) d x = E (X^{i})$

Proof (2)

We start from the right-hand side expression, and find that
$V (E (X ∣ Y)) + E (V (X ∣ y)) = E (E (X ∣ Y)^{2} - E (E (X ∣ Y)))^{2} + E (E (X^{2} ∣ Y) - (E (X ∣ Y))^{2}) = E (E (X ∣ Y)^{2}) - [E (X)]^{2} + E (E (X^{2} ∣ Y)) - E (E (X ∣ Y))^{2} = E (X^{2}) - (E (X))^{2} = V (X)$

Theorem: Law of Total Expectation

We have
$E (X) = E (E (X ∣ Y)) = y \sum E (X ∣ Y = y) P (Y = y)$
We can generalize this fact further. If $A_{1}, \dots A_{n}$ partition the sample space, then
$E (X) = i = 1 \sum n E (X ∣ A_{i}) P (A_{i})$

Example: Conditional Expectation

Suppose that
$P (y = 1) = \frac{1}{8} P (y = 2) = \frac{7}{8}$
Now suppose we have random variable $Z = X ∣ Y$ , and define
$P (z = 2 y) = \frac{3}{4} P (z = 3 y) = \frac{1}{4}$
On the cases $y = 1, 2$ , then
$Z = X ∣ (Y = 1) = {2 (1) 3 (1) P = 3/4 P = 1/4 Z = X ∣ (Y = 2) = {2 (2) 3 (2) P = 3/4 P = 1/4$
We use this to find expected values
$E (X ∣ Y = 1) = 2 \cdot \frac{3}{4} + 3 \cdot \frac{1}{4} = \frac{9}{4} E (X ∣ Y = 2) = 4 \cdot \frac{3}{4} + 6 \cdot \frac{1}{4} = \frac{18}{4}$
This gives us the expected value distribution
$E (X ∣ Y) = {9/4 18/4 y = 1 y = 2$
We can then use this to find $E (E (X ∣ Y)) = E (X)$ !
$E (X) = E (E (X ∣ Y)) = \frac{9}{4} P (y = 1) + \frac{18}{4} P (y = 2) = \frac{9}{4} \cdot \frac{1}{8} + \frac{18}{4} \cdot \frac{7}{8}$

Example: Conditional Expectation (2)

Let
$f_{X, Y} (x, y) = {x^{2} e^{- x (y + 1)} 0 x, y > 0 else$
Find $E (X ∣ Y)$ .

Given that
$g_{Y} (y) = \frac{2}{( y + 1 ) ^{3}}$
We can find $E (X ∣ Y)$ by finding
$E (X ∣ Y) = \int_{- \infty}^{\infty} x f (x ∣ y) d x = \int_{0}^{\infty} xx (\frac{x ^{2} e ^{- x (y + 1)}}{g _{Y} ( y )}) = \int_{0}^{\infty} \frac{1}{2} (y + 1)^{3} x^{3} e^{- x (y + 1)} d x$
Let $u = x (y + 1) \to d u = y + 1$ . We find
$E (X ∣ Y) = \frac{1}{2 ( y + 1 )} \int_{0}^{\infty} u^{3} e^{- u} d u = \frac{1}{2 ( y + 1 )} Γ (4) = \frac{3 !}{2 ( y + 1 )}$

Sometimes, we can use conditional expectation to find expected values that are recursively defined in terms of themselves. See the following examples.

Example: Conditional Expectation (3)

A man is stuck in a cave. There are 3 tunnel exits:

Tunnel 1 takes 2 hours to get out.

Tunnel 2 takes 5 hours, but returns to the starting point.

Tunnel 3 takes 7 hours, but returns to the starting point.

If he picks a tunnel at random each time, what is the expected time until he gets out?

Let $X$ be the total hours to get out, and $Y$ be the tunnel he enters initially. We want
$E (X) = y = 1 \sum 3 E (X ∣ Y = y) P (Y = y) = E (X ∣ Y = 1) P (Y = 1) + P (X ∣ Y = 2) P (Y = 2) + P (X ∣ Y = 3) P (Y = 3) = 2 \frac{1}{3} + (5 + E (X)) \frac{1}{3} + (7 + E (X)) \frac{1}{3}$
Note that we express $E (X)$ recursively in terms of itself, as if the man enters tunnel 2 or 3, he ends up back where he started but with additional time spent.

Solving for $E (X)$ , we find $E (X) = 14$ .

Example: Conditional Expectation (4)

A biased coin has $P (Heads) = p$ . We toss the coin until we get 2 consecutive heads. What is the number of expected tosses?

Let:

$A_{1}$ be the event of the sequence $H, H$ .

$A_{2}$ be the event of the sequence $H, T$ .

$A_{3}$ be the event of the sequence of just $T$ .

As $A_{1}, A_{2}, A_{3}$ partition the sample space of the toss sequences, we can find our expected value by breaking it up across these different sequences.

Define $X$ as the total number of tosses needed until we get 2 consecutive heads. Then,
$E (X) = E (X ∣ A_{1}) P (A_{1}) + E (X ∣ A_{2}) P (A_{2}) + E (X ∣ A_{3}) P (A_{3}) = 2 \cdot p^{2} + (2 + E (X)) \cdot p (1 - p) + (1 + E (X)) \cdot (1 - p)$
Solving for $E (X)$ , we find $E (X) = \frac{p + 1}{p ^{2}}$ .

Quartz 5

Explorer

Random Variables (Multivariate)

Introduction to Joint Random Variables

Discrete Case

Continuous Case

Conditional Distributions and Independence

Transformations of Joint Distributions

Expectation and Variance

Conditional Expectation

Graph View

Backlinks