ps5_solution_W19 University of California, San Diego ECE 250

UNIVERSITY OF CALIFORNIA, SAN DIEGO
Electrical & Computer Engineering Department
ECE 250 - Winter Quarter 2019
Random Processes

Solutions to P.S. #5


1. Cauchy–Schwartz inequality.

(a) Prove the following inequality: (E(XY ))2 ≤ E(X 2 )E(Y 2 ). (Hint: Use the fact that for
any real t, E((X + tY )2 ) ≥ 0.)
(b) Prove that equality holds if and only if X = cY for some constant c. Find c in terms of
the second moments of X and Y .
(c) Use the Cauchy–Schwartz inequality to show the correlation coefficient |ρX,Y | ≤ 1.
p p p
(d) Prove the triangle inequality: E((X + Y )2 ) ≤ E(X 2 ) + E(Y 2 ).

Solution:
 
(a) We have, for every t ∈ R, E[(X + tY )2 ] ≥ 0, i.e., min t2 E[Y 2 ] + 2tE[XY ] + E[X 2 ] ≥ 0.
t∈R

From calculus, we see that the expression on the left attains its minimum value when
E[XY ]
t=− , and this minimum value is given by
E[Y 2 ]
(E[XY ])2
E[X 2 ] − . Thus, since E[Y 2 ] is non-negative, we have
E[Y 2 ]

E[X 2 ]E[Y 2 ] − (E[XY ])2 ≥ 0, i.e.

(E[XY ])2 ≤ E[X 2 ]E[Y 2 ].
(b) For the “if” part, we see that if X = cY for some constant c, then E[X 2 ] = c2 E[Y 2 ] and
E[XY ] = cE[Y 2 ], thus

(E[XY ])2 = c2 (E[Y 2 ])2
= c2 E[Y 2 ]E[Y 2 ]
= E[X 2 ]E[Y 2 ].

Thus, equality holds in this case.

For the “only if” part, we see from part (a) that equality will hold only if for some
real t, E[(X + tY )2 ] = 0.

Since (X + tY )2 is non-negative, this implies that X + tY = 0, i.e., X = −tY . Writing
c = −t, the result follows.

1

, (c) Writing X1 = X − E[X] and Y1 = Y − E[Y ], we have, from the Cauchy-Schwartz
inequality,
(E[X1 Y1 ])2 ≤ E[X12 ]E[Y12 ]
(E[X1 Y1 ])2
=⇒ ≤1
E[X12 ]E[Y12 ]
(E[(X − E[X])(Y − E[Y ])])2
=⇒ ≤1
E[(X − E[X])2 ]E[(Y − E[Y ])2 ]
(Cov(X, Y ))2
=⇒ ≤1
Var(X) Var(Y )
=⇒ ρ2X,Y ≤1
=⇒ |ρX,Y | ≤ 1.

(d) We have (E[XY ])2 ≤ E[X 2 ]E[Y 2 ] and thus,
p
E[XY ] ≤ |E[XY ]| ≤ E[X 2 ]E[Y 2 ].

Thus,
E[(X + Y )2 ] = E[X 2 ] + E[Y 2 ] + 2E[XY ]
≤ E[X 2 ] + E[Y 2 ] + 2 E[X 2 ]E[Y 2 ]
p
p p 2
= E[X 2 ] + E[Y 2 ] ,

and taking square roots, the result follows.

2. Neural net. Let Y = X + Z, where the signal X ∼ U[−1, 1] and noise Z ∼ N(0, 1) are
independent.
(a) Find the function g(y) that minimizes
MSE = E (sgn(X) − g(Y ))2 ,
 

where (
−1 x ≤ 0
sgn(x) =
+1 x > 0.
(b) Plot g(y) vs. y.

Solution: The minimum MSE is achieved when g(Y ) = E(sgn(X) | Y ). We have
Z ∞
g(y) = E(sgn(X) | Y = y) = sgn(x)fX|Y (x|y) dx .
−∞

To find the conditional pdf of X given Y , we use
(1
fY |X (y|x)fX (x) 2 −1 ≤ x ≤ 1
fX|Y (x|y) = , where fX (x) =
fY (y) 0 otherwise.

2