Shivani Chauhan (1229601401) Homework-1 CSE575 Statistical Machine Learning
Statistical Machine Learning
CSE575 Fall 2023
Shivani Chauhan
1229601401
---------------------------------------------------------------------------------------------------------------------
Homework-1
All the questions are answered in the same sequence.
1. Probability, MLE and PAC
(A) Suppose that � and � are independent events, and �(�) > 0, �(�) = 0.5. What is
the value of �(�|� ) ?
Sol. Given that X and Y are independent events, we can write:
P(XY) = P(X)*P(Y)
Also, P(Y) > 0 and P(X) = 0.5
P ( XY ) P ( X )∗P (Y )
Using the conditional probability, P(X|Y) = = = 0.5
P (Y ) P (Y )
(B) Suppose that X and Y are disjoint events (i.e. �(�, �) = 0) and �(�) > 0. What is the
value of �(�|� ) ?
Sol. It is stated in the question that X and Y are disjoint events and P(Y) > 0
For disjoint events, we have P(X,Y) = 0
Hence, value of P(X|Y) can be calculated as:
P( XY )
P(X|Y) = =0
P(Y )
(C) Suppose that we have two coins C1 and C2. The probability of C1 having head is 0.6,
and the probability of C2 having head is 0.4. In each test, we toss both coins, and read
the faces of C1 and C2 (note that we read C2 after reading C1). For example, if the
toss resulted in C1 head up and C2 tail up, we will record the result as HT. Suppose
we perform the test 4 times. What is the probability for us to observe the following
result?
HT, HT, TT, TT?
Sol. The probability of C1 having a head is: P(C1H) = 0.6
1
, Shivani Chauhan (1229601401) Homework-1 CSE575 Statistical Machine Learning
The probability of C1 having a tail is: P(C1T) = 0.4
Similarly, the probability of C2 having a head is: P(C2H) = 0.4
The probability of C2 having a tail is: P(C2T) = 0.6
Now, the probability of observing the result HT, HT, TT, TT is calculated as below,
i) Probability of getting HT in one toss = P(C1H)*P(C2T)
= (0.6) * (0.6)
= 0.36
ii) Probability of getting TT in one toss = P(C1T)*P(C2T)
= (0.4) * (0.6)
= 0.24
Using the above data, the probability of observing HT, HT, TT, TT is,
(0.36)*(0.36)*(0.24)*(0.24) = 0.0074
(D) You are given a coin and are asked to toss as many times as you wish to decide the
probability of having heads-up for a toss of the coin. You tossed the coin 20 times,
and observed 15 heads and 5 tails. What is your best estimate of the probability � of
having heads-up?
Sol. The best estimate or MLE estimation of the probability of having heads up can be
given by,
Number of heads
(Probability of heads up) =
Totla number of tosses
15
= = 0.75
20
The likelihood is,
1−¿
P(X | ) = ¿
¿
15
❑ ∗¿
By increasing the number of times the coin is tossed, the estimate of probability would
become more accurate.
(E) If you want to be at least 99% sure that the difference between your estimated value
of θ and the true probability of the coin having heads-up is no more than 0.1, how
2
Statistical Machine Learning
CSE575 Fall 2023
Shivani Chauhan
1229601401
---------------------------------------------------------------------------------------------------------------------
Homework-1
All the questions are answered in the same sequence.
1. Probability, MLE and PAC
(A) Suppose that � and � are independent events, and �(�) > 0, �(�) = 0.5. What is
the value of �(�|� ) ?
Sol. Given that X and Y are independent events, we can write:
P(XY) = P(X)*P(Y)
Also, P(Y) > 0 and P(X) = 0.5
P ( XY ) P ( X )∗P (Y )
Using the conditional probability, P(X|Y) = = = 0.5
P (Y ) P (Y )
(B) Suppose that X and Y are disjoint events (i.e. �(�, �) = 0) and �(�) > 0. What is the
value of �(�|� ) ?
Sol. It is stated in the question that X and Y are disjoint events and P(Y) > 0
For disjoint events, we have P(X,Y) = 0
Hence, value of P(X|Y) can be calculated as:
P( XY )
P(X|Y) = =0
P(Y )
(C) Suppose that we have two coins C1 and C2. The probability of C1 having head is 0.6,
and the probability of C2 having head is 0.4. In each test, we toss both coins, and read
the faces of C1 and C2 (note that we read C2 after reading C1). For example, if the
toss resulted in C1 head up and C2 tail up, we will record the result as HT. Suppose
we perform the test 4 times. What is the probability for us to observe the following
result?
HT, HT, TT, TT?
Sol. The probability of C1 having a head is: P(C1H) = 0.6
1
, Shivani Chauhan (1229601401) Homework-1 CSE575 Statistical Machine Learning
The probability of C1 having a tail is: P(C1T) = 0.4
Similarly, the probability of C2 having a head is: P(C2H) = 0.4
The probability of C2 having a tail is: P(C2T) = 0.6
Now, the probability of observing the result HT, HT, TT, TT is calculated as below,
i) Probability of getting HT in one toss = P(C1H)*P(C2T)
= (0.6) * (0.6)
= 0.36
ii) Probability of getting TT in one toss = P(C1T)*P(C2T)
= (0.4) * (0.6)
= 0.24
Using the above data, the probability of observing HT, HT, TT, TT is,
(0.36)*(0.36)*(0.24)*(0.24) = 0.0074
(D) You are given a coin and are asked to toss as many times as you wish to decide the
probability of having heads-up for a toss of the coin. You tossed the coin 20 times,
and observed 15 heads and 5 tails. What is your best estimate of the probability � of
having heads-up?
Sol. The best estimate or MLE estimation of the probability of having heads up can be
given by,
Number of heads
(Probability of heads up) =
Totla number of tosses
15
= = 0.75
20
The likelihood is,
1−¿
P(X | ) = ¿
¿
15
❑ ∗¿
By increasing the number of times the coin is tossed, the estimate of probability would
become more accurate.
(E) If you want to be at least 99% sure that the difference between your estimated value
of θ and the true probability of the coin having heads-up is no more than 0.1, how
2
