Consider the following probability model. For a set of points x1, . , xn, we have k possible generating distributions, f1,.

Consider the following probability model. For a set of points x1, . . . , xn, we have k possible generating distributions, f1,...,fk. (That is, we know each point xi was generated from one of the fj; we just don't know which j.) Let zi = {1, . . . , k} be an indicator variable which indicates that the i'th data point xi was generated from fj if zi = j. Furthermore, we specify that p(zi = j) = πj. Thus, our model specifies the following:

(i)p(xi |zi =j)=fj(xi), (ii)p(zi =j)=πj.

Hint: The key to this problem is applying the basic rules of probability-marginalization, Bayes Rule, the chain rule, and/or conditional independence (not necessarily in that order). Note that some of the answers may follow directly from the definitions given above. You may make the independence assumption: x1, x2, ..., xn are independent of each other.

1. [4 points] Derive the formula for p(xi) in terms of (i) and (ii) above.

2. [7 points] Derive the formula for p(x1, . . . , xn) in terms of (i) and (ii) above.

3. [7 points] Derive the formula for p(zu = v | x1, . . . , xn) in terms of (i) and (ii) above.