Set #3: Applied Bayesian Analysis Instructions Although discussion of exercises are encouraged among students, each assignment must be completed individually and each student is responsible for turning in their own work. Please adhere to the following: Homework should be typed and answered in the order given (problem 1(a)(b)(c)(d) first, problem 2(a)(b)... second, etc...). If typing your homework proves to take longer than the actual homework (especially for those of you unfamiliar with either LaTeX or Math Equation editor in Word), please print neatly. Include in each part of the homework only the answer. This document must be uploaded to Blackboard using the following name for the file: PS1.LastNameFirstInitial.doc or PS1.LastNameFirstInitial.pdf. R code and R output (without mistakes), must be included in the appendix at the END of your homework document (this Appendix, saved as a separate .R file, should be uploaded to Blackboard using the following name for the file: PS1.LastNameFirstInitial.Appendix.R). - Use the following format when labeling your code in the Appendix: ############### PS 1, problem 1(a) ############### ############### PS 1, problem 1(b) ############### ############### PS 1, problem 1(c) ############### No late homework under any circumstances. At the top of each homework, include the following: Name, Drexel ID, Course Number/Section, Assignment # Do not just give a number as an answer. For example, if asked for the probability that a posterior proportion is larger than 0.7, write P r(p > 0.7) = 0.3, say and write comments or explanations if needed. The homework (paper copy WITHOUT the Appendix) must be turned in at the beginning of lecture. 1 Problems 1. Exercise 6.1 in the text 2. Exercise 6.3 in the text 3. A common prior you may come across is the Cauchy distribution, particularly as a starting point for modeling regression coefficients or as a hyperprior (in the form of a 'half-Cauchy') for variance parameters. (a) Plot a standard Cauchy distribution with location 0 and scale 1. Plot it along with a standard Normal distribution and t distributions with various degrees of freedom. Discuss the relationship between these distributions. What aspect of the Cauchy sets it apart from Normal or t distributions? (b) Based on the comparisons made in (a), what makes the standard Cauchy an ideal default (i.e., starting point) prior. Of the distributions in (a), which distribution would you use if you were very confident in your prior information? What about in the case you are very unconfident? (c) Based on (b), it should be clear when the Cauchy is useful over the Normal or t distributions. Since the Cauchy distribution as a prior is more applicable to regression or hierarchical modeling situations, let's instead directly apply the distribution to some data, using Gibbs to approximate our posterior. Before we do that, though, we have to rewrite the Cauchy in a form that exploits its relationship with the other distributions mentioned above. Given the Cauchy distribution p(xi |) = (1+(x1i )2 ) , show R that p(xi |) = p(xi |, i )p(i )di where p(xi |, i ) Normal(, 1/i ), p(i ) Gamma(, ), and = = 12 . HINT: Write the normal distribution in the following form: r exp[ (x )2 ] 2 2 Also note the following equalities: Z ( + 12 ) 1 1 2 exp[( + (x )2 ]d = 1 2 ( + 12 (x )2 )+ 2 2 \u0012 \u0013 1 2 (d) Identify the density for the distriubtion p(x|) (this may be more apparent before inserting the values for and ). (e) Now let's construct a Gibbs sampler for the following model: \u0013 \u0012 2 for i, . . . , n p(xi |i ) Normal , i \u0012 \u0013 1 1 p(i ) Gamma , for i, . . . , n 2 2 1 p(, 2 ) 2 (1) (2) (3) 2 where i is the scaled variance for the Normal distribution and 1 is a noninformative prior placed on and 2 . Write out the 2 complete joint density p(xi |i )p(i )p(, 2 ). HINT: Note the index on . (f) Show that the Gibbs updates below can be obtained from the joint density in (e). \u0013 (xi )2 1 + i Gamma 1, 2 2 2 ! Pn 2 x i i , P2 Normal Pi=1 n i=1 i i=1 i ! P2 2 (x ) n i i 2 IG , i=1 2 2 \u0012 (4) (5) (6) (g) Using the data X provided, construct a Gibbs sampler to approximate the posterior distribution. Begin with a 1000 iteration burn-in and then run the sampler for an additional 1500 iterations. Your output should consist of an N 1500 matrix of i values and two N -vectors for and 2 HINT: To download the data, do the following: 3 X <- unlist(read.csv("https://dl.dropboxusercontent.com/u/907375/ps3_data.csv")) ### check data is correct length(X) # should be 30 round(min(X),2) # should be -7.93 round(max(X),2) # should be 12.11 (h) Calculate the posterior means and 95% credible intervals for and . Plot and interpret appropriate diagnostic figures. (i) Plot the 95% credible intervals for each i against the raw data X and interpret the significance of the figure. 4. Exercise 7.3 in the text 5. Exercise 7.4 in the text 4