DOG. BIG DOG, LITTLE DOG. BIG DOGS AND LITTLE DOGS. BLACK AND WHITE DOGS. HELLO. HELLO. DO YOU LIKE MY HAT? I DO NOT. GOODBYE. GOODBYE. ONE LITTLE DOG GOING IN. THREE BIG DOGS GOING OUT. A RED DOG ON A BLUE TREE. A BIG BLUE DOG ON A RED TREE. A GREEN DOG ON A YELLOW TREE. SOME BIG DOGS AND SOME LITTLE DOGS GOING AROUND IN CARS. A DOG OUT OF A CAR. TWO BIG DOGS GOING UP, ONE LITTLE DOG GOING DOWN. THE GREEN DOG IS UP. THE YELLOW DOG IS DOWN. THE BLUE DOG IS IN. THE RED DOG IS OUT. ONE DOG UP ON A HOUSE. THREE DOGS DOWN IN THE WATER. A GREEN DOG OVER A TREE. A YELLOW DOG UNDER A TREE. TWO DOGS ON A HOUSE IN A BOAT IN THE WATER. A DOG OVER THE WATER. A DOG UNDER THE WATER. HELLO AGAIN. HELLO. DO YOU LIKE MY HAT? I DO NOT LIKE IT. GOODBYE AGAIN. GOODBYE. THE DOGS ARE ALL GOING AROUND AND AROUND AND AROUND. GO AROUND AGAIN. THE SUN IS UP. THE SUN IS YELLOW. THE YELLOW SUN IS OVER THE HOUSE. IT IS HOT OUT HERE IN THE SUN. IT IS NOT HOT HERE UNDER THE HOUSE. NOW IT IS NIGHT. THREE DOGS AT A PARTY ON A BOAT AT NIGHT. DOGS AT WORK. WORK, DOGS. WORK. DOGS AT PLAY. PLAY, DOGS. PLAY. HELLO AGAIN. HELLO. DO YOU LIKE MY HAT? I DO NOT LIKE THAT HAT. GOODBYE AGAIN. GOODBYE. DOGS IN CARS AGAIN, GOING AWAY. GOING AWAY FAST. LOOK AT THOSE DOGS GO. GO, DOGS. GO! STOP, DOGS. STOP! THE LIGHT IS RED NOW. GO, DOGS. GO! THE LIGHT IS GREEN NOW. TWO DOGS AT PLAY, AT PLAY UP ON TOP. GO, DOGS. GO! DO NOT PLAY UP THERE. GO DOWN. NOW IT IS NIGHT. NIGHT IS NOT A TIME FOR PLAY. IT IS A TIME FOR SLEEP. THE DOGS GO TO SLEEP. THEY WILL SLEEP ALL NIGHT. NOW IT IS DAY. THE SUN IS UP. NOW IS THE TIME FOR ALL DOGS TO GET UP. GET UP. IT IS DAY. TIME TO GET GOING. GO, DOGS. GO! THERE THEY GO. LOOK AT THOSE DOGS GO. WHY ARE THEY GOING FAST IN THOSE CARS? WHAT ARE THEY GOING TO DO? WHERE ARE THOSE DOGS GOING? LOOK WHERE THEY ARE GOING. THEY ARE ALL GOING TO THAT BIG TREE OVER THERE. NOW THE CARS STOP AND ALL THE DOGS GET OUT. AND NOW LOOK WHERE ALL THOSE DOGS ARE GOING. TO THE TREE. TO THE TREE. UP THE TREE. UP THE TREE. UP THEY GO TO THE TOP OF THE TREE. WHY? WILL THEY WORK THERE? WILL THEY PLAY THERE? WHAT IS UP THERE ON TOP OF THAT TREE? A BIG DOG PARTY! BIG DOGS, LITTLE DOGS, RED DOGS, BLUE DOGS, YELLOW DOGS, GREEN DOGS, BLACK DOGS, AND WHITE DOGS ARE ALL AT A DOG PARTY. WHAT A DOG PARTY! HELLO AGAIN AND NOW DO YOU LIKE MY HAT? I DO! WHAT A HAT! I LIKE IT. I LIKE THAT PARTY HAT. GOODBYE. I AM SAM. I AM SAM. SAM I AM. THAT SAM I AM! THAT SAM I AM! I DO NOT LIKE THAT SAM I AM! DO WOULD YOU LIKE GREEN EGGS AND HAM? I DO NOT LIKE THEM, SAM I AM. I DO NOT LIKE GREEN EGGS AND HAM. WOULD YOU LIKE THEM HERE OR THERE? I WOULD NOT LIKE THEM HERE OR THERE. I WOULD NOT LIKE THEM ANYWHERE. I DO NOT LIKE GREEN EGGS AND HAM. I DO NOT LIKE THEM, SAM I AM. WOULD YOU LIKE THEM IN A HOUSE? WOULD YOU LIKE THEN WITH A MOUSE? I DO NOT LIKE THEM IN A HOUSE. I DO NOT LIKE THEM WITH A MOUSE. I DO NOT LIKE THEM HERE OR THERE. I DO NOT LIKE THEM ANYWHERE. I DO NOT LIKE GREEN EGGS AND HAM. I DO NOT LIKE THEM, SAM I AM. WOULD YOU EAT THEM IN A BOX? WOULD YOU EAT THEM WITH A FOX? NOT IN A BOX. NOT WITH A FOX. NOT IN A HOUSE. NOT WITH A MOUSE. I WOULD NOT EAT THEM HERE OR THERE. I WOULD NOT EAT THEM ANYWHERE. I WOULD NOT EAT GREEN EGGS AND HAM. I DO NOT LIKE THEM, SAM I AM. WOULD YOU? COULD YOU? IN A CAR? EAT THEM! EAT THEM! HERE THEY ARE. I WOULD NOT, COULD NOT, IN A CAR. YOU MAY LIKE THEM. YOU WILL SEE. YOU MAY LIKE THEM IN A TREE! I WOULD NOT, COULD NOT IN A TREE. NOT IN A CAR! YOU LET ME BE. I DO NOT LIKE THEM IN A BOX. I DO NOT LIKE THEM WITH A FOX. I DO NOT LIKE THEM IN A HOUSE. I DO NOT LIKE THEM WITH A MOUSE. I DO NOT LIKE THEM HERE OR THERE. I DO NOT LIKE THEM ANYWHERE. I DO NOT LIKE GREEN EGGS AND HAM. I DO NOT LIKE THEM, SAM I AM. A TRAIN! A TRAIN! A TRAIN! A TRAIN! COULD YOU, WOULD YOU ON A TRAIN? NOT ON TRAIN! NOT IN A TREE! NOT IN A CAR! SAM! LET ME BE! I WOULD NOT, COULD NOT, IN A BOX. I WOULD NOT, COULD NOT, WITH A FOX. I WILL NOT EAT THEM IN A HOUSE. I WILL NOT EAT THEM HERE OR THERE. I WILL NOT EAT THEM ANYWHERE. I DO NOT EAT GREEN EGGS AND HAM. I DO NOT LIKE THEM, SAM I AM. SAY! IN THE DARK? HERE IN THE DARK! WOULD YOU, COULD YOU, IN THE DARK? I WOULD NOT, COULD NOT, IN THE DARK. WOULD YOU COULD YOU IN THE RAIN? I WOULD NOT, COULD NOT IN THE RAIN. NOT IN THE DARK. NOT ON A TRAIN. NOT IN A CAR. NOT IN A TREE. I DO NOT LIKE THEM, SAM, YOU SEE. NOT IN A HOUSE. NOT IN A BOX. NOT WITH A MOUSE. NOT WITH A FOX. I WILL NOT EAT THEM HERE OR THERE. I DO NOT LIKE THEM ANYWHERE! YOU DO NOT LIKE GREEN EGGS AND HAM? I DO NOT LIKE THEM, SAM I AM. COULD YOU, WOULD YOU, WITH A GOAT? I WOULD NOT, COULD NOT WITH A GOAT! WOULD YOU, COULD YOU, ON A BOAT? I COULD NOT, WOULD NOT, ON A BOAT. I WILL NOT, WILL NOT, WITH A GOAT. I WILL NOT EAT THEM IN THE RAIN. NOT IN THE DARK! NOT IN A TREE! NOT IN A CAR! YOU LET ME BE! I DO NOT LIKE THEM IN A BOX. I DO NOT LIKE THEM WITH A FOX. I WILL NOT EAT THEM IN A HOUSE. I DO NOT LIKE THEM WITH A MOUSE. I DO NOT LIKE THEM HERE OR THERE. I DO NOT LIKE THEM ANYWHERE! I DO NOT LIKE GREEN EGGS AND HAM! I DO NOT LIKE THEM, SAM I AM. YOU DO NOT LIKE THEM. SO YOU SAY. TRY THEM! TRY THEM! AND YOU MAY. TRY THEM AND YOU MAY, I SAY. SAM! IF YOU LET ME BE, I WILL TRY THEM. YOU WILL SEE. SAY! I LIKE GREEN EGGS AND HAM! I DO! I LIKE THEM, SAM I AM! AND I WOULD EAT THEM IN A BOAT. AND I WOULD EAT THEM WITH A GOAT. AND I WILL EAT THEM, IN THE RAIN. AND IN THE DARK. AND ON A TRAIN. AND IN A CAR. AND IN A TREE. THEY ARE SO GOOD, SO GOOD, YOU SEE! SO I WILL EAT THEM IN A BOX. AND I WILL EAT THEM WITH A FOX. AND I WILL EAT THEM IN A HOUSE. AND I WILL EAT THEM WITH A MOUSE. AND I WILL EAT THEM HERE AND THERE. SAY! I WILL EAT THEM ANYWHERE! I DO SO LIKE GREEN EGGS AND HAM! THANK YOU! THANK YOU, SAM I AM. The assignment is to code up the Gibbs sampler algorithm as described in the document "Gibbs Sampling for the Uninitiated" The psuedo-code in section 2.5.4 will be very helpful. For the assignment, I have written the commands that create the corpus of "documents." I took the text from two popular children's books: *Green Eggs and Ham* by Dr. Seuss, and *Go Dog, Go!* by P.D. Eastman. (They are also popular with my daughter.) I took the raw text of the book, and chopped each book into about 10 chunks. I'm calling each of these text chunks a "document." The commands make use of the library `tm` (text mining) and `SnowballC` (truncating words to their roots). If you code it correctly, the algorithm should be able to do a decent job of classifying the documents despite the data being completely unlabeled. At the end, print out the vector `l` which shows the classifications. Also print out the vectors theta_0 and theta_1. Keep in mind, these are just random draws from the posterior distribution, and will not necessarily be the values that maximize the posterior probability. They should, however, be drawn from regions of fairly high probability and reflect values that are somewhat close to the true values. Include any plots or other output if you feel they help show the success of the algorithm. Please do not print out the results of each iteration or do something that produces thousands of lines of unnecessary output. ```{r} ## the following code is adapted from the page: ## https://www.r-bloggers.com/text-mining-the-complete-works-of-williamshakespeare/ library(tm) library(SnowballC) setwd("C:/Users/jin1/Desktop/spring") geah <- readLines("geah.txt") # text of Green Eggs and Ham by Dr. Seuss gdg <- readLines("gdg.txt") # text of Go Dog Go by P.D. Eastman corpus <- c(geah, gdg) doc.vec <- VectorSource(corpus) doc.corpus <- Corpus(doc.vec) # summary(doc.corpus) doc.corpus <- tm_map(doc.corpus, tolower) doc.corpus <- tm_map(doc.corpus, removePunctuation) doc.corpus <- tm_map(doc.corpus, removeNumbers) doc.corpus <- tm_map(doc.corpus, removeWords, stopwords("english")) # removes very common English words doc.corpus <- tm_map(doc.corpus, stemDocument) # stems words so that words like running, runs, runner just become run doc.corpus <- tm_map(doc.corpus, stripWhitespace) inspect(doc.corpus[4]) # resulting text for one 'document' inspect(doc.corpus[15]) DTM <- DocumentTermMatrix(doc.corpus) # creates a matrix of the words and their frequencies in each document DTM inspect(DTM[1:10,1:10]) word_counts <- as.matrix(DTM) # You can now use this as it shows the frequency of each word dim(word_counts) ## our corpus has 19 documents, and a total of 59 words head(word_counts) ## a visual inspection already shows some patterns in the word usage tail(word_counts) View(word_counts) # initial parameters V = dim(word_counts)[2] # number of words in the vocabulary #colums N = dim(word_counts)[1] # number of documents in the corpus gamma_pi_1 = 5 # hyper parameter for pi gamma_pi_0 = 5 # hyper parameter for pi gamma_theta = rep(2, V) # vector of hyper parameters for the vector theta library(MCMCpack) set.seed(1) # randomly initialize the label assignments pi <- rbeta(1, gamma_pi_0, gamma_pi_1) l <- rbinom(N, 1, pi) theta_0 <- rdirichlet(1, gamma_theta) theta_1 <- rdirichlet(1, gamma_theta) ``` ```{r} ## Write the code for the gibbs sampler here ## it might take a long time to run ``` ```