$16\mathrm{.}1$ University Rankings. The dataset miba:: Universities on American College and Uni$-$verity Rankings contains information on $1302$ American colleges and universities ofering an undergraduate program. For each university, there are $17$ measurements, including continuous variables $($such as tuition and graduation rate$)$ and categorical variables $($such as location by state and whether it is a private or public school$).$

Note that many records are missing some measurements. Our first goal is to estimate these missing values from "similar" records. This will be done by clustering the complete records and then finding the closest cluster for each of the partial records.

The missing values will be imputed from the information in that cluster.

a$.$ Remove all records with missing measurements from the dataset.

b$.$ For all the continuous variables, run hierarchical clustering using complete linkage and Euclidean distance. Make sure to normalize the variables. From the dendro$-$gram: how many clusters seem reasonable for describing these data?

c$.$ Compare the summary statistics for each cluster, and describe each cluster in this context $($e$.$g$.,$ "Universities with high tuition, low acceptance rate..."$).$ Hint: To obtain cluster statistics for hierarchical clustering, use the aggregate function.

d$.$ Use the categorical variables that were not used in the analysis $($State and Private$/$Public$)$ to characterize the different clusters. Is there any relationship between the clusters and the categorical information?

e$.$ What other external information can explain the contents of some or all of these clusters?

f$.$ Consider Tufts University, which is missing some information. Compute the Euclidean distance of this record from each of the clusters that you found above $($using only the variables that you have$).$ Which cluster is it closest to$?$ Impute the missing values for Tufts by taking the average of the cluster on those variables.

# a

data m$7$ba::Universities

# Remove records with missing measurements

complete$\_$data na$.$omit$($data$)$

# b

continuous$\_$vars complete$\_$data$[,$ c $($

$"$X$..$app$7$i$..$rec.d$","$X$..$app$1..$accepted", $"$X$..$new.stud..enro$7$led",

$"$X$..$ new. stud. $.$from. top$\mathrm{.}10.","$X$..$new.stud. $.$from. top$\mathrm{.}25.","$X$..$FT$.$ undergrad",

$"$X$..$PT$.$ undergrad", $"$in$.$state.tuition", "out.of$.$state.tuition", "room",

"board", "add. $.$fees", "estim. $.$book. costs", "estim. persona$7..",$

$"$x$..$fac..w$.$PHD$",$ "stud..fac..ratio", "Graduation.rate"

$]$

# Normalize the continuous variables

normalized$\_$continuous scale$($continuous$\_$vars$)$

# Perform hierarchical clustering

hclust$\_$result hclust$($dist$($normalized$\_$continuous$),$ method $=$ "complete"$)$

# Plot dendrogram

plot $($hclust$\_$result$)$

# $C$

num$\_$clusters

cluster$\_$assignments cutree$($hclust$\_$result, $k=$ num$\_$clusters$)$

# Combine cluster assignments with data

clustered$\_$data cbind$($complete$\_$data, Cluster $=$ cluster$\_$assignments$)$

# Compute summary statistics for each cluster

cluster$\_$stats aggregate$($continuous$\_$vars, by $=$ list$($Cluster $=$ cluster$\_$assignments$),$ FUN $=$ summary$)$

# d

categorical$\_$vars complete$\_$data$[,$ c$("$State$",$ "Public.$\mathrm{.}1...$Private.. $2.")]$

clustered$\_$categorical cbind$($categorical$\_$vars, Cluster $=$ cluster$\_$assignments$)$

# $f$

$($Top Level$)$

# $f$

tufts complete$\_$data$[$complete$\_$data$col$7$ege. Name $==$ "Tufts University", $]$ tufts$\_$continuous tufts $[,$ c

$"$X$..$ app$71...$rec.d$","$X$..$ app$7..$accepted", $"$X$..$new.stud.. enro$77$ed$",$

$"$X$.$ new. stud. $.$ from. top$\mathrm{.}10.","$X$..$ new.stud..from.top$\mathrm{.}25.","$X$..$FT$.$undergrad",

$"$X$..$PT$.$ undergrad", $"$in$.$state.tuition", "out. of$.$state.tuition", "room", "board", "add..fees", "estim. book.costs", "estim..persona$7..",$

$"$X$..$fac..w$.$PHD$",$ "stud..fac..ratio", "Graduation. rate"

$]$

tufts$\_$distances sapply$(1$:num$\_$clusters, function$($i$)\{$

cluster$\_$mean colMeans $($continuous$\_$vars $[$cluster$\_$assignments $=i,$