$($a$)$ Explain the concepts of Principal Component Analysis $($PCA$)$ and Linear Discriminant Analysis $($LDA$).$ Discuss how each technique can be used for dimensionality reduction.

$($b$)$ Compare PCA and LDA in terms of their objectives, methodologies, and applications.

$($c$)$ Support your explanation with visual illustrations.

$($d$)$ Write the optimization functions for PCA and LDA.

Instructions

$$ Provide clear and concise explanations for each part.

$$ Use appropriate notations and diagrams to support your explanations.

$$ Clearly state any assumptions you make.