$1.$ Multi$-$Arm Bandit Problem $(5$ Marks$)$: Imagine an innovative digital advertising agency, AdMasters Inc., that specializes in maximizing click$-$through rates $($CTR$)$ for their clients' advertisements. One of their clients has identified four key tunable elements in their ads: Age, City, Gender, and Mobile Operating System $($OS$).$ These elements significantly influence user engagement and conversion rates. The client is keen to optimize their CTR while minimizing resource expenditure.