Optimize the CTR of digital ads by employing Multi Arm Bandit algorithms. System should dynamically and efficiently allocate ad displays to maximize overall CTR$.$

Dataset

The dataset for Ads contains $4$ unique features$/$characteristics$.$ with $1000$ rows

Age $($Range: $25$:$50)$

City $($Possible Values: 'New York', 'Los Angeles', 'Chicago','Houston', 'Phoenix'$)$

Gender $($Possible Values: 'Male', 'Female'$)$

OS: $($Possible Values: 'iOS', 'Android', 'Other'$)$

Environment Details

Arms: Each arm represents a different ad from the dataset.

Reward Function:

Probability of a Male clicking on an Ad $->0\mathrm{.}7($randomly generated$)$

Probability of a Female clicking on an Ad $->0\mathrm{.}6($randomly generated$)$

Once probabilities are assigned to all the values, create a final reward $($clicked or not clicked binary outcome$)$ based on the assumed probabilities in step $1($by combining the probabilities of each feature value present in that ad$)$

Assumptions

Assume alpha $=$ beta $=1$ for cold start

Explore Percentage $=10\%$

Run the simulation for min $1000$ iterations

Requirements and Deliverables:

Implement the Multi$-$Arm Bandit Problem for the given above scenario for all the below mentioned policy methods. using random, greedy, ucb and epsilon$-$greedy