Multi$-$Arm Bandit Problem:

Background

In digital advertising, Click$-$Through Rate $($CTR$)$ is a critical metric that measures the effectiveness of an advertisement. It is calculated as the ratio of users who click on an ad to the number of users who view the ad$.$ A higher CTR indicates more successful engagement with the audience, which can lead to increased conversions and revenue. From time$-$to$-$time advertisers experiment with various elements$/$targeting of an ad to optimise the ROI.

Scenario

Imagine an innovative digital advertising agency, AdMasters Inc., that specializes in maximizing click$-$through rates $($CTR$)$ for their clients' advertisements. One of their clients has identified four key tunable elements in their ads: Age, City, Gender, and Mobile Operating System $($OS$).$ These elements significantly influence user engagement and conversion rates. The client is keen to optimize their CTR while minimizing resource expenditure.

Objective

Optimize the CTR of digital ads by employing Multi Arm Bandit algorithms. System should dynamically and efficiently allocate ad displays to maximize overall CTR$.$

Dataset

The dataset for Ads contains $4$ unique features$/$characteristics$.$

Age $($Range: $25$:$50)$

City $($Possible Values: 'New York', 'Los Angeles', 'Chicago','Houston', 'Phoenix'$)$

Gender $($Possible Values: 'Male', 'Female'$)$

OS: $($Possible Values: 'iOS', 'Android', 'Other'$)$

Attached the SS for reference of dataset AD$\_$Click.csv$.$

Requirements and Deliverables:

Implement the Multi$-$Arm Bandit Problem for the given above scenario for all the below mentioned policy methods.

Initialize constants

# Constants

epsilon $=0\mathrm{.}1$

$$

# Initialize value function and policy

Load Dataset

# Python Code for Dataset loading and print dataset statistics

#$-----$write your python code below this line$---------$

$$

Design a CTR Environment $(1$M$)$

# Code for Dataset loading and print dataset statistics along with reward function

#$-----$write your python code below this line$---------$

class CTREnvironment:

$$

Using Random Policy $(0\mathrm{.}5$M$)$

Print all the iterations with random policy selected for the given Ad$.($Mandatory$)$

# run the environment with an agent that is guided by a random policy

#$-----$write your code below this line$---------$

Using Greedy Policy $(0\mathrm{.}5$M$)$

Print all the iterations with random policy selected for the given Ad$.($Mandatory$)$

# run the environment with an agent that is guided by a greedy policy

#$-----$write your code below this line$---------$

Using Epsilon$-$Greedy Policy $(0\mathrm{.}5$M$)$

Print all the iterations with random policy selected for the given Ad$.($Mandatory$)$

# run the environment with an agent that is guided by a epsilon$-$greedy policy

#$-----$write your code below this line$---------$

Using UCB $(0\mathrm{.}5$M$)$

Print all the iterations with random policy selected for the given Ad$.($Mandatory$)$

# run the environment with an agent that is guided by a UCB

#$-----$write your code below this line$---------$

Plot CTR distribution for all the appraoches as a spearate graph $(0\mathrm{.}5$M$)$

#$-----$write your code below this line$---------$

Changing Exploration Percentage $(1$M$)$

How does changing the exploration percentage $($EXPLORE$\_$PERCENTAGE$)$ affect the performance of the algorithm? Test with different values $($e$.$g$.0\mathrm{.}15$ and $0\mathrm{.}2)$ and discuss the results.

#Implement with any MAB algorithm

#Try with different EXPLORE$\_$PERCENTAGE

#Different value of alpha

Requirements and Deliverables:

Implement the Multi$-$Arm Bandit Problem for the given above scenario for all the below mentioned policy methods.

Initialize constantsepsilon $=0\mathrm{.}1$

Load Dataset

In $[]$: # Code for Dataset loading and print dataset statistics

#$-----$write your code below this line$--\_------$

Design a CTR Environment $(1$M$)$

In $[]$: # Code for Dataset loading and print dataset statistics along with reward function

#$-----$write your code below this line$----------$

class CTREnvironment: Using Random Policy $(0\mathrm{.}5$M$)$

Print all the iterations with random policy selected for the given Ad$.($Mandatory$)$

In $[]$: # run the environment with an agent that is guided by a random policy

#$-----$write your code below this line$---------$

Using Greedy Policy $(0\mathrm{.}5$M$)$

Print all the iterations with random policy selected for the given Ad$.($Mandatory$)$

In $[]$: # run the environment with an agent that is guided by a greedy policy

#$-----$write your code below this line$---------$

Using Epsilon$-$Greedy Policy $(0\mathrm{.}5$M$)$

Print all the iterations with random policy selected for the given Ad$.($Mandatory$)$

In $[]$: # run the environment with an agent that is guided by a epsilon$-$greedy policy

#$-----$write your code below this line$---------$

Using UCB $(0\mathrm{.}5$M$)$

Print all the iterations with random policy selected for the given Ad$.($Mandatory$)$

In $[]$: # run the environment with an agent that is guided by a UCB

#$-----$write your code below this line$---------$ Plot CTR distribution for all the appraoches as a spearate graph $($ Conclusion $(0\mathrm{.}5$M$)$

Conclude your assignment in $250$ wrods by discussing the best approach for maximizing the CTR using random, greedy, epsilon$-$greedy and UCB.