Background

In digital advertising, Click$-$Through Rate $($CTR$)$ is a critical metric that measures the effectiveness of an advertisement. It is calculated as the ratio of users who click on an ad to the number of users who view the ad$.$ A higher CTR indicates more successful engagement with the audience, which can lead to increased conversions and revenue. From time$-$to$-$time advertisers experiment with various elements$/$targeting of an ad to optimise the ROI.

Scenario

Imagine an innovative digital advertising agency, AdMasters Inc., that specializes in maximizing click$-$through rates $($CTR$)$ for their clients' advertisements. One of their clients has identified four key tunable elements in their ads: Age, City, Gender, and Mobile Operating System $($OS$).$ These elements significantly influence user engagement and conversion rates. The client is keen to optimize their CTR while minimizing resource expenditure.

Objective

Optimize the CTR of digital ads by employing Multi Arm Bandit algorithms. System should dynamically and efficiently allocate ad displays to maximize overall CTR$.$

Dataset

The dataset for Ads contains $4$ unique features$/$characteristics$.$

Age $($Range: $25$:$50)$

City $($Possible Values: 'New York', 'Los Angeles', 'Chicago','Houston', 'Phoenix'$)$

Gender $($Possible Values: 'Male', 'Female'$)$

OS: $($Possible Values: 'iOS', 'Android', 'Other'$)$

Environment Details

Arms: Each arm represents a different ad from the dataset.

Reward Function:

Probability of a Male clicking on an Ad $->0\mathrm{.}7($randomly generated$)$

Probability of a Female clicking on an Ad $->0\mathrm{.}6($randomly generated$)$

Once probabilities are assigned to all the values, create a final reward $($clicked or not clicked binary outcome$)$ based on the assumed probabilities in step $1($by combining the probabilities of each feature value present in that ad$)$

Assumptions

Assume alpha $=$ beta $=1$ for cold start

Explore Percentage $=10\%$

Run the simulation for min $1000$ iterations

Requirements and Deliverables:

Implement the Multi$-$Arm Bandit Problem for the given above scenario for all the below mentioned policy methods.

Initialize constants

Design a CTR Environment $(1$M$)$

Using Random Policy $(0\mathrm{.}5$M$)$

Using Greedy Policy $(0\mathrm{.}5$M$)$

Using Epsilon$-$Greedy Policy $(0\mathrm{.}5$M$)$

Using UCB $(0\mathrm{.}5$M$)$

Plot CTR distribution for all the appraoches as a spearate graph $(0\mathrm{.}5$M$)$

Changing Exploration Percentage $(1$M$)$

Conclusion $(0\mathrm{.}5$M$)$