Section $2$: Product Experiment Analysis

At RummyCircle, we are in a constant pursuit of making our gaming platform more engaging and much

more fun. When a player falls in love with the gaming experience we provide, he$/$she wants more of it$.$

And the real fun comes with Cash Gaming.

You can start playing cash games by depositing money in your RummyCircle Wallet. You can start

playing with as small an amount as Rs$.25.$

To enhance the gaming experience, the Product Team keeps coming up with innovative and amazing

Product Features which needs to be tested before going live for all the players.

Here is how we setup our tests:

$1.$ Define the segments on which the feature needs to be tested

$2.$ Split the segments randomly into two groups. We call them Control & Test Groups

$3.$ Implement the new feature for the Test Group while maintaining the status quo for the Control

Once the Test is run for a desired period, we look at the data to decide eventually which group

performed better. If the Test Group performs better, then we go ahead with implementing the feature

for all players of the chosen segments.

In one of these tests, we selected one segment and split them into two groups: Control & Test. Players

from the Test Group were exposed to the new feature while the players in the Control Group were not.

Attached with this is the data file. Based on the data, please answer the questions below:

$1.$ Please create appropriate visualizations to show the relative performance of the Test Path

compared to the Control on the following metrics:

a$.$ LTP

b$.$ Conversion Rate

c$.$ Fun Games

d$.$ Cash Games

$2.$ Basis the data provided, would you recommend going with the new feature? If yes, then why.

And if no$,$ then why not?

$3.$ Bonus Question: Please use appropriate statistical Tests to establish significance of the

differences between Control and Test Path on the four metrics listed in $1$ and state the results.

Definitions

User ID

A numerical ID assigned to each user on their registration.

Fun Games

Games played with fun chips in your account. These chips are provided free of cost to the players so that

they get a first hand experience of Rummy without having to worry about cost.

Cash Games

Games played by wagering money. To play a cash game, you have to deposit some cash first.

Paid$/$Unpaid User

If the user has deposited some cash atleast once, he$/$she will be called a Paid User $($irrespective of

whether this user plays a cash game or not after depositing cash$),$ else if they have played a Fun

Game, they will be called Fun Players or Unpaid User, else if they have not played any cash or fun

games, they shall simply be called Users.

Conversion

We define conversion when a user plays his first cash Game after depositing Cash. Base for

Calculating Conversion Rate is the set of all registered users.

LTP

Defined as $$Lead to Paid ratio$.$ LTP $=$ Count of Paid Users$/$Count of Total Users

Revenue

A part of a players$$ winning in Cash Game which the Platform charges as a commission from the

winner in a game is called Revenue. We do not charge any commission from a player who lost a

game.