Content
Introduction
A/B Experiments are a research method that allows you to compare the performance of different variants by sending it to different audience groups to see which of them is more effective.
Hawk allows you to randomly allocate different sets of users using Variants. Each variant determines the percentage of the audience assigned to the experiment.
In Hawk, you can do A/B experiment with the following entities:
Payment Catalogs & Shops
Campaign & App Links
Events
Piggy Banks
Table of Contents
Creating A/B Experiments
The first step to perform A/B Experiments in Hawk is creating your experiment, which will serve as a container for the entities to be used in the experiment. To do so, follow the steps below:
Navigate to A/B Testing > Experiments. In this page you’ll be able to see a list of the existing A/B Tests, copy them with the same settings or edit them.
Click [+Add Experiment] on the upper right hand corner
Fill in the following fields:
About Priority in A/B Experiments
In Hawk, Events take priority over A/B Experiments
- Then, select whether you want your Experiment to apply to all players or to specific segments of players.
The segments use the operator OR to function, so the experiment will apply to players in either of one of the segments you use. If a player is assigned to 2 different segments, Hawk will select first the segment with higher priority.
- Once that’s done, create your variants for the Experiment by clicking on [+ Add Variant] and assign weights to them.
Deleting Experiment Variants
Please note that experiment variants can be only deleted during the experiment creation. If the experiment has already been created, variants cannot be deleted.
- Click [Save] on the bottom left corner of the screen to end the configuration.
Once your experiment is created, it’s time to add your entities to it.
Adding Entities to Experiments
Make sure the entity you want to test has been already created, as entities can be only added to experiments while editing the entity, not during the creation stage of the entity. In Hawk, an experiment must contain entities to allocate players. If the entities are set to Test (or segments of entities are Test), players without Test access will not be allocated to the experiment.
Navigate to the entity you’d like to add to the experiment, and click the [Edit] button.
Click [Add to Experiment], and select your experiment. A new blue banner will appear with all the information about the experiment status.
A new tab will appear with the experiment and its variants. Control Variant will always match the base configuration of your entity, but the rest of variants of the experiment are editable.
Edit the settings in the variants for your experiment, and when you’re done, click [Save].
Activating your Experiment
The last step of the configuration of A/B Experiments in Hawk is activating your experiment so that it can be tested with players.
Navigate back to A/B Experiments and click the [Edit] button. You’ll see that the Experiment now shows which entities are being used and new information is displayed inside variants.
If you’re happy with the configuration, click [Save and Change Status], and then Schedule/Launch. Schedule will appear if the experiment is set to start in a date in the future, and Launch will appear if the experiment is set to start in a date/time in the past.
If you just simply click [Save], the experiment’s status will remain in Draft. To make it Active, click [Save and Change Status].
Analysing your A/B Experiment
Your A/B Experiments can be analysed in Lab. For further info, check the Lab User Guides.
🔍 Frequently Asked Questions
If a player belongs to segment X and is allocated to an experiment with a catalog (or different entity) served to segment X, will the player still receive the served catalog from the experiment if they change to segment Y?
Does an experiment allocate based on the specified segment in the experiment or entity? For instance, if an A/B experiment is set for all players but the entity added to the experiment is only for a specific segment (like old players), are all players allocated there or only old players?
What happens, if experiment ends before the event/campaign ends?
My experiment is running between 01.12-31.12. On 07.12, I changed the the weight of Variant A from 5 to 0. What will happen with players that were already in Variant A?
Chunks 1
**Introduction**
A/B Experiments are a research method that allows you to compare the performance of different variants by sending it to different audience groups to see which of them is more effective.
Hawk allows you to randomly allocate different sets of users using Variants. Each variant determines the percentage of the audience assigned to the experiment.
In Hawk, you can do A/B experiment with the following entities:
- Payment Catalogs & Shops
- Campaign & App Links
- Events
- Piggy Banks
**Table of Contents**
* * *
## Creating A/B Experiments
The first step to perform A/B Experiments in Hawk is creating your experiment, which will serve as a container for the entities to be used in the experiment. To do so, follow the steps below:
1. Navigate to **A/B Testing \> Experiments.** In this page you’ll be able to see a list of the existing A/B Tests, copy them with the same settings or edit them.
1. Click **[+Add Experiment]** on the upper right hand corner
2. Fill in the following fields:
**About Priority in A/B Experiments**
In Hawk, Events take priority over A/B Experiments
1. Then, select whether you want your Experiment to apply to all players or to specific segments of players.
The segments use the operator **OR** to function, so the experiment will apply to players in either of one of the segments you use. If a player is assigned to 2 different segments, **Hawk will select first the segment with higher priority**.
1. Once that’s done, create your variants for the Experiment by clicking on **[+ Add Variant]** and assign weights to them.
**Deleting Experiment Variants**
Please note that experiment variants can be only deleted during the experiment creation. If the experiment has already been created, variants **cannot** be deleted.
1. Click **[Save]** on the bottom left corner of the screen to end the configuration.
Once your experiment is created, it’s time to add your entities to it.
## Adding Entities to Experiments
Make sure the entity you want to test has been already created, as entities can be only added to experiments while editing the entity, not during the creation stage of the entity. In Hawk, an experiment must contain entities to allocate players. If the entities are set to _Test_ (or segments of entities are _Test_), players without _Test_ access will not be allocated to the experiment.
1. Navigate to the entity you’d like to add to the experiment, and click the **[Edit]** button.
2. Click **[Add to Experiment]**, and select your experiment. A new blue banner will appear with all the information about the experiment status.
1. A new tab will appear with the experiment and its variants. Control Variant will always match the base configuration of your entity, but the rest of variants of the experiment are editable.
2. Edit the settings in the variants for your experiment, and when you’re done, click **[Save].**
## Activating your Experiment
The last step of the configuration of A/B Experiments in Hawk is activating your experiment so that it can be tested with players.
1. Navigate back to A/B Experiments and click the **[Edit]** button. You’ll see that the Experiment now shows which entities are being used and new information is displayed inside variants.
2. If you’re happy with the configuration, click **[Save and Change Status]**, and then **Schedule/Launch**. Schedule will appear if the experiment is set to start in a date in the future, and Launch will appear if the experiment is set to start in a date/time in the past.
If you just simply click **[Save]**, the experiment’s status will remain in Draft. To make it Active, click **[Save and Change Status].**
#### Analysing your A/B Experiment
Your A/B Experiments can be analysed in Lab. For further info, check the [Lab User Guides](/wiki/spaces/DBF/pages/2979627271/LAB+LookerAB).
* * *
## 🔍 Frequently Asked Questions
- **If a player belongs to segment X and is allocated to an experiment with a catalog (or different entity) served to segment X, will the player still receive the served catalog from the experiment if they change to segment Y?**
- **Does an experiment allocate based on the specified segment in the experiment or entity? For instance, if an A/B experiment is set for all players but the entity added to the experiment is only for a specific segment (like old players), are all players allocated there or only old players?**
- **What happens, if experiment ends before the event/campaign ends?**
- **My experiment is running between 01.12-31.12. On 07.12, I changed the the weight of Variant A from 5 to 0. What will happen with players that were already in Variant A?**