Making the Decision
Making a decision for an experiment enables you to 'ship' the winning group to all your users. After you make the decision, the variant that your users see depends on whether you're using a targeting gate for your experiment. The results for your experiment will still be accessible after you make a decision, but they will stop updating. The last day of metrics will be the day you "make a decision" on the experiment.
Experiments With No Targeting Gate
When you ship a group in an experiment with no targeting gate, the parameter values from the shipped group will become the default values for all your users going forward.
If the experiment happens to use parameters from a layer, the layer's parameters will now take on the shipped group's parameter values as their defaults. These are the values that all your users will see going forward.
For example, suppose you have a Demo Layer that's configured with a parameter, a_param. It's default value is set to layer_default as shown below.
Say you decide to create an experiment, Demo Experiment in Demo Layer as shown below.
You set up Demo Experiment with two groups: Control and Test, intending to experiment with new values for the layer-level parameter, a_param as shown below.
Now if you decide to ship the Control group for the Demo Experiment, a_param will take the value set for the Control group as its default: experiment_one_control
On the other hand, if you decide to ship the Test group, a_param will take the value set for the Test group as its default: experiment_two_test
Experiment With a Targeting Gate
When you decide ship a group in an experiment configured with a targeting gate, you can decide whether to continue targeting after shipping.
- If you decide to discontinue targeting, the parameter values from the shipped group will become the default values for all your users going forward. If the experiment happens to use parameters from a layer, the layer's parameters will now take on the shipped group's parameter values as their defaults. These are the values that your users will see going forward.
- If you decide to continue targeting, this will add an override to the experiment layer so that:
- all users who pass the targeting gate will see shipped group's parameter values
- all users who fail the targeting gate will see the default value (layer-level parameter defaults or the defaults you set for the parameter in your code)
Note that if you decide to continue targeting, shipping a group will not update the default value of any layer parameters.
For example, suppose Demo Experiment in a Demo Layer that has a parameter called targeted_layer_param, whose default value is set to targeted_layer_default_value.
When you decide to ship Demo Experiment, if you discontinue targeting, targeted_layer_param will now take on the value from the Control group, targeted_layer_control, as its default.
On the other hand, if you decide to continue targeting, targeted_layer_param will now acquire an override so that:
- all users who pass the targeting gate will see the overridden value of the parameter
- all users who fail the targeting gate will see the default value of the parameter
In this case, the default value of targeted_layer_param in Demo Layer will not change. Also, any users who pass the targeting gate will not be eligible for future experiments run in this layer. For this reason, we do not encourage shipping experiments with targeting on, especially when the experiment is in a layer.
