How Much Longer Do I Need to Run?

What is this?

This calculator estimates how many more days you need to run your experiment to achieve statistical significance, assuming your current conversion rates hold steady.

Why use it?

Sometimes you reach your pre-calculated sample size but your test hasn't reached significance yet. This tool helps you understand whether it's just a matter of time, or if your test is underpowered given the observed lift.

How it works

We take the current observed conversion rates and use the absolute lift between variant and control as the new assumed MDE. We then calculate the required sample size per variant needed to detect that lift with the desired confidence and power.

The remaining sample size per variant (the number of additional users/exposures still needed per variant) is then converted to days using your current daily traffic (assuming equal 50/50 split).

Formulas

We calculate required days using the observed absolute lift as the assumed MDE (Minimum Detectable Effect):

Then compute:

Where:

• $p$ is the observed control conversion rate
• "Observed Lift" is the absolute difference between variant and control conversion rates, used as the assumed MDE
• $Z_{\alpha}$ is based on the confidence level (e.g. 1.96 for 95%)
• $Z_{\beta}$ is based on the power (e.g. 0.84 for 80%)
• "Remaining Sample Size" is how many more users you still need per variant
• We assume a 50/50 traffic split between control and variant