Which evidence designs does a parameter set?
Asked from Minimum detectable effect, a Parameters: The smallest lift an experiment is designed to detect, stated before it runs. The sample and the holdout are sized to it, and an effect below it is not evidence of absence.
Markdown gives the tables as they read here. The file gives every path as one line per step, which opens in a spreadsheet and pivots into whatever you need.
3 evidence designs reached
The whole answer is drawn here; the file beside it carries it away.
Minimum detectable effect parameter of Holdout experiment, Geo experiment and Matched-market test.
Drawn as a list ·how this question is walked