Conversion experiments most often fail by concluding after a few days: before the sample is large enough and the variance has settled, a single day's lift may just be noise. Write the sample size and read-out rules before the experiment starts — which metric, how many samples before reading, how large a difference counts — so results are not explained by cherry-picking afterwards.
Fix the primary metric and denominator up front, estimate a sample size for the difference you want to detect and log it; do not change the definition mid-run or declare success because an interim looks good. Anything touching payment or checkout is verified in sandbox only, never A/B tested on real orders; archive the sample size, start/end time and metric version together when reading out.
This article covers sample size and read-out discipline only; it issues no compliance verdict on any result and decides no launch for anyone. Write conclusions as recorded — an unclear difference stays unclear, flat numbers stay flat — and do not use them to push more experiments or a change of direction.