AgentBIAgent with Business Intelligence for DTC sites

Back to the knowledge base

BIZ-4

If the primary metric is flat but a secondary one rose, is that a success?

Published:2026-11-21Views:1

Decide before the experiment which metric is primary and which are only observations. If the primary metric is flat and a secondary one rose, the pre-written rule says that is not a success; otherwise you are swapping metrics after the fact and can spin any fluctuation as good news.

Lock the primary metric in the experiment log; use secondary metrics only to explain why the primary did not move or to suggest a next hypothesis. If a secondary metric moves consistently in the same direction across batches, validate it as a new hypothesis in a separate experiment rather than rewriting this one's conclusion.

This article covers the roles of primary and secondary metrics and read-out discipline only; it judges no experiment result and decides no launch for anyone. Write the result by the pre-agreed rule — a flat primary metric stays flat, flat numbers stay flat — and do not use it to swap metrics or change direction.

2026-11-22