The early experiment winner

Only 150 users have reached the outcome, and the team checks the result every hour. The original analysis plan required a longer observation period.

Consider uncertainty, delayed effects, repeated peeking, and the practical decision threshold.

Inspect the effect estimate and uncertainty, assignment health, and metric maturity. Repeatedly stopping at a favorable conventional significance result can increase false positives unless the method accounts for it.

Do not call the variant a winner because its point estimate is positive. If there is an urgent reason to change the plan, document the reason and interpret the resulting evidence appropriately.

A test needs a valid stopping approach and enough information about the outcome that matters.