when does it refuse to give you a number?

Never being named in five draws is compatible with an appearance rate of 43.4%. In ten draws it is 27.8%. The textbook normal approximation collapses to a point at the edges and hands back a confident nothing, so this uses the Wilson score interval, which keeps a bound where the evidence is thinnest. A dashboard that prints 0% for a brand that was asked about five times is not reporting absence, it is reporting that nobody looked hard enough to tell.

Appearing twice at position one is a different fact from appearing nine times at position five, so rate and rank are kept apart and never blended into one score. And a rank is only printed where the ordering repeats across draws. Where it does not, the ordering was noise, and a number computed from noise reads exactly like a number computed from evidence.

A competitor at six of ten against your four of ten is not ahead of you. Both of those samples support a wide range of true rates and those ranges sit on top of each other, so the ordering is not supported and no ranking is printed. The word ahead has to be earned by non-overlapping intervals, not by a bigger number.

Where an interval spans the base rate the result is recorded as undecided, and undecided is not a quiet pass. The gate opened at the place the evidence was thinnest is a gate that never closed. A question that carries the brand’s own name is thrown out of the rate entirely for the same reason, because a question with the name in it is answered with the name whatever the model knows.

Two rounds collected from two different engines are not one round, so a file holding both is refused and the engines it holds are named. A before and after taken across a model version change gets no verdict either, because the thing underneath the measurement moved.

how many draws it takes to say anything ± the word a causal claim has to earn