In our publication"First determination of the electric charge of the top quark", V. M. Abazov et al., PRL 98, 041801 (2007), we state:
"To discriminate between the SM and the exotic hypotheses, we form the ratio of the likelihood of the observed set of charges qi arising from a SM top quark to the likelihood for the set of qi arising from the exotic scenario, Λ = Πipsm(qi)/Πipex(qi). The subscript i runs over all 32 available measurements. The value of the ratio is determined in data and compared with the expected distributions for Λ in the SM and exotic scenarios. We find that the observed set of charges agrees well with those of a SM top quark. The probability of our observation is 7.8% in the case where the selected sample contains only exotic quarks with charge |q| = 4e/3, including systematic uncertainties. Thus, we exclude at the 92.2% C.L. that the selected data set is solely composed of an exotic quark with |q| = 4e/3."
We would like to clarify what this means: Let x(Λ|f) be the probability density function (pdf) of Λ if a fraction f of the events contain exotic quarks. From the Monte Carlo simulation we determine the pdf for the standard model hypothesis x(Λ|0) and for the exotic hypothesis x(Λ|1). Large values of Λ give evidence for the standard model hypothesis and against the exotic hypothesis. Assuming that only exotic quarks are produced, the probability to get a value of Λ that is at least as large as the observed value Λ' is:
Assuming that only standard top quarks are produced, the corresponding probability is much larger:
We therefore conclude that our observation agrees much better with the standard model than with the exotic hypothesis.
Formal hypothesis testing is not the usual practice in our field. The usual practice is to exclude parameter space, typically at 90% or 95% confidence level, and/or to state tail probabilities pertaining to the hypotheses, as we have done here.
Had we, however, a priori chosen a rejection region with fixed probability αex = 10%, we would have concluded that we can reject the f = 1 hypothesis with a confidence level of 90%, given our observed value Λ'. But had we chosen αex = 5% we would not have been able to exclude f = 1 at 95% confidence level. The maximum confidence level at which we can exclude f = 1 is the 92.2% quoted in our paper.
Since the approach chosen to determine this confidence level is somewhat unconventional, we suggest to compare the sensitivity of different analyses based on the p-value or the Bayes' factor. The p-value for the published analysis, as mentioned above, is 7.8% and the Bayes factor is 4.3.Questions? please contact Ulrich Heintz, Elizaveta Shabalina, Michele Weber.