Another Look at Confidence Intervals: Proposal for a More Relevant and Transparent Approach - INSPIRE

DataBETA

Another Look at Confidence Intervals: Proposal for a More Relevant and Transparent Approach

Steven D. Biller
(
- Oxford U.
)
,
Scott M. Oser
(
- British Columbia U.
)

May 20, 2014

17 pages

Published in:

Nucl.Instrum.Meth.A 774 (2015) 103-119

Published: Feb 21, 2015

e-Print:

1405.5010 [physics.data-an]

DOI:

10.1016/j.nima.2014.11.081

View in:

ADS Abstract Service

reference search17 citations

Citations per year

Abstract: (Elsevier)

The behaviors of various confidence/credible interval constructions are explored, particularly in the region of low event numbers where methods diverge most. We highlight a number of challenges, such as the treatment of nuisance parameters, and common misconceptions associated with such constructions. An informal survey of the literature suggests that confidence intervals are not always defined in relevant ways and are too often misinterpreted and/or misapplied. This can lead to seemingly paradoxical behaviors and flawed comparisons regarding the relevance of experimental results. We therefore conclude that there is a need for a more pragmatic strategy which recognizes that, while it is critical to objectively convey the information content of the data, there is also a strong desire to derive bounds on model parameter values and a natural instinct to interpret things this way. Accordingly, we attempt to put aside philosophical biases in favor of a practical view to propose a more transparent and self-consistent approach that better addresses these issues.

Note:

23 pages, 11 figures

Frequentist
Bayesian
Confidence intervals
interpretation of experiments
experimental results
frequentist
statistics: Bayesian
statistical analysis
numerical calculations
data analysis method

References(27)

Figures(11)

[1]

Outline of a Theory of Statistical Estimation Based on the Classical Theory of Probability

J. Neyman
(
- University Coll. London
)

- Phil.Trans.Roy.Soc.Lond.A 236 (1937) 767, 333-380
•
DOI:
- 10.1098/rsta.1937.0005

[2]

A Unified approach to the classical statistical analysis of small signals

Gary J. Feldman
(
- Harvard U.
)
,
Robert D. Cousins
(
- UCLA
)

- Phys.Rev.D 57 (1998) 3873-3889
•
e-Print:
- physics/9711021
•
DOI:
- 10.1103/PhysRevD.57.3873

[3]

Kendall’s Advanced Theory of h Ed. (Oxford University, New York,)

A. Stuart
,
J.K. Ord

- Statistics 2 (1991) 5

[4]

Improved probability method for estimating signal in the presence of background

Byron P. Roe
(
- Michigan U.
)
,
Michael B. Woodroofe
(
- Michigan U.
)

- Phys.Rev.D 60 (1999) 053009
•
e-Print:
- physics/9812036
•
DOI:
- 10.1103/PhysRevD.60.053009

[5]

Comment on [Improved probability method for estimating signal in the presence of background]

Robert D. Cousins
(
- UCLA
)

- Phys.Rev.D 62 (2000) 098301
•
DOI:
- 10.1103/PhysRevD.62.098301

[6]

Setting confidence belts

Byron P. Roe
(
- Michigan U.
)
,
Michael B. Woodroofe
(
- Michigan U.
)

- Phys.Rev.D 63 (2001) 013009
•
e-Print:
- hep-ex/0007048
•
DOI:
- 10.1103/PhysRevD.63.013009

[7]

First observation of PeV-energy neutrinos with IceCube

IceCube

Collaboration

•

M.G. Aartsen
(
- Adelaide U.
)

et al.

- Phys.Rev.Lett. 111 (2013) 021103
•
e-Print:
- 1304.5356
•
DOI:
- 10.1103/PhysRevLett.111.021103

[8]

The Search for neutrino oscillations muon anti-neutrino ---> electron anti-neutrino with KARMEN

KARMEN

Collaboration

•

K. Eitel
(
- Karlsruhe U., EKP and
- Karlsruhe U.
)

et al.

- Nucl.Phys.B Proc.Suppl. 77 (1999) 212-219,
•
e-Print:
- hep-ex/9809007
•
DOI:
- 10.1016/S0920-5632(99)00420-X

[9]

Latest results of the KARMEN2 experiment

KARMEN

Collaboration

•

K. Eitel
(
- Karlsruhe U., EKP
)

for the collaboration.

- Nucl.Phys.B Proc.Suppl. 91 (2001) 191-197,
•
e-Print:
- hep-ex/0008002
•
DOI:
- 10.1016/S0920-5632(00)00940-3

[10]

Search for the standard model Higgs boson at LEP

LEP Working Group for Higgs boson searches and
ALEPH and
DELPHI and
L3 and
OPAL

Collaborations

•

R. Barate

et al.

- Phys.Lett.B 565 (2003) 61-75
•
e-Print:
- hep-ex/0306033
•
DOI:
- 10.1016/S0370-2693(03)00614-2

[11]

Presentation of search results: The $CL_s$ technique

Alexander L. Read
(
- Oslo U.
)

- J.Phys.G 28 (2002) 2693-2704,
•
DOI:
- 10.1088/0954-3899/28/10/313

[12]

Result from the First Science Run of the ZEPLIN-III Dark Matter Search Experiment

V.N. Lebedenko
(
- Imperial Coll., London
)
,
H.M. Araujo
(
- Imperial Coll., London and
- Rutherford
)
,
E.J. Barnes
(
- Edinburgh U.
)
,
A. Bewick
(
- Imperial Coll., London
)
,
R. Cashmore
(
- Oxford U.
)

et al.

- Phys.Rev.D 80 (2009) 052010
•
e-Print:
- 0812.1150
•
DOI:
- 10.1103/PhysRevD.80.052010

[13]

Search for Neutrinoless Double-Beta Decay in $^{136}$ Xe with EXO-200

EXO-200

Collaboration

•

M. Auger
(
- Bern U.
)

et al.

- Phys.Rev.Lett. 109 (2012) 032505
•
e-Print:
- 1205.5608
•
DOI:
- 10.1103/PhysRevLett.109.032505

[14]

Search for Majorana neutrinos with the first two years of EXO-200 data

EXO-200

Collaboration

•

J.B. Albert
(
- Indiana U.
)

et al.

- Nature 510 (2014) 229-234
•
e-Print:
- 1402.6956
•
DOI:
- 10.1038/nature13432

[15]

Details provided by the EXO collaboration (private communication)

[16]

G.M. Cordeiro

- J.Stat.Comput.Simul. 53 (1995) 211

[17]

Frequentist hypothesis testing with background uncertainty

Kyle S. Cranmer
(
- Wisconsin U., Madison
)

- eConf C030908 (2003) WEMT004,
•
e-Print:
- physics/0310108

[18]

Incorporating systematic uncertainties into an upper limit

Robert D. Cousins
(
- UCLA
)
,
Virgil L. Highland
(
- Temple U.
)

- Nucl.Instrum.Meth.A 320 (1992) 331-335
•
DOI:
- 10.1016/0168-9002(92)90794-5

[19]

Coverage of confidence intervals for Poisson statistics in presence of systematic uncertainties

•
e-Print:
- hep-ex/0206034

[20]

Limits and confidence intervals in the presence of nuisance parameters

Wolfgang A. Rolke
(
- Puerto Rico U., Mayaguez
)
,
Angel M. Lopez
(
- Puerto Rico U., Mayaguez
)
,
Jan Conrad
(
- CERN
)

- Nucl.Instrum.Meth.A 551 (2005) 493-503,
•
e-Print:
- physics/0403059
•
DOI:
- 10.1016/j.nima.2005.05.068

[21]

Precise Measurement of the Neutrino Mixing Parameter $\theta_{23}$ from Muon Neutrino Disappearance in an Off-Axis Beam

T2K

Collaboration

•

K. Abe
(
- Tokyo U., ICRR
)

et al.

- Phys.Rev.Lett. 112 (2014) 18, 181801
•
e-Print:
- 1403.1532
•
DOI:
- 10.1103/PhysRevLett.112.181801

[22]

Memoir on the probability of causes of events

P.S. Laplace

- Statist.Sci. 1 (1986) 364378

[23]

H. Jeffreys

- Proc.Roy.Soc.Lond.A 186 (1946) 453461

[24]

An Independent Measurement of the Total Active B-8 Solar Neutrino Flux Using an Array of He-3 Proportional Counters at the Sudbury Neutrino Observatory

SNO

Collaboration

•

B. Aharmim
(
- Laurentian U.
)

et al.

- Phys.Rev.Lett. 101 (2008) 111301
•
e-Print:
- 0806.0989
•
DOI:
- 10.1103/PhysRevLett.101.111301

[25]

Statistical Evaluation of Experimental Determinations of Neutrino Mass Hierarchy

X. Qian
(
- Caltech
)
,
A. Tan
(
- Iowa State U.
)
,
W. Wang
(
- William-Mary Coll.
)
,
J.J. Ling
(
- Brookhaven
)
,
R.D. McKeown
(
- William-Mary Coll. and
- Jefferson Lab
)

et al.

- Phys.Rev.D 86 (2012) 113011
•
e-Print:
- 1210.3651
•
DOI:
- 10.1103/PhysRevD.86.113011

1-25 of 27
1
2
25 / page