Supporting High-Performance and High-Throughput Computing for Experimental Science

The advent of experimental science facilities—instruments and observatories, such as the Large Hadron Collider, the Laser Interferometer Gravitational Wave Observatory, and the upcoming Large Synoptic Survey Telescope —has brought about challenging, large-scale computational and data processing requirements. Traditionally, the computing infrastructure to support these facility’s requirements were organized into separate infrastructure that supported their high-throughput needs and those that supported their high-performance computing needs. We argue that to enable and accelerate scientific discovery at the scale and sophistication that is now needed, this separation between high-performance computing and high-throughput computing must be bridged and an integrated, unified infrastructure provided. In this paper, we discuss several case studies where such infrastructure has been implemented. These case studies span different science domains, software systems, and application requirements as well as levels of sustainability. A further aim of this paper is to provide a basis to determine the common characteristics and requirements of such infrastructure, as well as to begin a discussion of how best to support the computing requirements of existing and future experimental science facilities.

Note:

13 pages, 7 figures. Accepted to Computing and Software for Big Science

HPC
HTC
LIGO
CMS
ATLAS
Blue Waters
Titan
OSG
Containers
activity report

References(107)

Figures(10)

[1]

Distributed computing in practice: the Condor experience

- Concurrency Comput.Pract.Exp. 17 (2005) 2-4, 323-356
•
DOI:
- 10.1002/cpe.938

[2]

The Open Science Grid

Ruth Pordes
(
- Fermilab
)
,
Bill Kramer
(
- LBL, Berkeley
)
,
Doug Olson
(
- LBL, Berkeley
)
,
Miron Livny
(
- Wisconsin U., Madison
)
,
Alain Roy
(
- Wisconsin U., Madison
)

et al.

- J.Phys.Conf.Ser. 78 (2007) 012057
•
DOI:
- 10.1088/1742-6596/78/1/012057

[3]

GW150914: The Advanced LIGO Detectors in the Era of First Discoveries

LIGO Scientific and
Virgo

Collaborations

•

B.P. Abbott
(
- Caltech
)

et al.

- Phys.Rev.Lett. 116 (2016) 13, 131103
•
e-Print:
- 1602.03838
•
DOI:
- 10.1103/PhysRevLett.116.131103

[4]

Advanced LIGO

LIGO Scientific

Collaboration

•

J. Aasi
(
- Caltech
)

et al.

- Class.Quant.Grav. 32 (2015) 074001
•
e-Print:
- 1411.4547
•
DOI:
- 10.1088/0264-9381/32/7/074001

[7]

The foundation of the general theory of relativity.

Albert Einstein

- Annalen Phys. 49 (1916) 7, 769-822,
- Annalen Phys. 14 (2005) 517-571,
- ,
- Annalen Phys. 354 (1916) 7, 769-822
•
DOI:
- 10.1002/andp.19163540702

[7]

The foundation of the general theory of relativity.

Albert Einstein

- Annalen Phys. 49 (1916) 7, 769-822,
- Annalen Phys. 14 (2005) 517-571,
- ,
- Annalen Phys. 354 (1916) 7, 769-822
•
DOI:
- 10.1002/andp.19163540702

[7]

The foundation of the general theory of relativity.

Albert Einstein

- Annalen Phys. 49 (1916) 7, 769-822,
- Annalen Phys. 14 (2005) 517-571,
- ,
- Annalen Phys. 354 (1916) 7, 769-822
•
DOI:
- 10.1002/andp.19163540702

[8]

Advanced Virgo: a second-generation interferometric gravitational wave detector

VIRGO

Collaboration

•

F. Acernese
(
- Salerno U. and
- INFN, Naples
)

et al.

- Class.Quant.Grav. 32 (2015) 2, 024001
•
e-Print:
- 1408.3978
•
DOI:
- 10.1088/0264-9381/32/2/024001

[9]

Observation of Gravitational Waves from a Binary Black Hole Merger

LIGO Scientific and
Virgo

Collaborations

•

B.P. Abbott
(
- Caltech
)

et al.

- Phys.Rev.Lett. 116 (2016) 6, 061102
•
e-Print:
- 1602.03837
•
DOI:
- 10.1103/PhysRevLett.116.061102

[10]

GW151226: Observation of Gravitational Waves from a 22-Solar-Mass Binary Black Hole Coalescence

LIGO Scientific and
Virgo

Collaborations

•

B. P. Abbott
(
- LIGO Lab., Caltech
)

et al.

- Phys.Rev.Lett. 116 (2016) 24, 241103
•
e-Print:
- 1606.04855
•
DOI:
- 10.1103/PhysRevLett.116.241103

[11]

GW170104: Observation of a 50-Solar-Mass Binary Black Hole Coalescence at Redshift 0.2

LIGO Scientific and
VIRGO

Collaborations

•

Benjamin P. Abbott
(
- LIGO Lab., Caltech
)

et al.

- Phys.Rev.Lett. 118 (2017) 22, 221101,
- Phys.Rev.Lett. 121 (2018) 12, 129901 (erratum)
•
e-Print:
- 1706.01812
•
DOI:
- 10.1103/PhysRevLett.118.221101

[12]

GW170814: A Three-Detector Observation of Gravitational Waves from a Binary Black Hole Coalescence

LIGO Scientific and
Virgo

Collaborations

•

B.P. Abbott
(
- LIGO Lab., Caltech
)

et al.

- Phys.Rev.Lett. 119 (2017) 14, 141101
•
e-Print:
- 1709.09660
•
DOI:
- 10.1103/PhysRevLett.119.141101

[13]

GW170608: Observation of a 19-solar-mass Binary Black Hole Coalescence

LIGO Scientific and
Virgo

Collaborations

•

B.. P.. Abbott
(
- LIGO Lab., Caltech
)

et al.

- Astrophys.J.Lett. 851 (2017) L35
•
e-Print:
- 1711.05578
•
DOI:
- 10.3847/2041-8213/aa9f0c

[14]

GWTC-1: A Gravitational-Wave Transient Catalog of Compact Binary Mergers Observed by LIGO and Virgo during the First and Second Observing Runs

LIGO Scientific and
Virgo

Collaborations

•

B.P. Abbott
(
- LIGO Lab., Caltech
)

et al.

- Phys.Rev.X 9 (2019) 3, 031040
•
e-Print:
- 1811.12907
•
DOI:
- 10.1103/PhysRevX.9.031040

[15]

GW170817: Observation of Gravitational Waves from a Binary Neutron Star Inspiral

LIGO Scientific and
Virgo

Collaborations

•

B.P. Abbott
(
- LIGO Lab., Caltech
)

et al.

- Phys.Rev.Lett. 119 (2017) 16, 161101
•
e-Print:
- 1710.05832
•
DOI:
- 10.1103/PhysRevLett.119.161101

[16]

Multi-messenger Observations of a Binary Neutron Star Merger

LIGO Scientific and
Virgo and
Fermi GBM and
INTEGRAL and
IceCube and
IPN and
Insight-Hxmt and
ANTARES and
Swift and
Dark Energy Camera GW-EM and
DES and
DLT40 and
GRAWITA and
Fermi-LAT and
ATCA and
ASKAP and
OzGrav and
DWF (Deeper Wider Faster Program) and
AST3 and
CAASTRO and
VINROUGE and
MASTER and
J-GEM and
GROWTH and
JAGWAR and
CaltechNRAO and
TTU-NRAO and
NuSTAR and
Pan-STARRS and
KU and
Nordic Optical Telescope and
ePESSTO and
GROND and
Texas Tech University and
TOROS and
BOOTES and
MWA and
CALET and
IKI-GW Follow-up and
H.E.S.S. and
LOFAR and
LWA and
HAWC and
Pierre Auger and
ALMA and
Pi of Sky and
Chandra Team at McGill University and
DFN and
ATLAS Telescopes and
High Time Resolution Universe Survey and
RIMAS and
RATIR and
SKA South Africa/MeerKAT

Collaborations

•

B.P. Abbott
(
- LIGO Lab., Caltech
)

et al.

- Astrophys.J.Lett. 848 (2017) 2, L12
•
e-Print:
- 1710.05833
•
DOI:
- 10.3847/2041-8213/aa91c9

[17]

Estimating the Contribution of Dynamical Ejecta in the Kilonova Associated with GW170817

LIGO Scientific and
Virgo

Collaborations

•

B.P. Abbott
(
- LIGO Lab., Caltech
)

et al.

- Astrophys.J.Lett. 850 (2017) 2, L39
•
e-Print:
- 1710.05836
•
DOI:
- 10.3847/2041-8213/aa9478

[18]

NUMERICAL CONSTRUCTION OF SPACE-TIME

L. Smarr
(
- Harvard U.
)

- Proc.Roy.Soc.Lond.A 368 (1979) 15-16
•
DOI:
- 10.1098/rspa.1979.0109

[19]

Hobill DW, Smarr LL Supercomputing and numerical relativity: a look at the past, present and future supercomputing and numerical relativity: a look at the past, present and future. Cambridge Univerty Press, Cambridge, pp 1-17

[20]