Evolution of the ATLAS PanDA workload management system for exascale computational science

An important foundation underlying the impressive success of data processing and analysis in the ATLAS experiment [1] at the LHC [2] is the Production and Distributed Analysis (PanDA) workload management system [3]. PanDA was designed specifically for ATLAS and proved to be highly successful in meeting all the distributed computing needs of the experiment. However, the core design of PanDA is not experiment specific. The PanDA workload management system is capable of meeting the needs of other data intensive scientific applications. Alpha-Magnetic Spectrometer [4], an astro-particle experiment on the International Space Station, and the Compact Muon Solenoid [5], an LHC experiment, have successfully evaluated PanDA and are pursuing its adoption. In this paper, a description of the new program of work to develop a generic version of PanDA will be given, as well as the progress in extending PanDA's capabilities to support supercomputers and clouds and to leverage intelligent networking. PanDA has demonstrated at a very large scale the value of automated dynamic brokering of diverse workloads across distributed computing resources. The next generation of PanDA will allow other data-intensive sciences and a wider exascale community employing a variety of computing platforms to benefit from ATLAS' experience and proven tools.

ATLAS
data management
computer: network
programming
information management

References(23)

Figures(0)

[1]

The ATLAS experiment

- http://www.atlas.ch/

[2]

The LHC experiment

- http://lhc.web.cern.ch/

[3]

2011 Overview of ATLAS PanDA Workload Management J Phys. Conf Ser.,331

Maeno T

[4]

The Alpha Magnetic Spectrometer (AMS) on the International Space Station. I: Results from the test flight on the space shuttle

AMS

Collaboration

•

M. Aguilar

et al.

- Phys.Rept. 366 (2002) 331-405,
- Phys.Rept. 380 (2003) 97-98 (erratum)
•
DOI:
- 10.1016/S0370-1573(02)00013-3

[5]

The CMS experiment

- http://cms.web.cern.ch/

[6]

The CHARMM molecular modeling software

- http://www.charmm.org/

[7]

2013 The Common Analysis Framework Project Int. Conf. on Computing in High Energy and Nuclear Physics 2013 (Amsterdam)

Spiga D

[8]

The Advanced Scientific Computing Research program

- http://science.energy.gov/ascr/

[9]

Ledership Computing Facilities

- http://science.energy.gov/ascr/facilities/

[10]

2013 Task Management in the New ATLAS Production System Int. Conf. on Computing in High Energy and Nuclear Physics 2013 (Amsterdam)

Potekhin M

[11]

2013 Next Generation PanDA Pilot for ATLAS and Other Experiments Int. Conf. on Computing in High Energy and Nuclear Physics 2013 (Amsterdam)

Nilsson P

[12]

2013 Next generation database relational solutions for ATLAS distributed computing Int. Conf. on Computing in High Energy and Nuclear Physics 2013 (Amsterdam)

Dimitrov G

[13]

perfSONAR

- http://www.perfsonar.net/

[14]

2012 Virtual Network On Demand: Dedicating Network Resources to Distributed Scientific Workflows Int. Workshop on Data Intensive Distributed Computing 2012 (Delft)

Katramatos D

[15]

The Amazon Elastic Computing Cloud

- http://aws.amazon.com/ec2/

[16]

The Large Synoptic Survey Telescope

- http://www.lsst.org/lsst/

[17]

The Google Computing Engine com/products/compute-engine

- https://cloud.google

[18]

2013 ATLAS Cloud Computing R&amp

Panitkin S

[18]

D Int. Conf. on Computing in High Energy and Nuclear Physics 2013 (Amsterdam)

[19]

The Titan platform

- http://www.olcf.ornl.gov/titan/

[20]

2013 The True Cost of Data Access in ATLAS Int. Conf. on Computing in High Energy and Nuclear Physics 2013 (Amsterdam)

Vukotic I

[21]

2013 Integrating the Network into LHC Experiments: Update on the ANSE (Advanced Network Services for Experiments) Project Int. Conf. on Computing in High Energy and Nuclear Physics 2013 (Amsterdam)

Melo A

[22]

2013 AGIS: The ATLAS Grid Information System Int. Conf. on Computing in High Energy and Nuclear Physics 2013 (Amsterdam)

Anisenkov A