Global Optimisation of Black-Box Functions with Generative Models in the Wasserstein Space

Jul 16, 2024
e-Print:
DOI:

Citations per year

0 Citations
Abstract: (arXiv)
We propose a new uncertainty estimator for gradient-free optimisation of black-box simulators using deep generative surrogate models. Optimisation of these simulators is especially challenging for stochastic simulators and higher dimensions. To address these issues, we utilise a deep generative surrogate approach to model the black box response for the entire parameter space. We then leverage this knowledge to estimate the proposed uncertainty based on the Wasserstein distance - the Wasserstein uncertainty. This approach is employed in a posterior agnostic gradient-free optimisation algorithm that minimises regret over the entire parameter space. A series of tests were conducted to demonstrate that our method is more robust to the shape of both the black box function and the stochastic response of the black box than state-of-the-art methods, such as efficient global optimisation with a deep Gaussian process surrogate.
Note:
  • European Conference on Artificial Intelligence 2024 Main Track
  • [1]
    Optimization of parameters for molecular dynamics simulation using smooth particle-mesh Ewald in GROMACS 4.5
    • M.J. Abraham
      ,
    • J.E. Gready
      • J.Comput.Chem. 32 (2011) 2031-2040
  • [2]
    Exploration of differentiability in a proton computed tomography simulation framework. Physics in Medicine & Biology, 68 (24):244002, dec
    • M. Aehle
  • [2]
    URL https: //dx.doi.org/
    • M. Aehle
  • [3]

    Collaboration
  • [5]
    Two decades of blackbox optimization applications. EURO Journal on Computational Optimization, 9:100011
    • S. Alarie
      ,
    • C. Audet
      ,
    • A.E. Gheribi
      ,
    • M. Kokkolaras
      ,
    • S. Le Digabel
  • [6]
    Stochastic optimization of GeantV code by use of genetic algorithms. In Journal of Physics: Conference Series, volume 898(4), page 042026
    • G. Amadio
      ,
    • J. Apostolakis
      ,
    • M. Bandieramonte
      ,
    • S. Behera
      ,
    • R. Brun
    et al.
  • [7]
    Sequential stochastic blackbox optimization with zeroth-order gradient estimators
    • C. Audet
      ,
    • J. Bigeon
      ,
    • R. Couderc
      ,
    • M. Kokkolaras
  • [8]
    Genetic programming: an introduction: on the automatic evolution of computer programs and its applications. Morgan Kaufmann Publishers Inc
    • W. Banzhaf
      ,
    • P. Nordin
      ,
    • R.E. Keller
      ,
    • F.D. Francone
  • [9]
    Quantifying Distributional Model Risk via Optimal Transport
    • J. Blanchet
      ,
    • K.R.A. Murthy
  • [10]
    Conditioning by adaptive sampling for robust design. In International conference on machine learning, pages 773-782. PMLR
    • D. Brookes
      ,
    • H. Park
      ,
    • J. Listgarten
  • [11]
    Convergence Rates of Efficient Global Optimization Algorithms
    • A.D. Bull
  • [12]
    A Compositional Object-Based
    • M.B. Chang
      ,
    • T.D. Ullman
      ,
    • A. Torralba
      ,
    • J.B. Tenenbaum
  • [13]
    Stochastic optimization using a trust-region method and random models
    • R. Chen
      ,
    • M. Menickelly
      ,
    • K. Scheinberg
      • Math.Programming 169 (2018) 447-487
  • [14]
    Trust region methods
    • A.R. Conn
      ,
    • N.I. Gould
      ,
    • P.L. Toint
  • [15]
    Learning surrogate models for simulation-based optimization. AIChE Journal, 60(6): 2211-2227,. doi:
    • A. Cozad
      ,
    • N.V. Sahinidis
      ,
    • D.C. Miller
  • [15]
    com/doi/abs/
    • A. Cozad
      ,
    • N.V. Sahinidis
      ,
    • D.C. Miller
  • [16]
    Deep Gaussian Processes. In C. M. Carvalho and P. Ravikumar (eds.)
    • A. Damianou
      ,
    • N.D. Lawrence
  • [17]
    Tenenbaum, and J. Z. Kolter. End-to-End Differentiable Physics for Learning and Control. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.)
    • F. de Avila Belbute-Peres
      ,
    • K. Smith
      ,
    • J. K. Allen
  • [18]
    Bayesian Optimization in High Dimensions via Random Embeddings. In Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence
    • N. de Freitas
      ,
    • Z. Wang
  • [19]
    A Differentiable
    • J. Degrave
      ,
    • M. Hermans
      ,
    • J. Dambre
      ,
    • F. Wyffels
  • [20]
    High-dimensional gaussian process bandits
    • J. Djolonga
      ,
    • A. Krause
      ,
    • V. Cevher
      • Adv.Neural Inf.Process.Syst. 26 2013
  • [22]
    Automatic model construction with Gaussian processes. PhD thesis, University of Cambridge
    • D. Duvenaud
  • [23]
    Data-driven Distributionally Robust Optimization Using the Wasserstein Metric: Performance Guarantees and Tractable Reformulations
    • P.M. Esfahani
      ,
    • D. Kuhn