Approximation by superpositions of a sigmoidal function

Dec 1, 1989
12 pages
Published in:
  • Math.Control Signals Syst. 2 (1989) 4, 303-314
  • Published: Dec 1, 1989

Citations per year

19912000200920182025010203040
Abstract: (Springer)
In this paper we demonstrate that finite linear combinations of compositions of a fixed, univariate function and a set of affine functionals can uniformly approximate any continuous function ofn real variables with support in the unit hypercube; only mild conditions are imposed on the univariate function. Our results settle an open question about representability in the class of single hidden layer neural networks. In particular, we show that arbitrary decision regions can be arbitrarily well approximated by continuous feedforward neural networks with only a single internal, hidden layer and any continuous sigmoidal nonlinearity. The paper discusses approximation properties of other possible types of nonlinearities that might be implemented by artificial neural networks.
  • Neural networks
  • Approximation
  • Completeness
  • [A]
    Real Analysis and Probability
    • R.B. Ash
  • [BH]
    What size net gives valid generalization?,Neural Comput. (to appear)
    • E. Baum
      ,
    • D. Haussler
    • [B]
      Special section on neural networks for systems and control
      • B. Bavarian
    • [BEHW]
      Classifying learnable geometric concepts with the Vapnik-Chervonenkis dimension,Proceedings of the 18th Annual ACM Symposium on Theory of Computing, Berkeley, CA,, pp. 273-282
      • A. Blumer
        ,
      • A. Ehrenfeucht
        ,
      • D. Haussler
        ,
      • M.K. Warmuth
    • [BST]
      Spectral synthesis and the Pompeiu problem
      • L. Brown
        ,
      • B. Schreiber
        ,
      • B.A. Taylor
        • Annales Inst.Fourier 23 (1973) 125-154
    • [CD]
      Construction of neural nets using the Radon transform, preprint
      • S.M. Carroll
        ,
      • B.W. Dickinson
    • [C]
      Continuous Valued Neural Networks with Two Hidden Layers are Sufficient, Technical Report, Department of Computer Science, Tufts University
      • G. Cybenko
    • [DS]
      On nonlinear functions of linear combinations
      • P. Diaconis
        ,
      • M. Shahshahani
    • [F]
      On the approximate realization of continuous mappings by neural networks,Neural Networks (to appear)
      • K. Funahashi
      • [G]
        Special section on neural networks
        • L.J. Griffiths
      • [HSW]
        Multi-layer feedforward networks are universal approximators, preprint
        • K. Hornik
          ,
        • M. Stinchcombe
          ,
        • H. White
      • [HL1]
        Comparisons Between Neural Net and Conventional Classifiers, Technical Report, Lincoln Laboratory, MIT
        • W.Y. Huang
          ,
        • R.P. Lippmann
      • [HL2]
        Neural Net and Traditional Classifiers, Technical Report, Lincoln Laboratory, MIT
        • W.Y. Huang
          ,
        • R.P. Lippmann
      • [H]
        Projection pursuit
        • P.J. Huber
          • Annals Statist. 13 (1985) 435-475
      • [J]
        Constructive approximations for neural networks by sigmoidal functions, Technical Report Series, No. 7, Department of Mathematics, University of Lowell
        • L.K. Jones
      • [K]
        On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition
        • A.N. Kolmogorov
          • Dokl.Akad.Nauk SSSR 114 (1957) 953-956
      • [LF]
        Nonlinear Signal Processing Using Neural Networks: Prediction and System Modeling, Technical Report, Theoretical Division, Los Alamos National Laboratory
        • A. Lapedes
          ,
        • R. Farber
      • [L1]
        An introduction to computing with neural nets
        • R.P. Lippmann
      • [L2]
        The 13th problem of Hilbert, inMathematical Developments Arising from Hilbert’s Problems F. Browder, ed
        • G.G. Lorentz
      • [MSJ]
        Classification capabilities of two-layer neural nets.Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, Glasgow,(to appear)
        • J. Makhoul
          ,
        • R. Schwartz
          ,
        • A. El-Jaroudi
      • [MP]
        and S. Papert,Perceptrons, MIT Press, Cambridge, MA,.MATH
        • M. Minsky
      • [N]
        Learning Machines
        • N.J. Nilsson
      • [P]
        On representation and approximation of nonlinear systems, Part II: Discrete systems,Biol. Cybernet.,34 , 49-52
        • G. Palm
      • [R1]
        Real and Complex Analysis
        • W. Rudin
      • [R2]
        Functional Analysis
        • W. Rudin