Learning relevant features of data with multi-scale tensor networks - INSPIRE

DataBETA

Learning relevant features of data with multi-scale tensor networks

E.Miles Stoudenmire

Apr 25, 2018

Published in:

Quantum Sci.Technol. 3 (2018) 3, 034003

Published: Apr 25, 2018

DOI:

10.1088/2058-9565/aaba1a

reference search77 citations

Citations per year

Abstract: (IOP)

Inspired by coarse-graining approaches used in physics, we show how similar algorithms can be adapted for data. The resulting algorithms are based on layered tree tensor networks and scale linearly with both the dimension of the input and the training set size. Computing most of the layers with an unsupervised algorithm, then optimizing just the top layer for supervised classification of the MNIST and fashion MNIST data sets gives very good results. We also discuss mixing a prior guess for supervised weights together with an unsupervised representation of the data, yielding a smaller number of features nevertheless able to give good performance.

References(73)

Figures(0)

Tensor decompositions for learning latent variable models

Anandkumar A
,
Ge R
,
Hsu D
,
Kakade S. M
,
Telgarsky M

- J.Machine Learning Res. 15 (2014) 2773

Training input–output recurrent neural networks through spectral methods

Sedghi H
,
Anandkumar A

•
e-Print:
- 1603.00954

Tensorizing neural networks

Novikov A
,
Podoprikhin D
,
Osokin A
,
Vetrov D

•
e-Print:
- 1509.06569

On the expressive power of deep learning: a tensor analysis

Cohen N
,
Sharir O
,
Shashua A

Exponential Machines

e-Print:
- 1605.03795

Supervised learning with tensor networks

Stoudenmire E. M
,
Schwab D. J

- Adv.Neural Inf.Process.Syst. 29 (2016) 4799

Long-term forecasting using tensor-train RNNs

Yu R
,
Zheng S
,
Anandkumar A
,
Yue Y

•
e-Print:
- 1711.00073

Tensor contraction & regression networks

Kossaifi J
,
Lipton Z. C
,
Khanna A
,
Furlanello T
,
Anandkumar A

•
e-Print:
- 1707.08308

Compact neural networks based on the multiscale entanglement renormalization ansatz

Hallam A
,
Grant E
,
Stojevic V
,
Severini S
,
Green A. G

•
e-Print:
- 1711.03357

Inductive bias of deep convolutional networks through pooling geometry

Cohen N
,
Shashua A

•
e-Print:
- 1605.06743

Polynomial networks and factorization machines: new insights and efficient training algorithms

Blondel M
,
Ishihata M
,
Fujino A
,
Ueda N

•
e-Print:
- 1607.08810

A Practical Introduction to Tensor Networks: Matrix Product States and Projected Entangled Pair States

Roman Orus
(
- Mainz U., Inst. Phys.
)

- Annals Phys. 349 (2014) 117-158
•
e-Print:
- 1306.2164
•
DOI:
- 10.1016/j.aop.2014.06.013

Renormalization group and critical phenomena. 1. Renormalization group and the Kadanoff scaling picture

Kenneth G. Wilson
(
- Cornell U., LNS
)

- Phys.Rev.B 4 (1971) 3174-3183
•
DOI:
- 10.1103/PhysRevB.4.3174

PROBLEMS IN PHYSICS WITH MANY SCALES OF LENGTH

K.G. Wilson

- Sci.Am. 241 (1979) 140-157
•
DOI:
- 10.1038/scientificamerican0879-158

Representation and design of wavelets using unitary circuits

Evenbly G
,
White S. R

•
e-Print:
- 1605.07312

Erhan D
,
Bengio Y
,
Courville A
,
Vincent P

An exact mapping between the Variational Renormalization Group and Deep Learning

e-Print:
- 1410.3831

PCA meets RG

Bradde S
,
Bialek W

- J.Statist.Phys. 167 (2016) 462
•
DOI:
- 10.1007/s10955-017-1770-6

FINITELY CORRELATED STATES ON QUANTUM SPIN CHAINS

M. Fannes
(
- Leuven U.
)
,
B. Nachtergaele
(
- Chile U., Santiago
)
,
R.F. Werner
(
- Dublin Inst.
)

- Commun.Math.Phys. 144 (1992) 443-490
•
DOI:
- 10.1007/BF02099178

Thermodynamic Limit of Density Matrix Renormalization for the spin-1 heisenberg chain

Stellan Ostlund
(
- Chalmers U. Tech.
)
,
Stefan Rommer
(
- Chalmers U. Tech.
)

- Phys.Rev.Lett. 75 (1995) 3537-3540
•
e-Print:
- cond-mat/9503107
•
DOI:
- 10.1103/PhysRevLett.75.3537

Efficient Classical Simulation of Slightly Entangled Quantum Computations

Guifré Vidal
(
- Caltech, IQI
)

- Phys.Rev.Lett. 91 (2003) 14, 147902
•
DOI:
- 10.1103/PhysRevLett.91.147902

Renormalization algorithms for quantum-many body systems in two and higher dimensions

e-Print:
- cond-mat/0407066

Entanglement Renormalization

G. Vidal
(
- Queensland U. and
- Caltech, IQI
)

- Phys.Rev.Lett. 99 (2007) 22, 220405
•
e-Print:
- cond-mat/0512165
•
DOI:
- 10.1103/PhysRevLett.99.220405

Density matrix formulation for quantum renormalization groups

Steven R. White
(
- UC, Irvine
)

- Phys.Rev.Lett. 69 (1992) 2863-2866
•
DOI:
- 10.1103/PhysRevLett.69.2863

The density-matrix renormalization group

U. Schollwock
(
- RWTH Aachen U.
)

- Rev.Mod.Phys. 77 (2005) 259-315
•
e-Print:
- cond-mat/0409292
•
DOI:
- 10.1103/RevModPhys.77.259

1-25 of 73
1
2
3
25 / page