literature
Literature
Authors
Jobs
Seminars
Conferences
DataBETA

Some comments on computational mechanics, complexity measures, and all that

Aug 10, 2017
10 pages
e-Print:
View in:
pdf
reference search0 citations

Citations per year

0 Citations
Abstract: (arXiv)
We comment on some conceptual and and technical problems related to computational mechanics, point out some errors in several papers, and straighten out some wrong priority claims. We present explicitly the correct algorithm for constructing a minimal unifilar hidden Markov model ("ϵ\epsilon-machine") from a list of forbidden words and (exact) word probabilities in a stationary stochastic process, and we comment on inference when these probabilities are only approximately known. In particular we propose minimization of forecasting complexity as an alternative basis for statistical inference of time series, in contrast to the traditional maximum entropy principle. We present a simple and precise way of estimating excess entropy (aka "effective measure complexity". Most importantly, however, we clarify some basic conceptual problems. In particular, we show that there exist simple models (called "totally recurrent graphs") where none of the nodes of the "ϵ\epsilon-machine" (the "causal states") corresponds to an element of a state (or history) space partition.
Note:
  • 12 pages, 4 figures; note added to version 1 in reply to arXiv:1710.06832
  • [1]
    A word with low probability might not yet have shown up in the data and thus seems forbidden, although it is not (false negative)
    • [2]
      A false positive word, i.e. a word which appeared by mistake although it should be forbidden
      • [3]
        Two conditional probabilities pr{an+1|an, an-1,...} and pr{an+1|an, an-1,...} might be very similar and look the same, although they are not (false equality)
        • [4]
          The opposite (false inequality)
          • [5]
            Long forbidden words are not seen. This is different from point #2 (false positive), in that the occurrence of such words is then inferred by Occam’s razor, not on the basis of wrong data. It was in view of these problems that inference of mUHMCs was deliberately avoided in [27-33]. In retrospect this was an error. The reason is mainly Popper’s observation that empirical statements can never be verified anyhow: Although we can never, in view of the above list, be sure that what we are doing is correct, attempts at statistical inference can nevertheless provide useful conjectures which should then be subjected to further scrutiny. Indeed, compared to most other statistical inference problems, the present problem is worse in that any typical characteristic of the inferred mUHMC (such as the FC ) depends non-continuously on the input data. More precisely, we expect lower semi-continuity (as proven in [24] for the FC), i.e. any infinitesimal change of the input data can only induce a jump towards higher complexity, not lower. Take e.g. the example of Fig. 2, but assume that the probabilities were derived from finite observations, with the result that pr{1|11} = 2/3 - ± δ with, say, = 3δ. If we (wrongly) assume this difference from pr{1|11} = 2/3 to be significant and thus real, we would