Sequential Monte Carlo methods in practice. Information Science and Statistics / New York, 2001. Boomsma, W., Mardia, K. V., Taylor, C. C., FerkinghoffBorg, J., Krogh, A., and Hamelryck, T. A / generative, probabilistic model of local protein structure / Davidson, T. R., Falorsi, L., De Cao, N., Kipf, T., and Tomczak, J. M / Hyperspherical variational auto-encoders. Conference on Uncertainty in Artificial Intelligence, 2018. Dinh, L., Sohl-Dickstein, J., and Bengio, S

Density estimation using Real NVP. International Conference on Learning Representations, 2017 / Orientation statistics

Circular regression / Durkan, C., Bekasov, A., Murray, I., and Papamakarios, G / Cubic-spline flows. ICML workshop on Invertible Neural Networks and Normalizing Flows, 2019a. Durkan, C., Bekasov, A., Murray, I., and Papamakarios, G / Neural spline flows. Advances in Neural Information Processing Systems, 2019b

Normalizing Flows on Tori and Spheres and a Cauchy family on the sphere. arXiv preprint / 2015 / circular-circular regression model / Kent, J. T / The Fisher-Bingham distribution on the sphere. Journal of the Royal Statistical Society. Series B / (Methodological), 44(1):71-80, 1982. Kingma, D. P / and Ba, J. Adam / : A method for stochastic optimization. International Conference for Learning Representations, 2015. Kingma, D. P., Salimans, T., Jozefowicz, R., Chen, X., Sutskever, I., and Welling, M / Improved variational inference with inverse autoregressive flow. Advances in Neural Information Processing Systems, 2016. Kobayashi, S. and Nomizu, K / Foundations of differential geometry, volume 1

Publishers / Monte Carlo strategies in scientific computing

Science & Business Media / Directional statistics

Science & Business Media / Series in Probability and Statistics. John / & Sons,. Mardia, K. V., Taylor, C. C., and Subramaniam, G. K / Protein bioinformatics and mixtures of bivariate von Mises

Normalizing Flows on Tori and Spheres A. Density Transformations on Manifolds In this section, we explain how to update the density of a distribution transformed from one Riemannian manifold to another by a smooth map. We only consider the case where both manifolds are sub-manifolds of Euclidean spaces / and N be D-dimensional manifolds embedded into Euclidean spaces Rm and Rn respectively. For example, M and N could be SD embedded in RD+1 as in Section 2.3. Both manifolds inherit a Riemannian metric from their embedding spaces. Let T / be a smooth injective map T : M → N. We / will assume that T can be extended to a smooth map between open neighbourhoods of the embedding spaces that contain M and N, and that we have chosen such an extension. For example, the exponential-map flow in Equation (19) can be written using the coordinates of the embedding space RD+1, and can thus be extended to open neighbourhoods of the embedding spaces as desired. In what follows, we will use the fact that if u1,..., uD are

Normalizing Flows on Tori and Spheres At first, Proposition 1 might seem worrying since the density ratio in that proposition vanishes when r is -1 or 1. So, as r approaches the boundary of the interval [-1, 1], it seems that the correction term to the density will tend to infinity and lead to numerical instability. What saves us is that we do not use Tc→s on its own, and instead combine it with a particular flow transformation on SD-1 × [-1, 1] and the inverse Ts→c, as shown in Equations (12) to (14). In these formulas, the map g is a spline on the interval [-1, 1] which maps -1 to -1, 1 to 1, and has strictly positive slopes g0 (-1) and g0 (1). Looking only at -1 (the case 1 can be similarly dealt with), this means g(-1 + ) ≈ -1 + g0 (-1). As goes to 0, the density corrections coming from Tc→s and Ts→c combine to D 2 - 1

Normalizing Flows on Tori and Spheres Figure 7. Probability density functions of convex combinations of 15 Möbius transformations applied to a uniform base distribution on the circle S1. Each of these distributions required 30 = 15 × 2 parameters. Target Expression Parameters Unimodal pA(θ1, θ2) ∝ exp[cos(θ1 - φ1) + cos(θ2 - φ2)] φ = (4.18, 5.96) Multi-modal pB(θ1, θ2) ∝ 1 3 P3 i=1 pA(θ1, θ2

φi) φ = {(0.21, 2.85), (1.89, 6.18), (3.77, 1.56)} Correlated pC(θ1, θ2) ∝ exp[cos(θ1 + θ2 - φ)] φ = 1.94 Table 2. Target densities used for experiments on T2. Figure 8. Same as Figure 2, with KL and values for the Fourier transforms added. For the Fourier models, the numbers between brackets represent used frequencies, and a number before the bracket means each frequency was repeated. For example, Fourier3[1 - 4] is a Fourier model with 12 frequencies: 3 frequencies of k for each k = 1,..., 4

Normalizing Flows on Tori and Spheres coordinates, and x ∈ R3 is a point on the embedded sphere in Euclidean coordinates. On SU(2) ∼ = S3, the target was a mixture of the same form where µ1 = (1.7, -1.5, 2.3), µ2 = (-3.0, 1.0, 3.0), µ3 = (0.6, -2.6, 4.5), µ4 = (-2.5, 3.0, 5.0), and x ∈ R4 is a point on the embedded sphere in Euclidean coordinates / on S2 The recursive formulas shown in Equations (12) to (14) require choosing a sequence of axes in order to construct the cylindrical coordinate system. This may introduce artifacts to the density related to this choice of axes. To test if this results in numerical problems, we compare the flow from Equations (12) to (14) on a target density that forms a nonaxis-aligned ring against a composition of the same flow with a learned rotation. The results of this experiment are shown in Figure 9. We compared both large (Ks = 32, Km = 12) and small (Ks = 3, Km = 3) versions of the auto-regressive MöbiusSpline flow and observed no significant differences between

Normalizing Flows on Tori and Spheres configurations of this arm are points in T6. The position rk of a joint k = 1,..., 6 of the robot arm is given by rk = rk-1 +  lk cos   X j≤k θj  , lk sin   X j≤k θj    , where r0 = (0, 0) is the position where the arm is affixed

Normalizing Flows on Tori and Spheres