Sparse identification of non-linear dynamics

Sparse identification of nonlinear dynamics (SINDy) is a data-driven algorithm for obtaining dynamical systems from data. Given a series of snapshots of a dynamical system and its corresponding time derivatives, SINDy performs a sparsity-promoting regression (such as LASSO and sparse Bayesian inference) on a library of nonlinear candidate functions of the snapshots against the derivatives to find the governing equations. This procedure relies on the assumption that most physical systems only have a few dominant terms which dictate the dynamics, given an appropriately selected coordinate system and quality training data. It has been applied to identify the dynamics of fluids, based on proper orthogonal decomposition, as well as other complex dynamical systems, such as biological networks.

Mathematical Overview

First, consider a dynamical system of the form

$\dot{𝐱} = \frac{d}{d t} 𝐱 (t) = 𝐟 (𝐱 (t)),$

where $𝐱 (t) \in ℝ^{n}$ is a state vector (snapshot) of the system at time $t$ and the function $𝐟 (𝐱 (t))$ defines the equations of motion and constraints of the system. The time derivative may be either prescribed or numerically approximated from the snapshots.

With $𝐱$ and $\dot{𝐱}$ sampled at $m$ equidistant points in time ( $t_{1}, t_{2}, \dots, t_{m}$ ), these can be arranged into matrices of the form

$𝐗 = [\begin{matrix} 𝐱^{𝖳} (𝐭_{𝟏}) \\ 𝐱^{𝖳} (𝐭_{𝟐}) \\ ⋮ \\ 𝐱^{𝖳} (𝐭_{𝐦}) \end{matrix}] = [\begin{matrix} 𝐱_{𝟏} (𝐭_{𝟏}) & 𝐱_{𝟐} (𝐭_{𝟏}) & \dots & 𝐱_{𝐧} (𝐭_{𝟏}) \\ 𝐱_{𝟏} (𝐭_{𝟐}) & 𝐱_{𝟐} (𝐭_{𝟐}) & \dots & 𝐱_{𝐧} (𝐭_{𝟐}) \\ ⋮ & ⋮ & ⋱ & ⋮ \\ 𝐱_{𝟏} (𝐭_{𝐦}) & 𝐱_{𝟐} (𝐭_{𝐦}) & \dots & 𝐱_{𝐧} (𝐭_{𝐦}) \end{matrix}],$

and similarly for $\dot{𝐗}$ .

Next, a library $𝜣 (𝐗)$ of nonlinear candidate functions of the columns of $𝐗$ is constructed, which may be constant, polynomial, or more exotic functions (like trigonometric and rational terms, and so on):

$𝜣 (𝐗) = [\begin{matrix} | & | & | & | & | & | \\ 𝟏 & 𝐗 & 𝐗^{𝟐} & 𝐗^{𝟑} & \dots & \sin (𝐗) & \cos (𝐗) & \dots \\ | & | & | & | & | & | \end{matrix}]$

The number of possible model structures from this library is combinatorially high. $𝐟 (𝐱 (t))$ is then substituted by $𝜣 (𝐗)$ and a vector of coefficients $𝜩 = [𝝃_{𝟏} 𝝃_{𝟐} \dots 𝝃_{𝐧}]$ determining the active terms in $𝐟 (𝐱 (t))$ :

$\dot{𝐗} = 𝜣 (𝐗) 𝜩$

Because only a few terms are expected to be active at each point in time, an assumption is made that $𝐟 (𝐱 (t))$ admits a sparse representation in $𝜣 (𝐗)$ . This then becomes an optimization problem in finding a sparse $𝜩$ which optimally embeds $\dot{𝐗}$ . In other words, a parsimonious model is obtained by performing least squares regression on the system (4) with sparsity-promoting ( $L_{1}$ ) regularization

$𝝃_{𝐤} = \underset{𝝃'_{𝐤}}{\arg min} {| | {\dot{𝐗}}_{k} - 𝜣 (𝐗) 𝝃'_{𝐤} | |}_{𝟐} + 𝝀 {| | 𝝃'_{𝐤} | |}_{𝟏},$

where $λ$ is a regularization parameter. Finally, the sparse set of $𝝃_{𝐤}$ can be used to reconstruct the dynamical system:

${\dot{x}}_{k} = 𝜣 (𝐱) 𝝃_{𝐤}$

References

↑ Brunton, Steven L.; Kutz, J. Nathan (2022-05-05). Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control. Higher Education from Cambridge University Press. doi:10.1017/9781009089517. ISBN 9781009089517. Retrieved 2022-10-25.
↑ Huang, Yunfei.; et al. (2022). "Sparse inference and active learning of stochastic differential equations from data". Scientific Reports. 12 (1) 21691. arXiv:2203.11010. Bibcode:2022NatSR..1221691H. doi:10.1038/s41598-022-25638-9. PMC 9755218. PMID 36522347.
↑ Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan (2016-04-12). "Discovering governing equations from data by sparse identification of nonlinear dynamical systems". Proceedings of the National Academy of Sciences. 113 (15): 3932–3937. arXiv:1509.03580. Bibcode:2016PNAS..113.3932B. doi:10.1073/pnas.1517384113. ISSN 0027-8424. PMC 4839439. PMID 27035946.
↑ Mangan, Niall M.; Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan (2016-05-26). "Inferring biological networks by sparse identification of nonlinear dynamics". arXiv:1605.08368 [math.DS].

[1] Brunton, Steven L.; Kutz, J. Nathan (2022-05-05). Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control. Higher Education from Cambridge University Press. doi:10.1017/9781009089517. ISBN 9781009089517. Retrieved 2022-10-25.

[2] Huang, Yunfei.; et al. (2022). "Sparse inference and active learning of stochastic differential equations from data". Scientific Reports. 12 (1) 21691. arXiv:2203.11010. Bibcode:2022NatSR..1221691H. doi:10.1038/s41598-022-25638-9. PMC 9755218. PMID 36522347.

[3] Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan (2016-04-12). "Discovering governing equations from data by sparse identification of nonlinear dynamical systems". Proceedings of the National Academy of Sciences. 113 (15): 3932–3937. arXiv:1509.03580. Bibcode:2016PNAS..113.3932B. doi:10.1073/pnas.1517384113. ISSN 0027-8424. PMC 4839439. PMID 27035946.

[4] Mangan, Niall M.; Brunton, Steven L.; Proctor, Joshua L.; Kutz, J. Nathan (2016-05-26). "Inferring biological networks by sparse identification of nonlinear dynamics". arXiv:1605.08368 [math.DS].