Announcements‎ > ‎

Embedding Modeling

There is a clear distinction between modeling a system from observed data and measuring a specific set of features from the data set. Ideally, modeling should be approached without any pre-conception about the system's architecture. The training data should stand for the unique relevant source of information from which our task is to derive as much knowledge and understanding about the system's mechanism as possible. Conversely, measuring a specific feature from input data implies the prior knowledge of its nature. Ironically though, these two tasks are traditionally so closely related that their distinction resides only in their purposes and not all that much in their implementation or mechanism.

Until recently, linear system theory was the only modeling tool available and its extensive use made us forget about the strong assumptions it relies upon. The notion of a deterministic (or predictable) process that was introduced by Wold's decomposition characterizes only a subclass of deterministic systems: deterministic and linear systems (the future is a linear combination of the past). Through this decomposition a process that may appear non-deterministic or even purely non-deterministic, is not guaranteed to be stochastic at all. It might be the chaotic output of a non-linear deterministic dynamical system. The estimates of the second order statistics of a deterministic, but chaotic, system can be amazingly similar to the ones of a random white noise.

Inferring non-linear models from observed data without any pre-conception concerning the architecture of an eventual model is no longer a dream. Floris Takens' 1981 Embedding theorem can be applied to time-series and lead to a general scheme for the inference of physically meaningful models from observed behaviors.

Modeling a time series as a non-linear hyper-surface in a lag-space was the inspiration behind Cluster-Based Embedding Modeling, which has since been the object of a patent and a letter to Nature.


[1] E. Metois. Determinism and stochasticity for sound modeling. Reflection on the parallels between classical linear system theory and non-linear dynamics, 1995.

[2] E. Metois. Recursive estimation of global models for embedding surfaces. 1995.

[3] E. Metois. Musical Sound Information: Musical Gestures and Embedding Synthesis. PhD thesis, Massachusetts Institute of Technology, 1997.