BYY Independent State Space and Harmony Learning
(L. Xu)

We develop a so called Bayesian Ying Yang (BYY) independent state space (ISS) system that provides a general statistical framework for parameter learning, regularization and model selection on various independent factor analyses (FA) related learning tasks. On three typical architectures of the system, adaptive algorithms, regularization methods and model selection criteria are provided for either or both of parameter learning with automated model selection and parameter learning followed by model selection. In the backward architectures, new results are provided for Gaussian and non-Gaussian FA, binary FA, independent Hidden Markov Model and Temporal FA, as well as extensions. In the forward architectures, adaptive algorithms are given for several extensions of independent component analysis (ICA), including competitive ICA, Gaussian and non-Gaussian temporal ICA. Moreover, the bi-directional architectures brings not only with new strength to the existing LMSER learning, but also with various LMSER extensions, including the so called principal ICA and its temporal extension.