Lu J. Bayesian Matrix Decomposition and Applications 2023

Textbook in PDF format

In 1954, Alston S. Householder published Principles of Numerical Analysis, one of the first modern treatments on matrix decomposition that favored a (block) LU decomposition-the factorization of a matrix into the product of lower and upper triangular matrices. And now, matrix decomposition has become a core technology in machine learning, largely due to the development of the back propagation algorithm in fitting a neural network, and its way of reducing the dimensionality of the data and representing it in a way that is easier for the machine learning algorithms to process. Bayesian matrix decomposition is a relatively new field within the broader area of matrix decomposition and machine learning. The concept of Bayesian matrix decomposition is rooted in Bayesian statistics in which case it combines the principles of Bayesian statistics with matrix decomposition methods to perform matrix factorization. The use of Bayesian methods in matrix decomposition was first introduced in the early 2000s, with the aim of addressing the limitations of traditional matrix factorization techniques, e.g., limited explanatory and predictive performance.
The sole aim of this book is to give a self-contained introduction to concepts and mathematical tools in Bayesian matrix decomposition in order to seamlessly introduce matrix decomposition techniques and their applications in subsequent sections. However, we clearly realize our inability to cover all the useful and interesting results concerning Bayesian matrix decomposition and given the paucity of scope to present this discussion, e.g., the separated analysis of variational inference for conducting the optimization. We refer the reader to literature in the field of Bayesian analysis for a more detailed introduction to the related fields. This book is primarily a summary of purpose, significance of important Bayesian matrix decomposition methods, e.g., real-valued decomposition, nonnegative matrix factorization, Bayesian interpolative decomposition, and the origin and complexity of the methods which shed light on their applications. The mathematical prerequisite is a first course in statistics and linear algebra. Other than this modest background, the development is self-contained, with rigorous proof provided throughout