Chauhan V. Stochastic Optimization...Machine Learning 2022

Textbook in PDF format

Advancements in the technology and availability of data sources have led to the `Big Data' era. Working with large data offers the potential to uncover more fine-grained patterns and take timely and accurate decisions, but it also creates a lot of challenges such as slow training and scalability of machine learning models. One of the major challenges in machine learning is to develop efficient and scalable learning algorithms, i.e., optimization techniques to solve large scale learning problems.
Stochastic Optimization for Large-scale Machine Learning identifies different areas of improvement and recent research directions to tackle the challenge. Developed optimisation techniques are also explored to improve machine learning algorithms based on data access and on first and second order optimisation methods.
Key Features
Bridges machine learning and Optimisation.
Bridges theory and practice in machine learning.
Identifies key research areas and recent research directions to solve large-scale machine learning problems.
Develops optimisation techniques to improve machine learning algorithms for big data problems.
BACKGROUND
Introduction
Optimization Problem, Solvers,Challenges and Research Directions
FIRST ORDERMETHODS
Mini-batch and Block-coordinate Approach
Variance Reduction Methods
Learning and Data Access
SECOND ORDERMETHODS
Mini-batch Block-coordinate Newton Method
Stochastic Trust Region Inexact Newton Method
CONCLUSION
Conclusion and Future Scope