User Manual Privacy Policy Disclaimer Contact us
  Advanced SearchBrowse




Book Chapter

Projected Newton-type methods in machine learning


Sra,  S.
Dept. Empirical Inference, Max Planck Institute for Intelligent Systems, Max Planck Society;

Fulltext (public)
There are no public fulltexts available
Supplementary Material (public)
There is no public supplementary material available

Schmidt, M., Kim, D., & Sra, S. (2011). Projected Newton-type methods in machine learning. In S. Sra, S. Nowozin, & S. Wright (Eds.), Optimization for Machine Learning (pp. 305-330). Cambridge, MA, USA: MIT Press.

Cite as: http://hdl.handle.net/11858/00-001M-0000-0013-B8DE-7
We consider projected Newton-type methods for solving large-scale optimization problems arising in machine learning and related fields. We first introduce an algorithmic framework for projected Newton-type methods by reviewing a canonical projected (quasi-)Newton method. This method, while conceptually pleasing, has a high computation cost per iteration. Thus, we discuss two variants that are more scalable, namely, two-metric projection and inexact projection methods. Finally, we show how to apply the Newton-type framework to handle non-smooth objectives. Examples are provided throughout the chapter to illustrate machine learning applications of our framework.