• Faster Boosting with Smaller Memory - Julaiti Alafate


  • Abstract:

    Boosting, and in particular gradient boosted trees, are some of the most popular learning algorithms used in practice. There are two highly efficient implementations of boosting, XGBoost and LightGBM, that can process large training sets extremely fast. However, this performance requires that memory size to be sufficient to hold a 2-3 multiple of the training set size. We present an alternative approach to implement boosted trees. It combines two technologies: early stopping and weighted sampling. We discuss the design of our system, and show its significant speed-up over XGBoost and LightGBM, especially in the limited memory settings.

    Joint work with Yoav Freund.