The Hedged Random Forest

ADIA Lab Research Paper Series, No. 14.

Elliot Beck, Damian Kozbur, Michael Wolf (2024)

The random forest is one of the most popular and widely employed tools for supervised machine learning. It can be used for both classification and regression tasks; in this paper, the focus will be on regression only. In its standard form, the crux of the random forest is to use an equal-weighted ensemble of tree-based forecasts. Instead, we suggest a more general weighting scheme that borrows certain ideas from the related problem of financial portfolio selection and, in particular, allows for negative weights. Based on a benchmark collection of real-life data sets, we demonstrate the improved forecasting performance of our method not only relative to the standard random forest but also relative to two previous proposals for weighting the tree-based forecasts. It is noteworthy that our methodology is of a high-level nature and can also be applied to other forecast-combination problems, when forecasting methods are of arbitrary nature and not necessarily tree-based.