Scalable regression tree learning on Hadoop using OpenPlanet

As scientific and engineering domains attempt to effectively analyze the deluge of data from sensors and instruments, machine learning is becoming a key data mining tool to build prediction models. Regression tree is a popular learning model that combines decision trees and linear regression to forecast numerical target variables based on a set of input features. MapReduce is well suited for such data intensive learning applications, and a proprietary regression tree algorithm, PLANET, has been proposed earlier on MapReduce. In this paper, we describe an open source implementation of this algorithm, OpenPlanet, on the Hadoop framework using a hybrid approach. We evaluate the performance of OpenPlanet using real world datasets from the Smart Power Grid domain for energy use forecasting, and propose tuning strategies of Hadoop parameters to improve the performance of the default configuration by 75% for a training dataset of 17 million tuples on a 64-core Hadoop cluster on FutureGrid.