Towards a Surrogate-Assisted Multi-Objective Full Model Selection

This research approaches the full model selection problem. The full model selection problem is defined as a method in which, given a pool of pre-processing methods, feature selection and learning algorithms, to choose from, we select a combination of them, together with their hyper-parameters,in such a way, that we can provide the “best” generalization performance on a given dataset. We propose to face this as a multi-objective optimization problem, where the classification-error and the model complexity are defined as the objectives to be minimized. We propose to use a surrogate-assisted multi-objective evolutionary algorithm approach to explore the models space. Our proposal derives from the fact that estimating the values of the objective function could be computationally expensive. Therefore, by using surrogate-assisted optimization we expect to reduce the number of full models that should be trained and tested so that we can reduce the total number of fitness function evaluations, without degrading, in a significant manner, the quality of the models. Our preliminary results give evidence of the validity of our proposed approach.