Dropout and DropConnect based Ensemble of Random Vector Functional Link Neural Network

Ensemble of neural networks are widely studied and applied in machine learning. Ensemble reduces the generalization error by reducing either the bias or variance part of the error or both. For ensembles to perform better, the constituent base classifiers should be diverse and accurate. Diversity can be introduced by injecting randomness in the data or architecture. Random Vector Functional Link (RVFL) neural network with closed form solution is a randomized neural network suitable to use as base classifiers of the ensemble because of its extremely fast training time, good generalization and innate randomization in its architecture. Thus, we propose an ensemble of random vector functional link neural networks by introducing additional regularization or randomization in its architecture via well established regularization techniques in the literature namely Dropout and DropConnect. It is evident that stronger randomization helps ensembles generalize better. Based on the experiments on several datasets, we observe that our proposed ensemble performs better than other RVFL based ensembles in most of the datasets.