A new approach to select the best subset of predictors in linear regression modelling: bi-objective mixed integer linear programming

Abstract

We study the problem of choosing the best subset of features in linear regression, given observations. This problem naturally contains two objective functions including minimizing the amount of bias and minimizing the number of predictors. The existing approaches transform the problem into a single-objective optimization problem. We explain the main weaknesses of existing approaches and, to overcome their drawbacks, we propose a bi-objective mixed integer linear programming approach. A computational study shows the efficacy of the proposed approach. doi:10.1017/S1446181118000275