Browsing by Author "Alston, Brandon"
Now showing 1 - 2 of 2
Results Per Page
Sort Options
Item Bridging the Gap between Operations Research and Machine Learning with Decision Trees and Neural Nets(2024-04-19) Alston, Brandon; Hicks, Illya V.This thesis focuses on bridging the overlap between the fields of Operations Research and Machine Learning. We do so by providing efficient Mixed Integer Linear Optimization (MILO) formulations that solve the optimal decision tree, both the univariate and multivariate cases, and binarized neural nets. We provide four MILO formulations for designing optimal binary classification trees: two flow-based formulations and two cut-based formulations. Given that an optimal binary can be obtained by solving a biobjective optimization problem that seeks to (i) maximize the number of correctly classified datapoints and (ii) minimize the number of branching vertices we also are the first to introduce using a biobjective approach that avoids the numerical issues associated with tuning hyperparameters of weighted objective functions. We use a unique fractional separation procedure to speed up our cut-based models given that MILO solvers often employ reductions only to flow-based models. A binarized neural net, which cannot be trained using gradient descent based backpropagation, can be implemented using Boolean operations and is fundamentally a discrete optimization problem. In particular the fixed network structures, discrete edge weights, and our definition of decision variables allow us to transfer some of the techniques used for decision trees into binarized neural nets. We efficiently model the non-linear properties of neural nets by choosing activation factions to be sign(X). We provide computational results of our proposed models against benchmark methods from the literature.Item Mixed Integer Linear Optimization Formulations for Learning Optimal Binary Classification Trees(2021-11-10) Alston, Brandon; Hicks, Illya V.Decision trees are powerful tools for classification and regression that attract many researchers working in the burgeoning area of machine learning. A classification decision tree has two types of vertices: (i) branching vertices at which datapoints are tested on a selection of discrete features, and (ii) leaf vertices at which datapoints are assigned classes. An optimal binary classification tree is a special type of classification tree in which each branching vertex has exactly two children and can be obtained by solving a biobjective mixed integer linear optimization problem that seeks to minimize the (i) number of misclassified datapoints and (ii) number of branching vertices. In this thesis we present two new multicommodity flow formulations and a new cut-based formulation to learn such optimal binary classification trees. We then provide a comparison of the formulations' strength, valid inequalities to strengthen all formulations, and accompanying computational results.