Bridging the Gap between Operations Research and Machine Learning with Decision Trees and Neural Nets

Simple item page
dc.contributor.advisorHicks, Illya V.en_US
dc.creatorAlston, Brandonen_US
dc.date.accessioned2024-05-21T21:53:05Zen_US
dc.date.available2024-05-21T21:53:05Zen_US
dc.date.created2024-05en_US
dc.date.issued2024-04-19en_US
dc.date.submittedMay 2024en_US
dc.date.updated2024-05-21T21:53:05Zen_US
dc.description.abstractThis thesis focuses on bridging the overlap between the fields of Operations Research and Machine Learning. We do so by providing efficient Mixed Integer Linear Optimization (MILO) formulations that solve the optimal decision tree, both the univariate and multivariate cases, and binarized neural nets. We provide four MILO formulations for designing optimal binary classification trees: two flow-based formulations and two cut-based formulations. Given that an optimal binary can be obtained by solving a biobjective optimization problem that seeks to (i) maximize the number of correctly classified datapoints and (ii) minimize the number of branching vertices we also are the first to introduce using a biobjective approach that avoids the numerical issues associated with tuning hyperparameters of weighted objective functions. We use a unique fractional separation procedure to speed up our cut-based models given that MILO solvers often employ reductions only to flow-based models. A binarized neural net, which cannot be trained using gradient descent based backpropagation, can be implemented using Boolean operations and is fundamentally a discrete optimization problem. In particular the fixed network structures, discrete edge weights, and our definition of decision variables allow us to transfer some of the techniques used for decision trees into binarized neural nets. We efficiently model the non-linear properties of neural nets by choosing activation factions to be sign(X). We provide computational results of our proposed models against benchmark methods from the literature.en_US
dc.format.mimetypeapplication/pdfen_US
dc.identifier.citationAlston, Brandon. Bridging the Gap between Operations Research and Machine Learning with Decision Trees and Neural Nets. (2024). https://hdl.handle.net/1911/116142en_US
dc.identifier.urihttps://hdl.handle.net/1911/116142en_US
dc.language.isoengen_US
dc.rightsCopyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder.en_US
dc.subjectmachine learningen_US
dc.subjectmixed integer programmingen_US
dc.subjectcombinatorial optimizationen_US
dc.subjectgraph theoryen_US
dc.subjectdecision treeen_US
dc.subjectneural networken_US
dc.titleBridging the Gap between Operations Research and Machine Learning with Decision Trees and Neural Netsen_US
dc.typeThesisen_US
dc.type.materialTexten_US
thesis.degree.departmentComputational and Applied Mathematicsen_US
thesis.degree.disciplineEngineeringen_US
thesis.degree.grantorRice Universityen_US
thesis.degree.levelDoctoralen_US
thesis.degree.nameDoctor of Philosophyen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
ALSTON-DOCUMENT-2024.pdf
Size:
5.87 MB
Format:
Adobe Portable Document Format
Download
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
PROQUEST_LICENSE.txt
Size:
5.84 KB
Format:
Plain Text
Description:
Download
No Thumbnail Available
Name:
LICENSE.txt
Size:
2.98 KB
Format:
Plain Text
Description:
Download
Collections
Rice University Theses and Dissertations