Bridging the Gap between Operations Research and Machine Learning with Decision Trees and Neural Nets

Simple item page
dc.contributor.advisorHicks, Illya V.
dc.creatorAlston, Brandon
dc.date.accessioned2024-05-21T21:53:05Z
dc.date.available2024-05-21T21:53:05Z
dc.date.created2024-05
dc.date.issued2024-04-19
dc.date.submittedMay 2024
dc.date.updated2024-05-21T21:53:05Z
dc.description.abstractThis thesis focuses on bridging the overlap between the fields of Operations Research and Machine Learning. We do so by providing efficient Mixed Integer Linear Optimization (MILO) formulations that solve the optimal decision tree, both the univariate and multivariate cases, and binarized neural nets. We provide four MILO formulations for designing optimal binary classification trees: two flow-based formulations and two cut-based formulations. Given that an optimal binary can be obtained by solving a biobjective optimization problem that seeks to (i) maximize the number of correctly classified datapoints and (ii) minimize the number of branching vertices we also are the first to introduce using a biobjective approach that avoids the numerical issues associated with tuning hyperparameters of weighted objective functions. We use a unique fractional separation procedure to speed up our cut-based models given that MILO solvers often employ reductions only to flow-based models. A binarized neural net, which cannot be trained using gradient descent based backpropagation, can be implemented using Boolean operations and is fundamentally a discrete optimization problem. In particular the fixed network structures, discrete edge weights, and our definition of decision variables allow us to transfer some of the techniques used for decision trees into binarized neural nets. We efficiently model the non-linear properties of neural nets by choosing activation factions to be sign(X). We provide computational results of our proposed models against benchmark methods from the literature.
dc.format.mimetypeapplication/pdf
dc.identifier.citationAlston, Brandon. Bridging the Gap between Operations Research and Machine Learning with Decision Trees and Neural Nets. (2024). https://hdl.handle.net/1911/116142
dc.identifier.urihttps://hdl.handle.net/1911/116142
dc.language.isoeng
dc.rightsCopyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder.
dc.subjectmachine learning
dc.subjectmixed integer programming
dc.subjectcombinatorial optimization
dc.subjectgraph theory
dc.subjectdecision tree
dc.subjectneural network
dc.titleBridging the Gap between Operations Research and Machine Learning with Decision Trees and Neural Nets
dc.typeThesis
dc.type.materialText
thesis.degree.departmentComputational and Applied Mathematics
thesis.degree.disciplineEngineering
thesis.degree.grantorRice University
thesis.degree.levelDoctoral
thesis.degree.nameDoctor of Philosophy
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
ALSTON-DOCUMENT-2024.pdf
Size:
5.87 MB
Format:
Adobe Portable Document Format
Download
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
PROQUEST_LICENSE.txt
Size:
5.84 KB
Format:
Plain Text
Description:
Download
No Thumbnail Available
Name:
LICENSE.txt
Size:
2.98 KB
Format:
Plain Text
Description:
Download
Collections
Rice University Electronic Theses and Dissertations