Neural Network Verification as Piecewise Linear Optimization: Formulations for the Composition of Staircase Functions
| dc.contributor.advisor | Schaefer, Andrew J | en_US |
| dc.creator | Nguyen, Tu Anh | en_US |
| dc.date.accessioned | 2024-08-30T15:53:21Z | en_US |
| dc.date.available | 2024-08-30T15:53:21Z | en_US |
| dc.date.created | 2024-08 | en_US |
| dc.date.issued | 2024-05-16 | en_US |
| dc.date.submitted | August 2024 | en_US |
| dc.date.updated | 2024-08-30T15:53:21Z | en_US |
| dc.description.abstract | We present a technique for neural network verification using mixed-integer programming (MIP) formulations. We derive a \emph{strong formulation} for each neuron in a network using piecewise linear activation functions. Additionally, as in general, these formulations may require an exponential number of inequalities, we also derive a separation procedure that runs in super-linear time in the input dimension. We first introduce and develop our technique on the class of \emph{staircase} functions, which generalizes the ReLU, binarized, and quantized activation functions. We then use results for staircase activation functions to obtain a separation method for general piecewise linear activation functions. Empirically, using our strong formulation and separation technique, we can reduce the computational time in exact verification settings based on MIP and improve the false negative rate for inexact verifiers relying on the relaxation of the MIP formulation. While originally developed for neural network formulation, the MIP formulation and its technical results draw heavily on classical theory in linear optimization, and may be of independent interest to other applications. | en_US |
| dc.format.mimetype | application/pdf | en_US |
| dc.identifier.citation | Nguyen, Tu Anh. Neural Network Verification as Piecewise Linear Optimization: Formulations for the Composition of Staircase Functions. (2024). Masters thesis, Rice University. https://hdl.handle.net/1911/117765 | en_US |
| dc.identifier.uri | https://hdl.handle.net/1911/117765 | en_US |
| dc.language.iso | eng | en_US |
| dc.rights | Copyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder. | en_US |
| dc.subject | Strong MIP Formulation | en_US |
| dc.subject | Neural Networks | en_US |
| dc.title | Neural Network Verification as Piecewise Linear Optimization: Formulations for the Composition of Staircase Functions | en_US |
| dc.type | Thesis | en_US |
| dc.type.material | Text | en_US |
| thesis.degree.department | Computational and Applied Mathematics | en_US |
| thesis.degree.discipline | Engineering | en_US |
| thesis.degree.grantor | Rice University | en_US |
| thesis.degree.level | Masters | en_US |
| thesis.degree.name | Master of Arts | en_US |
Files
Original bundle
1 - 1 of 1