Analyzing robustness of models of chaotic dynamical systems learned from data with Echo state networks

Simple item page
dc.contributor.advisorSubramanian, Devikaen_US
dc.contributor.advisorCartwright, Robert S.en_US
dc.creatorAbdelrahman, Mohamed Mahmoud Hafez Mahmouden_US
dc.date.accessioned2019-11-22T20:24:53Zen_US
dc.date.available2019-11-22T20:24:53Zen_US
dc.date.created2019-12en_US
dc.date.issued2019-11-21en_US
dc.date.submittedDecember 2019en_US
dc.date.updated2019-11-22T20:24:53Zen_US
dc.description.abstractLarge scale engineering as well as natural systems, such as weather, often have high-dimensional state spaces and exhibit chaotic dynamics. To model the behavior of such systems, sets of coupled Partial Differential Equations (PDEs) are formulated and solved using high-performance computing systems. More recently, significant attention has grown toward the use of Artificial Intelligence (AI) and Machine Learning (ML) techniques, in particular, using data-driven modeling to learn fast and accurate surrogate process models trained on high-resolution data obtained from simulations, or observations of chaotic systems. Echo state networks (ESN), a family of recurrent neural network algorithms, have emerged as one of the most promising techniques to learn predictive models of chaotic dynamical systems directly from data. In spite of their success in learning chaotic dynamical systems from data, there are many open questions. Some of them are practical engineering concerns such as: how to choose training parameters (reservoir size, spectral radius, length of training sequence) for specific problems, how robust the learned models are to variations in data, and in training parameters (initialization of random weights, reservoir size, spectral radius). Others are open theoretical questions such as: why do ESNs work at all, in particular, which aspects of the underlying dynamical systems are captured by the learned reservoirs, and which factors determine the prediction horizon of the learned models. In this thesis, we study these practical and theoretical questions in the context of two models of chaotic dynamical systems, Lorenz63 and Lorenz96, which are prototypes of more complex weather models. We show that the predictive performance of the learned models is highly sensitive to initial conditions — i.e., for different training sequences all of the same lengths but with different initial states, there is considerable variation in prediction horizon from 0.1 MTU to 3.8 MTU in Lorenz63 and from 0.4 MTU to 2.8 MTU in Lorenz96. We also show that variations in the initialization of (random) input weights and (random) reservoir weights at the start of the training phase yields models with varying prediction horizon for the very same training sequence. We discuss the implications of these findings in the construction of robust ESN models for Lorenz systems. To help explain the observed variations in predictive performance with initial conditions, and to understand when and why ESNs work, we use dimensionality reduction and clustering algorithms to visualize the evolution of high-dimensional reservoir states during training and prediction. Our main finding is that, in a well-trained model, reservoir states mirror the dynamics of the chaotic system from which the data is derived. In particular, we can infer the number of dynamical components from the non-linear clustering of the reservoir states. In the context of Lorenz63, we show that the sensitivity to initial conditions stems from the locations of the initial condition relative to the two components of the underlying system.en_US
dc.format.mimetypeapplication/pdfen_US
dc.identifier.citationAbdelrahman, Mohamed Mahmoud Hafez Mahmoud. "Analyzing robustness of models of chaotic dynamical systems learned from data with Echo state networks." (2019) Master’s Thesis, Rice University. <a href="https://hdl.handle.net/1911/107733">https://hdl.handle.net/1911/107733</a>.en_US
dc.identifier.urihttps://hdl.handle.net/1911/107733en_US
dc.language.isoengen_US
dc.rightsCopyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder.en_US
dc.subjectESNen_US
dc.subjectchaotic dynamical systemsen_US
dc.subjectLorenz63en_US
dc.subjectLorenz96en_US
dc.subjectdimensionality reductionen_US
dc.subjectUMAPen_US
dc.titleAnalyzing robustness of models of chaotic dynamical systems learned from data with Echo state networksen_US
dc.typeThesisen_US
dc.type.materialTexten_US
thesis.degree.departmentComputer Scienceen_US
thesis.degree.disciplineEngineeringen_US
thesis.degree.grantorRice Universityen_US
thesis.degree.levelMastersen_US
thesis.degree.majorMachine Learningen_US
thesis.degree.nameMaster of Scienceen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
ABDELRAHMAN-DOCUMENT-2019.pdf
Size:
2.99 MB
Format:
Adobe Portable Document Format
Download
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
PROQUEST_LICENSE.txt
Size:
5.85 KB
Format:
Plain Text
Description:
Download
No Thumbnail Available
Name:
LICENSE.txt
Size:
2.61 KB
Format:
Plain Text
Description:
Download
Collections
Rice University Theses and Dissertations