Enhancing Exploration in Reinforcement Learning through Multi-Step Actions

dc.contributor.advisorShrivastava, Anshumalien_US
dc.creatorMedini, Tharunen_US
dc.date.accessioned2020-12-10T17:37:28Zen_US
dc.date.available2020-12-10T17:37:28Zen_US
dc.date.created2020-12en_US
dc.date.issued2020-12-03en_US
dc.date.submittedDecember 2020en_US
dc.date.updated2020-12-10T17:37:28Zen_US
dc.description.abstractThe paradigm of Reinforcement Learning (RL) has been plagued by slow and uncertain training owing to the poor exploration in existing techniques. This can be mainly attributed to the lack of training data beforehand. Further, querying a neural network after every step is a wasteful process as some states are conducive to multi-step actions. Since we train with data generated on-the-fly, it is hard to pre-identify certain action sequences that consistently yield great rewards. Prior research in RL has been focused on designing algorithms that can train multiple agents in parallel and accumulate information from these agents to train faster. Concurrently, research has also been done to dynamically identify action sequences that are suited for a specific input state. In this work, we provide insights into the necessity and training methods for RL with multi-step action sequences in conjunction with the main actions of an RL environment. We broadly discuss two approaches. First of them is A4C - Anticipatory Asynchronous Advantage Actor-Critic, a method that squeezes twice the gradients from the same number of episodes and thereby achieves higher scores and converges faster. The second one is an alternative to Imitation Learning that mitigates the need for having state-action pairs of expert. With as few as 20 action trajectories of expert, we can identify the most frequent action pairs and append to the novice's action space. We show the power of our approaches by consistently and significantly outperforming the state-of-the-art GPU-enabled-A3C (GA3C) on popular ATARI games.en_US
dc.format.mimetypeapplication/pdfen_US
dc.identifier.citationMedini, Tharun. "Enhancing Exploration in Reinforcement Learning through Multi-Step Actions." (2020) Master’s Thesis, Rice University. <a href="https://hdl.handle.net/1911/109644">https://hdl.handle.net/1911/109644</a>.en_US
dc.identifier.urihttps://hdl.handle.net/1911/109644en_US
dc.language.isoengen_US
dc.rightsCopyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder.en_US
dc.subjectReinforcement Learningen_US
dc.subjectImitation Learningen_US
dc.subjectMachine Learningen_US
dc.subjectATARIen_US
dc.subjectDeepMinden_US
dc.subjectA3Cen_US
dc.subjectGA3Cen_US
dc.subjectActor Criticen_US
dc.titleEnhancing Exploration in Reinforcement Learning through Multi-Step Actionsen_US
dc.typeThesisen_US
dc.type.materialTexten_US
thesis.degree.departmentElectrical and Computer Engineeringen_US
thesis.degree.disciplineEngineeringen_US
thesis.degree.grantorRice Universityen_US
thesis.degree.levelMastersen_US
thesis.degree.nameMaster of Scienceen_US
Files
Original bundle
Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
MEDINI-DOCUMENT-2020.pdf
Size:
1.93 MB
Format:
Adobe Portable Document Format
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
PROQUEST_LICENSE.txt
Size:
5.84 KB
Format:
Plain Text
Description:
No Thumbnail Available
Name:
LICENSE.txt
Size:
2.61 KB
Format:
Plain Text
Description: