Optimized Runtime Systems for MapReduce Applications in Multi-core Clusters
| dc.contributor.advisor | Sarkar, Vivek | en_US |
| dc.contributor.committeeMember | Cox, Alan | en_US |
| dc.contributor.committeeMember | Mellor-Crummey, John | en_US |
| dc.creator | Zhang, Yunming | en_US |
| dc.date.accessioned | 2016-01-07T22:00:11Z | en_US |
| dc.date.available | 2016-01-07T22:00:11Z | en_US |
| dc.date.created | 2014-12 | en_US |
| dc.date.issued | 2014-05-27 | en_US |
| dc.date.submitted | December 2014 | en_US |
| dc.date.updated | 2016-01-07T22:00:11Z | en_US |
| dc.description.abstract | This research proposes a novel runtime system, Habanero Hadoop, to tackle the inefficient utilization of multi-core machines' memory in the existing Hadoop MapReduce runtime system. Insufficient memory for each map task leads to the inability to tackle large-scale problems such as genome sequencing and data clustering. The Habanero Hadoop system integrates a shared memory model into the fully distributed memory model of the Hadoop MapReduce system. The improvements eliminate duplication of in-memory data structures used in the map phase, making more memory available to each map task. Previous works optimizing multi-core performance for MapReduce runtime focused on maximizing CPU utilization rather than memory efficiency. My work provided multiple approaches to significantly improve the memory efficiency of the Hadoop MapReduce runtime. The optimized Habanero Hadoop runtime can increase the throughput and maximum input size for certain widely used data analytics applications such as Kmeans and Hash Join by 2x. | en_US |
| dc.format.mimetype | application/pdf | en_US |
| dc.identifier.citation | Zhang, Yunming. "Optimized Runtime Systems for MapReduce Applications in Multi-core Clusters." (2014) Master’s Thesis, Rice University. <a href="https://hdl.handle.net/1911/87782">https://hdl.handle.net/1911/87782</a>. | en_US |
| dc.identifier.uri | https://hdl.handle.net/1911/87782 | en_US |
| dc.language.iso | eng | en_US |
| dc.rights | Copyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder. | en_US |
| dc.subject | MapReduce | en_US |
| dc.subject | Memory | en_US |
| dc.title | Optimized Runtime Systems for MapReduce Applications in Multi-core Clusters | en_US |
| dc.type | Thesis | en_US |
| dc.type.material | Text | en_US |
| thesis.degree.department | Computer Science | en_US |
| thesis.degree.discipline | Engineering | en_US |
| thesis.degree.grantor | Rice University | en_US |
| thesis.degree.level | Masters | en_US |
| thesis.degree.name | Master of Science | en_US |
Files
Original bundle
1 - 1 of 1