Quantifying the frequency of shifts to new host plants within diverse clades of specialist herbivorous insects is critically important to understand whether and how host shifts contribute to the origin of species. Oak gall wasps (Hymenoptera: Cynipidae: Cynipini) comprise a tribe of ∼1000 species of phytophagous insects that induce gall formation on various organs of trees in the family Fagacae—primarily the oaks (genus Quercus; ∼435 sp.). The association of oak gall wasps with oaks is ancient (∼50 my), and most oak species are galled by one or more gall wasp species. Despite the diversity of both gall wasp species and their plant associations, previous phylogenetic work has not identified the strong signal of host plant shifting among oak gall wasps that has been found in other phytophagous insect systems. However, most emphasis has been on the Western Palearctic and not the Nearctic where both oaks and oak gall wasps are considerably more species rich. We collected 86 species of Nearctic oak gall wasps from most of the major clades of Nearctic oaks and sequenced >1000 Ultraconserved Elements (UCEs) and flanking sequences to infer wasp phylogenies. We assessed the relationships of Nearctic gall wasps to one another and, by leveraging previously published UCE data, to the Palearctic fauna. We then used phylogenies to infer historical patterns of shifts among host tree species and tree organs. Our results indicate that oak gall wasps have moved between the Palearctic and Nearctic at least four times, that some Palearctic wasp clades have their proximate origin in the Nearctic, and that gall wasps have shifted within and between oak tree sections, subsections, and organs considerably more often than previous data have suggested. Given that host shifts have been demonstrated to drive reproductive isolation between host-associated populations in other phytophagous insects, our analyses of Nearctic gall wasps suggest that host shifts are key drivers of speciation in this clade, especially in hotspots of oak diversity. Although formal assessment of this hypothesis requires further study, two putatively oligophagous gall wasp species in our dataset show signals of host-associated genetic differentiation unconfounded by geographic distance, suggestive of barriers to gene flow associated with the use of alternative host plants.