We present a quantitative analysis of the electron transfer between single gold nanorods and monolayer graphene under no electrical bias. Using single-particle dark-field scattering and photoluminescence spectroscopy to access the homogeneous linewidth, we observe broadening of the surface plasmon resonance for gold nanorods on graphene compared to nanorods on a quartz substrate. Because of the absence of spectral plasmon shifts, dielectric interactions between the gold nanorods and graphene are not important and we instead assign the plasmon damping to charge transfer between plasmon-generated hot electrons and the graphene that acts as an efficient acceptor. Analysis of the plasmon linewidth yields an average electron transfer time of 160 ± 30 fs, which is otherwise difficult to measure directly in the time domain with single-particle sensitivity. In comparison to intrinsic hot electron decay and radiative relaxation, we furthermore calculate from the plasmon linewidth that charge transfer between the gold nanorods and the graphene support occurs with an efficiency of ∼10%. Our results are important for future applications of light harvesting with metal nanoparticle plasmons and efficient hot electron acceptors as well as for understanding hot electron transfer in plasmon-assisted chemical reactions.