The precise mechanism of light emission from metallic plasmonic nanoparticles is a contested issue. Photoluminescence and electronic Raman scattering are two different views proposed to explain the observed emission from plasmonic nanostructures. The Stokes and anti-Stokes emission spectroscopy studies of single gold nanorods are reported. A model was developed to quantitatively simulate the experimental emission spectra of gold nanorods. Correlated emission spectra and quantum yields of single gold nanorods with different excitation wavelengths and varied excitation powers demonstrate the underlying effects of hot carrier dynamics following interband and intraband transitions. The plasmonic resonance quality factors, modified by nanorod crystallinity, determine the Purcell enhancement effect and have a linear relation with the emission quantum yield. The central role of hot carriers in the systematical study on gold nanorods as a model system supports a Purcell effect enhanced hot carrier photoluminescence mechanism. The emission spectra of single copper and gold nanocubes are compared, unveiling the synergistic effects of interband transitions and plasmonic enhancement. The excitation wavelength dependent quantum yields are explained by the energy dependent lifetime of d-band holes.