We report on a pump wavelength-dependent study of the electron-phonon relaxation in colloidal solutions of isolated gold nanospheres and nanorods. We varied the pump wavelength from 400 to 800 nm to cover interband, plasmon, and intraband excitations in 40 nm gold nanospheres with a plasmon resonance at ~525 nm, as well as in 34×91 nm gold nanorods with longitudinal plasmon resonance at 720 nm. We also performed an excitation fluence dependence study at each pump wavelength and plotted the measured plasmon bleach recovery kinetics against change in electronic temperature. These plots were all linear and revealed as the intercept of zero excitation power the intrinsic electron-phonon relaxation time, which we found to be independent of pump wavelength and hence the same regardless of how the nanoparticles were excited. Similarly, the slopes of the excitation power dependencies varied remained constant, from taking into account the absorption spectra of each nanoparticle sample. The plasmon bleach recovery dynamics at a given excitation fluence simply describe the collective cooling of a hot thermalized electron distribution with the lattice to reach thermal equilibrium and are longer the larger the initial temperature rise.