We study the spectra of the molecular orbital Hessian (stability matrix) and random-phase approximation (RPA) Hamiltonian of broken-symmetry Hartree-Fock solutions, focusing on zero eigenvalue modes. After all negative eigenvalues are removed from the Hessian by following their eigenvectors downhill, one is left with only positive and zero eigenvalues. Zero modes correspond to orbital rotations with no restoring force. These rotations determine states in the Goldstone manifold, which originates from a spontaneously broken continuous symmetry in the wave function. Zero modes can be classified as improper or proper according to their different mathematical and physical properties. Improper modes arise from symmetry breaking and their restoration always lowers the energy. Proper modes, on the other hand, correspond to degeneracies of the wave function, and their symmetry restoration does not necessarily lower the energy. We discuss how the RPA Hamiltonian distinguishes between proper and improper modes by doubling the number of zero eigenvalues associated with the latter. Proper modes in the Hessian always appear in pairs which do not double in RPA. We present several pedagogical cases exemplifying the above statements. The relevance of these results for projected Hartree-Fock methods is also addressed.