A well-known conjecture asserts that the mapping class group of a surface (possibly with punctures/boundary) does not virtually surject onto Z if the genus of the surface is large. We prove that if this conjecture holds for some genus, then it also holds for all larger genera. We also prove that if there is a counterexample to this conjecture, then there must be a counterexample of a particularly simple form. We prove these results by relating the conjecture to a family of linear representations of the mapping class group that we call the higher Prym representations. They generalize the classical symplectic representation.