Low rank estimators for higher order statistics are considered in this paper. Rank reduction methods offer a general principle for trading estimator bias for reduced estimator variance. The bias-variance tradeoff is analyzed for low rank estimators of higher order statistics using a tensor product formulation for the moments and cumulants. In general the low rank estimators have a larger bias and smaller variance than the corresponding full rank estimator. Often a tremendous reduction in variance is obtained in exchange for a slight increase in bias. This makes the low rank estimators extremely useful for signal processing algorithms based on sample estimates of the higher order statistics. The low rank estimators also offer considerable reductions in the computational complexity of such algorithms. The design of subspaces to optimize the tradeoffs between bias, variance, and computation is discussed and a noisy input, noisy output system identification problem is used to illustrate the results.