Empirical risk minimization (ERM) is perhaps the most in uential idea in statistical learning, with applications to nearly all scienti c and technical domains in the form of regression and classi cation models. The growing concerns about the high energy cost of training and the increased prevalence of massive streaming datasets have led many ML practitioners to look for approximate ERM models that can achieve low cost on memory and latency for training. To this end, we propose STORM- Sketch Toward Online Risk Minimization, an online sketching-based method for empirical risk minimization. STORM compresses a data stream into a tiny array of integer counters. This sketch is su cient to estimate a variety of surrogate losses over the original dataset. We provide rigorous theoretical analysis and show that STORM can estimate a carefully chosen surrogate loss for regularized least-squares regression and a margin loss for classi cation. We perform an exhaustive experimental comparison for regression and classi cation training on real-world datasets, achieving an approximate solution with size even less than a data sample.