To make weather and climate models computationally affordable, small‐scale processes are usually represented in terms of the large‐scale, explicitly resolved processes using physics‐based/semi‐empirical parameterization schemes. Another approach, computationally more demanding but often more accurate, is super‐parameterization (SP). SP involves integrating the equations of small‐scale processes on high‐resolution grids embedded within the low‐resolution grid of large‐scale processes. Recently, studies have used machine learning (ML) to develop data‐driven parameterization (DD‐P) schemes. Here, we propose a new approach, data‐driven SP (DD‐SP), in which the equations of the small‐scale processes are integrated data‐drivenly (thus inexpensively) using ML methods such as recurrent neural networks. Employing multiscale Lorenz 96 systems as the testbed, we compare the cost and accuracy (in terms of both short‐term prediction and long‐term statistics) of parameterized low‐resolution (PLR) SP, DD‐P, and DD‐SP models. We show that with the same computational cost, DD‐SP substantially outperforms PLR and is more accurate than DD‐P, particularly when scale separation is lacking. DD‐SP is much cheaper than SP, yet its accuracy is the same in reproducing long‐term statistics (climate prediction) and often comparable in short‐term forecasting (weather prediction). We also investigate generalization: when models trained on data from one system are applied to a more chaotic system, we find that models often do not generalize, particularly when short‐term prediction accuracies are examined. However, we show that transfer learning, which involves re‐training the data‐driven model with a small amount of data from the new system, significantly improves generalization. Potential applications of DD‐SP and transfer learning in climate/weather modeling are discussed.