Deep neural networks (DNNs) have proven to be adept at accurately automating many tasks (e.g., image and text classification, object detection, text generation, and more). Across most domains, DNNs tend to achieve better performance with increasing scale, both in terms of dataset and model size. As such, the benefit of DNNs comes at a steep computational (and monetary) cost, which can limit their applicability. This document aims to identify novel and intuitive techniques that can make deep learning more usable across domains and communities.