Patel, Ankit B.Nguyen, TanBaraniuk, Richard G.2017-01-092017-01-092016Patel, Ankit B., Nguyen, Tan and Baraniuk, Richard G.. "A Probabilistic Framework for Deep Learning." <i>Advances in Neural Information Processing Systems 29,</i> (2016) Neural Information Processing Systems Foundation, Inc.: <a href="https://hdl.handle.net/1911/93745">https://hdl.handle.net/1911/93745</a>.https://hdl.handle.net/1911/93745NEWS COVERAGE: A news release based on this journal publication is available online: http://news.rice.edu/2016/12/16/rice-baylor-team-sets-new-mark-for-deep-learning/We develop a probabilistic framework for deep learning based on the Deep Rendering Mixture Model (DRMM), a new generative probabilistic model that explicitly capture variations in data due to latent task nuisance variables. We demonstrate that max-sum inference in the DRMM yields an algorithm that exactly reproduces the operations in deep convolutional neural networks (DCNs), providing a first principles derivation. Our framework provides new insights into the successes and shortcomings of DCNs as well as a principled route to their improvement. DRMM training via the Expectation-Maximization (EM) algorithm is a powerful alternative to DCN back-propagation, and initial training results are promising. Classification based on the DRMM and other variants outperforms DCNs in supervised digit classification, training 2-3x faster while achieving similar accuracy. Moreover, the DRMM is applicable to semi-supervised and unsupervised learning tasks, achieving results that are state-of-the-art in several categories on the MNIST benchmark and comparable to state of the art on the CIFAR10 benchmark.engJournal article