Deep neural networks (DNs) provide superhuman performance in numerous computer vision tasks, yet it remains unclear exactly which of a DN's units contribute to a particular decision. A deep network’s prediction cannot be explained in a formal mathematical manner such that you know how all the parameters contribute to the decision. NeuroView is a new family of DN architectures that are explainable by design. Each member of the family is derived from a standard DN architecture by concatenating all of the activations and feeding them into a global linear classifier. The resulting architecture establishes a direct, causal link between the state of each unit and the classification decision. We validate NeuroView on multiple datasets and classification tasks to show that it has on par performance to a typical DN. Also, we inspect how it’s unit/class mapping aids in understanding the decision-making process. In this thesis, we propose using NeuroView in other architectures such as convolutional and recurrent neural networks to show how it can aid in providing additional understanding in applications that need more explanation.