The ability to speak and understand language is consciously a fast and easy process. However, the language system can err, either in normal processes or as a result of neural damage following stroke. Often, in both production and comprehension, errors are semantically related to the intended word, such as saying or understanding “cat” when the intended meaning is “dog”. This semantic interference (SI) effect suggests that the processing stages involved in language production and comprehension overlap to some extent. However, because language production and comprehension are usually investigated separately, this has led to different conclusions about how SI arises in each language modality. By most accounts, SI in production occurs at the lexical-semantic level, whereas SI in comprehension arises within the semantic system itself. In this dissertation, I distinguish between SI in production and comprehension by examining how (cognitive mechanisms) and where (neural loci) SI arises during picture naming and word-picture matching tasks that elicit SI by manipulating the semantic context with which target items appear. Aim I of my dissertation directly compared the behavioral characteristics of SI in healthy participants’ production and comprehension performance in order to elucidate the level and cognitive mechanism by which SI arises in each language modality. Aim II explored patients’ susceptibility to SI as it related to cortical gray matter and subcortical white matter damage. The results provided converging evidence that not only do the SI characteristics differ in production and comprehension, but also the neural locus of SI differs across language modality. However, the time course of SI is similar in both language modalities. Accordingly, I conclude SI arises when mapping meanings with words in production vs. mapping words with meanings in comprehension, but that the same cognitive mechanism operates over lexical-semantic processes across modalities. In the end, I argue that because of inherent differences between the order with which lexical and semantic representations are accessed in production vs. comprehension, the mechanism produces different behavioral manifestations of SI in each language modality and places differential demands on cognitive control mechanisms required to resolve interference.