Coherent anti-Stokes Raman scattering (CARS) imaging technique has demonstrated great potential in clinical diagnosis by providing cellular-level resolution images without using exogenous contrast agents. This thesis contributes to the formation of an optical fiber based signal collection scheme and an automated image analytics platform to translate CARS microscopy for clinical uses. First, I introduce the concept of CARS by showing original images acquired from thyroid and parathyroid tissues. Second, I describe the use of a customized optical fiber bundle to collect and differentiate forward and backward generated CARS signals that contain different structural information. Third, I demonstrate the feasibility of using deep learning algorithms to characterize and classify CARS images automatically. In particular, I apply transfer learning on the CARS images and achieve 89.2% prediction accuracy in differentiating normal, small-cell carcinoma, adenocarcinoma, and squamous cell carcinoma human lung images. The combination of an optical fiber based microendoscopy and deep learning image classification algorithm will facilitate CARS imaging for on-the-spot cancer diagnosis, allowing medical practitioners to obtain essential information in real time and accelerate clinical decision-making. Meanwhile, the thesis also shows the generality of the deep learning algorithm developed by classifying screening images generated in drug discovery. As an example, for automated classification of large volumes of high-content screening images for Alzheimer’s disease drug discovery, by applying similar transfer learning method on hyperphosphorylated tau images, I categorize drug hits into ineffective, partially-effective, and significantly-effective groups with high speed and accuracy.