Nucleoside analogue drugs are pervasively used as antiviral and chemotherapy agents. Cytarabine and gemcitabine are anti-cancer nucleoside analogue drugs that contain C2′ modifications on the sugar ring. Despite carrying all the required functional groups for DNA synthesis, these two compounds inhibit DNA extension once incorporated into DNA. It remains unclear how the C2′ modifications on cytarabine and gemcitabine affect the polymerase active site during substrate binding and DNA extension. Using steady-state kinetics, static and time-resolved X-ray crystallography with DNA polymerase η (Pol η) as a model system, we showed that the sugar ring C2′ chemical groups on cytarabine and gemcitabine snugly fit within the Pol η active site without occluding the steric gate. During DNA extension, Pol η can extend past gemcitabine but with much lower efficiency past cytarabine. The Pol η crystal structures show that the -OH modification in the β direction on cytarabine locks the sugar ring in an unfavorable C2′-endo geometry for product formation. On the other hand, the addition of fluorine atoms on gemcitabine alters the proper conformational transition of the sugar ring for DNA synthesis. Our study illustrates mechanistic insights into chemotherapeutic drug inhibition and resistance and guides future optimization of nucleoside analogue drugs.