In this thesis, we propose nonparametric statistical methods to study the derivatives and localized features of functions, areas of paramount importance across diverse scientific disciplines such as cosmology, environmental science, and neuroscience. We first develop a plug-in kernel ridge regression estimator for derivatives of arbitrary order. We provide both non-asymptotic and asymptotic error bounds for this estimator and show its substantial improvements over existing techniques, particularly for high-order derivatives and multi-dimensional settings. We then transition to exploring the application of Gaussian processes (GPs) in modeling derivative functionals, addressing a long-standing skepticism surrounding derivative estimation strategies using GPs. We show that the GP prior exhibits a remarkable plug-in property with minimax optimality---this, to the best of our knowledge, provides the first positive result for using GPs in estimating function derivatives. A data-driven empirical Bayes approach is studied for hyperparameter tuning, which achieves theoretical optimality and computational efficiency. Lastly, we tackle the challenging problem of identifying and analyzing localized features of functions, such as local extrema, in the presence of noise. To this end, we introduce a semiparametric Bayesian method built upon derivative-constrained GP priors. We establish large sample frequentist properties for the proposed method, from which point and interval estimates are derived, offering the advantage of fast implementation without requiring complex sampling procedures. Overall, this thesis blends theoretical and methodological innovation with practical application, aiming to equip researchers and practitioners with more powerful and versatile tools for the analysis of nonparametric functions. The advancements presented herein not only contribute to the statistical community but also have broad implications across various fields where derivative estimation and localized feature analysis are essential. Our methods are illustrated via a climate change application for analyzing the global sea-level rise, and an application to cognitive science for analyzing event-related potentials.