This thesis considers the role of locality and sampling in graph signal processing and network science. In light of the prevalence of extremely large and complex network datasets, it is a timely problem to consider how the study of these objects can be reduced to the study of a distribution of simpler objects. Indeed, many methods in graph signal processing and machine learning on graphs can be described strictly in light of local graph substructures. The approach taken to this problem starts with defining the notion of taking a sample from a network, and then builds this out to a probabilistic framework for network theory. This framework is applied to understand questions of graph parameter estimation, fundamentals of graph signal processing and Fourier analysis on graphs, transferability of machine learning and signal processing methods for network data, and finally to construct meaningful limiting objects of graphs and simplicial complexes. The study undertaken in this thesis both enhances the understanding of current methods, as well as inspires new methods in light of the notions of transferability defined by sampling.