The recent experimental realization of strongly imbalanced mixtures of ultracold atoms opens new possibilities for studying impurity dynamics in a controlled setting. In this paper, we discuss how the techniques of atomic physics can be used to explore new regimes and manifestations of Anderson’s orthogonality catastrophe (OC), which could not be accessed in solid-state systems. Specifically, we consider a system of impurity atoms, localized by a strong optical-lattice potential, immersed in a sea of itinerant Fermi atoms. We point out that the Ramsey-interference-type experiments with the impurity atoms allow one to study the OC in the time domain, while radio-frequency (RF) spectroscopy probes the OC in the frequency domain. The OC in such systems is universal, not only in the long-time limit, but also for all times and is determined fully by the impurity-scattering length and the Fermi wave vector of the itinerant fermions. We calculate the universal Ramsey response and RF-absorption spectra. In addition to the standard power-law contributions, which correspond to the excitation of multiple particle-hole pairs near the Fermi surface, we identify a novel, important contribution to the OC that comes from exciting one extra particle from the bottom of the itinerant band. This contribution gives rise to a nonanalytic feature in the RF-absorption spectra, which shows a nontrivial dependence on the scattering length, and evolves into a true power-law singularity with the universal exponent 1 / 4 at the unitarity. We extend our discussion to spin-echo-type experiments, and show that they probe more complicated nonequilibirum dynamics of the Fermi gas in processes in which an impurity switches between states with different interaction strength several times; such processes play an important role in the Kondo problem, but remained out of reach in the solid-state systems. We show that, alternatively, the OC can be seen in the energy-counting statistics of the Fermi gas following a sudden quench of the impurity state. The energy distribution function, which can be measured in time-of-flight experiments, exhibits characteristic power-law singularities at low energies. Finally, systems in which the itinerant fermions have two or more hyperfine states provide an even richer playground for studying nonequilibrium impurity physics, allowing one to explore the nonequilibrium OC and even to simulate quantum transport through nanostructures. This provides a previously missing connection between cold atomic systems and mesoscopic quantum transport.