Magnetically-guided colloidal assembly has proven to be a versatile method for building hierarchical particle assemblies. This review describes the dipolar interactions that govern superparamagnetic colloids in time-varying magnetic fields, and how such interactions have guided colloidal assembly into materials with increasing complexity that display novel dynamics. The assembly process is driven by magnetic dipole-dipole interactions, whose strength can be tuned to be attractive or repulsive. Generally, these interactions are directional in static external magnetic fields. More recently, time-varying magnetic fields have been utilized to generate dipolar interactions that vary in both time and space, allowing particle interactions to be tuned from anisotropic to isotropic. These interactions guide the dynamics of hierarchical assemblies of 1-D chains, 2-D networks, and 2-D clusters in both static and time-varying fields. Specifically, unlinked and chemically-linked colloidal chains exhibit complex dynamics, such as fragmentation, buckling, coiling, and wagging phenomena. 2-D networks exhibit controlled porosity and interesting coarsening dynamics. Finally, 2-D clusters have shown to be an ideal model system for exploring phenomena related to statistical thermodynamics. This review provides recent advances in this fast-growing field with a focus on its scientific potential.