We present observations of wave steepening and signatures of shock formation during expansion of ultracold neutral plasmas formed with an initial density distribution that is centrally peaked and decays exponentially with distance. The plasma acceleration and velocity decrease at large distance from the plasma center, leading to central ions overtaking ions in the outer regions and the development of a steepening front that is narrow compared to the size of the plasma. The density and velocity change dramatically across the front, and significant heating of the ions is observed in the region of steepest gradients. For a reasonable estimate of electron temperature, the relative velocity of ions on either side of the front modestly exceeds the local sound speed (Mach number M≳1). This indicates that by sculpting steep density gradients, it is possible to create the conditions for shock formation, or very close to it, opening a new avenue of research for ultracold neutral plasmas.