Metal nanoparticles with a dumbbell-like geometry have plasmonic properties similar to those of their nanorod counterparts, but the unique steric constraints induced by their enlarged tips result in distinct geometries when self-assembled. Here, we investigate gold dumbbells that are assembled into dimers within polymeric micelles. A single-particle approach with correlated scanning electron microscopy and dark-field scattering spectroscopy reveals the effects of dimer geometry variation on the scattering properties. The dimers are prepared using exclusively achiral reagents, and the resulting dimer solution produces no detectable ensemble circular dichroism response. However, single-particle circular differential scattering measurements uncover that this dimer sample is a racemic mixture of individual nanostructures with significant positive and negative chiroptical signals. These measurements are complemented with detailed simulations that confirm the influence of various symmetry elements on the overall peak resonance energy, spectral line shape, and circular differential scattering response. This work expands the current understanding of the influence self-assembled geometries have on plasmonic properties, particularly with regard to chiral and/or racemic samples which may have significant optical activity that may be overlooked when using exclusively ensemble characterization techniques.