The 4-dimensional equivalence relation of concordance (smooth or topological) gives a group structure on the set of knots, under the connected-sum operation. The n-solvable filtration of the knot concordance group (denoted C), due to Cochran-Orr-Teichner, has been instrumental in the study of knot concordance in recent years. Part of its significance is due to the fact that certain geometric attributes of a knot imply membership in various levels of the filtration. We show the counterpart of this fact for two new filtrations of C due to Cochran-Harvey-Horn, the positive and negative filtrations. The positive and negative filtrations have definitions similar to that of the n-solvable filtration, but have the ability (unlike the n-solvable filtrations) to distinguish between smooth and topological concordance. Our geometric counterparts for the positive and negative filtrations of C are defined in terms of Casson towers, 4-dimensional objects which approximate disks in a precise manner. We establish several relationships between these new Casson tower filtrations and the various previously known filtrations of C, such as the n-solvable, positive, negative, and grope filtrations. These relationships allow us to draw connections between some well-known open questions in the field.