The phenotype of adults can be strongly influenced by the environmental conditions experienced during development. Consequently, variation in habitat quality across space and through time also leads to differences in the phenotypes of adults. This could create carry-over effects where differences in the natal habitat quality of colonizers influence population dynamics in new habitats. We tested this hypothesis experimentally by simulating dispersal of Tribolium castaneum from low- or high-quality natal habitat into new patches of low- or high-quality habitat. Differences in the natal habitat quality of colonizers altered population growth trajectories and led to carrying capacities that differed by up to 63% within a habitat type, indicating that patch dynamics are determined by the interaction of past and current habitat quality. Interestingly, even after multiple generations, the natal habitat of colonizers determined differences in adult traits that were related to density-dependent population regulation. These changes in adult phenotype could at least partially explain why carry-over effects continued to alter population dynamics for multiple generations until the end of the experiment. These results highlight the importance of variable habitat quality and carryover effects for population dynamics.