Non-volatile phase-change memory devices utilize local heating to toggle between crystalline and amorphous states with distinct electrical properties. Expanding on this kind of switching to two topologically distinct phases requires controlled non-volatile switching between two crystalline phases with distinct symmetries. Here, we report the observation of reversible and non-volatile switching between two stable and closely related crystal structures, with remarkably distinct electronic structures, in the near-room-temperature van der Waals ferromagnet Fe5−δGeTe2. We show that the switching is enabled by the ordering and disordering of Fe site vacancies that results in distinct crystalline symmetries of the two phases, which can be controlled by a thermal annealing and quenching method. The two phases are distinguished by the presence of topological nodal lines due to the preserved global inversion symmetry in the site-disordered phase, flat bands resulting from quantum destructive interference on a bipartite lattice, and broken inversion symmetry in the site-ordered phase.