The discovery of two-dimensional magnetism in van der Waals honeycomb magnetic insulators CrI3, CrGeTe3 has opened up a new degree of freedom in the two-dimensional realm of physics and material science. The robustness of the magnetic order of these systems in the monolayer limit shows promising future applications for spintronic devices. On the other hand, honeycomb magnetic insulators are by themselves an interesting topic to study, as it is a magnetic analog of graphene. This makes the magnon dispersion in honeycomb ferromagnets similar to the electron dispersion in graphene, with Dirac cones at the Brillouin zone corners. With proper additional terms in the Hamiltonian such as the Dzyaloshinskii-Moriya (DM) interaction and Kitaev interactions, nontrivial topological magnons can be created in these materials. Furthermore, the interplay between spin, orbital, and lattice degrees of freedom can give rise to richer excitation topology and textures.

This thesis is dedicated to presenting the neutron scattering results on several van der Waals magnetic insulators including CrI3, CrCl3, CrGeTe3, and FePSe3, with discussions on results on other similar compounds. After a brief introduction to linear spin wave theory, neutron scattering, and topological magnons in Chapter \ref{ch:Intro}, we show in Chapter 2 that a Dirac gap is observed in the magnon dispersion in CrI3, which partially closes upon applying an in-plane field. The strength of DM and Kitaev interactions is quantitatively estimated after considering the extrinsic origins of the magnon gap. If the topological magnon exists in CrI3, its edge modes will adapt non-dissipative magnon transport and can be utilized in spintronic applications. We also discussed other compounds that can potentially host topological magnons, such as the Dirac gaps in Cr(Si/Ge)Te3, and the anomalous thermal Hall effect in VI3, compared with Dirac magnons in CrBr3 and CrCl3.

In Chapter 3, magnon-phonon coupling effects on the magnon spectrum are extensively discussed. We start with a simple theory showing that nontrivial topological properties of magnons and phonons can arise from linear magnon-phonon coupling, then we present Raman scattering results on FePS3 and FePSe3, showing that linear magnon-phonon coupling can be realized in these honeycomb van der Waals systems, and can potentially host topological magnon polarons that can be easily tuned by a magnetic field. Finally, we introduce nonlinear magnon-phonon coupling originating from the Heisenberg interactions changing with atomic displacement and show that these interactions can cause magnon decay and renormalizations in the honeycomb ferromagnet CrGeTe3, making it unsuitable for topological magnon transfer. We conclude the thesis in Chapter 4 with a summary and state the challenge and prospect of this work.