Within this thesis I present several neutron scattering and transport works on BaFe2-xNixAs2, FeS and Y3Fe2(FeO4)3. Inelastic neutron scattering technique is an ideal and direct probe to magnetic moments and excitations in novel condensed matter materials. We carried out neutron scattering and transport experiments to investigate the nematic phase in BaFe2-xNixAs2, the relationship of magnetic excitation and superconductivity in FeS, and the magnon-phonon interaction in yttrium iron garnet.

The characteristic of neutrons, several types of neutron scattering spectrometers and related theories are introcuced in Chapter 1. We also include a brief introduction of iron based superconductors in Chapter 2.

In Chapter 3(published) and 4(unpublished), we reported our transport and neutron scattering to study electronic, structural, and magnetic properties of the parent and electron-doped BaFe2NixAs2 iron pnictides in the external stress free detwinned state. Using a specially designed in-situ mechanical detwinning device, we demonstrate that the in-plane resistivity anisotropy observed in the uniaxial strained tetragonal state of BaFe2-xNixAs2 below a temperature T*, previously identified as a signature of the electronic nematic phase, is also present in the stress free tetragonal phase below T** (<T*).
By carrying out neutron scattering measurements on BaFe2As2 and BaFe1.97Ni0.03As2, we observed that the anisotropy of spin excitation is absent above Ts, confirming that the previously measured anisotropy in spin excitation above Ts is induced by external pressure. We argue that the resistivity anisotropy in the stress free tetragonal state of iron pnictides arises from the magnetoelastic coupling associated with antiferromagnetic order.

In Chapter 5, we reported neutron scattering and quantum oscillation experiment on FeS. As high-temperature superconductivity occurs near antiferromagnetic instabilities and nematic state, debate remains on the origin of nematic order in FeSe and its relation with superconductivity. With transport, neutron scattering and Fermi surface measurements, we demonstrate that hydro-thermo grown superconducting FeS, an isostructure of FeSe, is a tetragonal paramagnet without nematic order and with a quasiparticle mass significantly reduced from that of FeSe. Only stripe-type spin excitation is observed up to 100 meV. No direct coupling between spin excitation and superconductivity in FeS is found, suggesting that FeS is less correlated and the nematic order in FeSe is due to competing checkerboard and stripe spin fluctuations.

In Chapter 6, we report the neutron scattering and spin Seebeck effect in the magnetic insulator yttrium iron garnet (YIG). YIG has a ferrimagnetic transition temperature of $\sim$560 K has been widely used in microwave and spintronic devices. Anomalous features in the spin Seebeck effect (SSE) voltages have been observed in Pt/YIG and attributed to the magnon-phonon coupling. We mapped out low-energy spin waves and acoustic phonons of YIG at 100 K as a function of increasing magnetic field. By comparing the zero and 9.1 T data, we find that instead of splitting and opening up gaps at the spin wave and acoustic phonon dispersion intersecting points, magnon-phonon coupling in YIG enhances the hybridized scattering intensity. These results are different from expectations of conventional spin-lattice coupling, calling for new paradigms to understand the scattering process of magnon-phonon interactions and the resulting magnon-polarons.