In this dissertation, novel solution algorithms are developed for large structured linear systems such as the discretized Helmholtz equation. Firstly, we develop two parallel randomized algorithms for constructing hierarchically semiseparable (HSS) matrices. Randomized sampling reduces the computational complexity and the communication cost simultaneously, and the resulting method is suitable for solving dense systems arising from multiple scattering problems. Secondly, a direct factorization update algorithm is proposed for solving the Helmholtz equation with changing wavespeed. The data dependency among a set of interior and exterior sub-problems is exploited to maximize the data reuse and to minimize the propagation of changes. Thirdly, interconnected HSS structures are designed to improve the efficiency of rank-structured factorization of PDE problems. Intermediate Schur complements at different levels share the same set of HSS bases during the factorization. Finally, we propose a contour integration preconditioner for solving 3D high-frequency Helmholtz equation. By solving systems with complex shifts, the problem is projected in subspaces with faster GMRES convergence. The shifted problems are solved by a polynomial fixed-point iteration, which is robust when the shift changes.