Domain decomposition and model order reduction are both very important techniques for scientific and engineering computing. Their goals are both trying to speed up computations, however by different approaches. Domain decomposition is based on the general concept of divide-and-conquer which partitions a large-scale problem into a sequence of smaller and easy-to-solve problems. Model order reduction tries to relieve the simulation loads of dynamical systems by reducing the size of the systems dramatically. In this thesis, I investigate some problems arising from these two areas and propose some potential applications of them. At first, I give a sensitivity analysis of Smith method via iterative solvers. Smith method is a very important for balance truncation model order reduction. Secondly, we introduce a new effective approach to compute the reduced order model based on balanced truncation for a class of descriptor systems. Computational results were presented which indicate this new approach is promising and very efficient computationally. Thirdly, by combining balanced truncation model order reduction and domain decomposition techniques together, the reduced order models for systems of discretized partial differential equations with a spatially localized nonlinearities is derived. Finally, I present fully parallel domain decomposition techniques for another kind of problems, fast simulations of large-scale linear circuits such as power grids.