Compressive sensing encodes a signal into a relatively small number of incoherent linear measurements. In theory, the optimal incoherence is achieved by completely random measurement matrices. However, such matrices are difficult and/or costly to implement in hardware realizations. After summarizing recent results of how random Toeplitz and circulant matrices can be easily (or even naturally) realized in various applications, we introduce fast algorithms for reconstructing signals from incomplete Toeplitz and circulant measurements. We present computational results showing that Toeplitz and circulant matrices are not only as effective as random matrices for signal encoding, but also permit much faster signal decoding.