In this thesis, I consider the problem of making linear algebra simple to use and efficient to run in a relational database management system. Relational database systems are widely used, and much of the data in the world is stored within them. Having linear algebra integrated into a relational database would provide great support for tasks such as in-database analytics and in-database machine learning. Currently, when it is necessary to perform such analyses, one must either extract the data from a database, and use an external tool such as MATLAB, or else use awkward, existing within-the-database linear algebra facilities. In this thesis, I will focus on my four main contributions: (1) I add vector and matrix types to SQL, the most commonly-used database programming language; (2) I design a few simple SQL language extensions to accommodate vectors and matrices; (3) I consider the problem of making vector and matrix operations efficient via integration with the database query optimizer; and (4) I conduct some experiments to show the efficacy of my language extensions.