Image mapping spectrometry (IMS) is a hyperspectral imaging technique that simultaneously captures spatial and spectral information about an object in real-time. We present a new calibration procedure for the IMS as well as the first detailed evaluation of system performance. We correlate optical components and device calibration to performance metrics such as light throughput, scattered light, distortion, spectral image coregistration, and spatial/spectral resolution. Spectral sensitivity and motion artifacts are also evaluated with a dynamic biological experiment. The presented methodology of evaluation is useful in assessment of a variety of hyperspectral and multi-spectral modalities. Results are important to any potential users/ developers of an IMS instrument and to anyone who may wish to compare the IMS to other imaging spectrometers.