As the trend of home-delivered healthcare grows, the number of healthcare devices being utilized in the home setting also increases greatly, but the usability of these devices has not been examined systematically. Although usability testing is regarded as the standard practice of usability assessment, it may not be as cost-effective as alternative methods such as retrospective measures. This study collected data for healthcare devices from a usability test and a retrospective survey. Four measures including the ISO 9241-11 metrics of effectiveness, efficiency, and satisfaction (ISO, 1998), and the System Usability Scale (SUS, Brooke, 1996) scores were compared between methods. Despite the inconsistency in effectiveness and efficiency with the usability test, the retrospective survey generated assessments that differentiated between devices. Retrospective assessments may thus be useful for comparative evaluations of medical devices, but cannot substitute standard usability tests for collecting objective usability data.