The future of computing is in allowing our devices to see what we see: Continuous mobile vision. Wearable systems will continuously interpret vision data for real-time analytics for rich context-awareness. Unfortunately, today’s system software and imaging hardware are ill-suited for this goal of “continuous mobile vision.” Current systems -- highly optimized for photography -- fail to achieve sufficient energy efficiency for the minuscule energy capacity requirements of wearable batteries. This thesis provides a rethinking of the vision system stack that includes application frameworks, operating system and sensor hardware to improve efficiency by two orders of magnitude.

This cross-layer rethinking contributes: (1) a split-process application framework that eliminates redundancy in data movement and processing across multiple concurrent applications, (2) operating system optimizations for energy proportional image capture, and (3) a mixed-signal image sensor architecture that processes data in the analog domain to eliminate the efficiency bottleneck of analog-digital conversion. By exploiting the hardware/software boundary for improved energy efficiency, this thesis opens the door to integrate our devices with our real-world environments and ultimately, our own lives.