In-Sensor Visual Computing: A Revolutionary Approach to Machine Vision Systems
A joint research team in China has published a review on the concept of in-sensor visual computing, a cutting-edge hardware solution that offers increased efficiency, cost-effectiveness, and security compared to conventional machine vision systems. The team’s review was recently published in Intelligent Computing, a Science Partner Journal.
In-sensor visual computing systems are inspired by the biological mechanisms used by humans and mammals to collect, extract, and process visual signals. These systems integrate sensing, storage, and computation onto the focal plane of image sensors. By processing data within each sensor and extracting critical information from raw signals, in-sensor visual computing systems overcome the major obstacles faced by conventional systems, namely high latency, high power consumption, and privacy risks.
The development of in-sensor computing devices has focused on novel circuit designs and new materials. The team’s review specifically centers on a vision chip known as the SCAMP pixel processor array, or SCAMP chip for short. This chip, which has been in development for two decades, is a mature and interdisciplinary research platform widely used in computing experiments.
The team introduces the most up-to-date system based on the SCAMP chip, known as SCAMP-5d. It is a general-purpose, programmable system extensively used in robotics and computer vision. The review also explores software tools and platforms that have been developed for the SCAMP chip, including frameworks for programming the chip and platforms for simulating its operations.
The review provides an overview of in-sensor visual computing algorithms and applications based on the versatile SCAMP chip. These algorithms range from lower-level techniques such as image enhancement and feature extraction to higher-level tasks such as classification, localization, and segmentation using neural networks. The applications enabled by these algorithms mainly revolve around state estimation and robot navigation.
Despite the advancements brought about by in-sensor visual computing systems using the SCAMP pixel processor array, there are still limitations such as low resolution, limited computing resources, noise, and suboptimal algorithm design and deployment. However, engineers and researchers are actively working to overcome these limitations by exploring non-conventional computing methods, such as sensor fusion and edge computing.
The authors of the review are actively involved in the co-development and co-optimization of circuit design, integration technologies, and associated algorithms, both for academic and commercial purposes. They believe that the next-generation SCAMP vision systems will offer even better performance at lower power consumption.
In summary, in-sensor visual computing is revolutionizing the field of machine vision systems by integrating sensing, storage, and computation onto image sensors. The SCAMP chip, with its novel circuit design, is a key component in enabling these advancements. As engineers and researchers continue to overcome current limitations and explore new computing methods, the future of in-sensor visual computing systems looks promising in terms of performance and energy efficiency.