Bimodal image signal processor and bimodal image signal processing system

Abstract

The embodiment of the invention discloses a bimodal image signal processor and a bimodal image signal processing system, and the bimodal image signal processor comprises a synchronous image signal processor which is used for processing a synchronous signal in a bimodal image signal; an asynchronous image signal processor used for processing asynchronous signals in the bimodal image signals; and a neural network unit used for processing and outputting the synchronous signal output by the synchronous image signal processor and the asynchronous signal output by the asynchronous image signal processor according to a neural network algorithm. According to the scheme, the dual-mode image signal processor can process synchronous signals and asynchronous signals in the dual-mode image signals at the same time, very high signal fidelity can be achieved when images are shot at a high speed, and meanwhile high dynamic range and high time resolution can be achieved. Moreover, artificial intelligence related tasks such as a region of interest can be marked in real time by utilizing the neural network unit on the basis of ensuring low power consumption.

Description

technical field [0001] Embodiments of the present invention relate to the technical field of image sensing, and in particular to a dual-mode image signal processor and a dual-mode image signal processing system. Background technique [0002] A traditional image signal processor (Image Signal Processor, ISP) generally includes an active pixel sensor (Active Pixel Sensor, APS) and a dynamic vision sensor (Dynamic Vision Sensor, DVS). Among them, the active pixel sensor can only realize the processing of color images arranged in the form of pixel matrix image frames, which has the advantages of high color reproduction and high image quality, but the dynamic range of the image signal acquired by it is small, and the shooting speed is slow . Dynamic vision sensors can perceive dynamic scenes, have a faster shooting speed, and obtain image signals with a larger dynamic range. However, the image quality collected by this type of sensor is poor and the resolution is low. Tradition...

AI Technical Summary

Problems solved by technology

Among them, the active pixel sensor can only realize the processing of color images arranged in the form of pixel matrix image frames, which has the advantages of high color reproduction and high image quality, but the dynamic range of the image signals it acquires is small, and the shooting speed is slow

Dynamic vision sensors can perceive dynamic scenes, have a faster shooting speed, and obtain a larger dynamic range of image signals. However, the image quality collected by this type of sensor is poor and the resolution is low.

Traditional image signal processors cannot process active pixel sensor signals and dynamic visual receptor signals at the same time, and traditional image signal processors do not integrate on-chip neural networks, which cannot achieve artificial intelligence-related image processing tasks

[0003] The intelligent image signal processor in the prior art can only support the processing of color images taken by traditional active pixel sensors, and cannot support the simultaneous processing of image information with sparse events taken by similar dynamic vision sensors, and thus cannot make full use of event-based image signals Due to the sparsity, high speed and high dynamic range characteristics of the medium, the image processing effect and image quality cannot be guaranteed

In addition, the current intelligent image signal processor can only support the artificial neural network (Artificial Neural Network, ANN) represented by the convolutional neural network, and realize the machine learning task of the traditional color image, but the data volume of the color image information is large, and it has It is difficult for the intelligent image signal processor of the artificial neural network to increase the frame rate of color image information while ensuring the image processing effect and low power consumption, and thus cannot process image information at high speed and low power consumption

Images