Lightweight web AR recognition method and system based on binary neural network

Abstract

An embodiment of the present invention provides a lightweight Web AR recognition method and system based on a binary neural network. The method includes: loading a target image by a mobile Web browser and preprocessing the image, and simultaneously sending a target recognition task request to an edge server ;Receive the binary neural network model and related executable scripts returned by the edge server, and execute the binary neural network feed-forward calculation, obtain the image recognition result and temporarily store the output result of the shared layer, and judge whether the cross entropy of the image recognition result meets the preset If the threshold is not met, the output of the shared layer is sent to the edge server for feed-forward reasoning. The embodiment of the present invention introduces a binary neural network to accelerate network reasoning, reduce the pressure of image recognition loading delay and device energy consumption, make full use of the computing resources of mobile terminals, effectively relieve the computing pressure of edge servers, and provide Web AR applications with a real-time solution.

Description

technical field [0001] The present invention relates to the field of augmented reality technology, and more specifically, to a lightweight Web AR recognition method and system based on a binary neural network. Background technique [0002] Augmented reality (Augmented Reality, referred to as AR) is a technology that seamlessly integrates real world information and virtual world information. It uses computer information to simulate entity information within a certain time and space range, and integrates this virtual information into In the real world, enhance the user's perception of the real world, so as to achieve an effect beyond reality. Augmented reality includes the fusion of various technologies, such as image recognition, 3D modeling, sensor fusion, real-time tracking and registration, and scene fusion. [0003] At present, augmented reality systems mainly use professional equipment (such as head-mounted and glasses equipment) as the core computing equipment to meet ...

AI Technical Summary

Problems solved by technology

However, usually the DNN model is relatively large, with a large amount of calculation, and it is mainly used on the server side, and it is difficult to directly deploy it on the web browser side.

Although some JavaScript inference libraries (such as Tensoflow.js, Keras.js, etc.) implement DNN networks on mobile web browsers, the latency of model loading and network feed-forward inference is still too high, and the energy consumption is high to be effective. Applied to mobile Web AR applications, so exploring a lightweight identification system is one of the important issues to promote WebAR applications

Images