Real-time multi-modal language analysis system and method based on mobile edge intelligence

Abstract

The invention discloses a real-time multi-modal language analysis system and method based on mobile edge intelligence, and the system comprises three types of mobile edge intelligent servers: a mobilebase station (MGS), an unmanned vehicle (UGV) and an unmanned aerial vehicle (UAV), and the computing resources of the three types of mobile edge intelligent servers are sequentially reduced, and themoving flexibility is sequentially improved. According to the real-time multi-modal language analysis system, language data of a user is divided into three modalities, namely characters, voice and images, and a calculation task is allocated to a proper MEI server to be executed according to the calculation and analysis difficulty and the size of required calculation resources. According to the method, a real-time multi-modal language analysis and calculation problem in a dynamic environment is constructed, then a task unloading matrix and a resource allocation matrix are generated through a deep learning online optimization method, and meanwhile, movement path planning of an MEI server is automatically implemented according to channel conditions and interference during communication. Thetrained DNN can be suitable for a dynamic scene in which the number of multi-modal computing tasks changes, and has very high practicability.

Description

technical field [0001] The invention belongs to the technical field of mobile edge computing, and in particular relates to a real-time multimodal language analysis system and method based on mobile edge intelligence. Background technique [0002] With the rapid development of artificial intelligence technology, speech recognition, image recognition, text recognition and other technologies have made breakthroughs in the field of computer human-computer interaction, but single-modal (single text, voice or image) language processing technology It is difficult to meet the complex scenes and environments in reality, but the analysis technology of multimodal language (combining multiple expressions such as text, voice and image) can allow the computer to hear and see clearly, and can better understand the meaning behind the language communication scene. Reducing misreading and misunderstanding in communication is a hot spot in the future analysis of language communication. [000...

AI Technical Summary

Problems solved by technology

[0002] With the rapid development of artificial intelligence technology, speech recognition, image recognition, text recognition and other technologies have made breakthroughs in the field of computer human-computer interaction, but single-modal (single text, voice or image) language processing technology It is difficult to meet the complex scenes and environments in reality, but the analysis technology of multimodal language (combining multiple expressions such as text, voice and image) can allow the computer to hear and see clearly, and can better understand the meaning behind the language communication scene. Reducing misreading and misunderstanding in communication is a hot spot in future language communication analysis

[0003] However, multimodal language analysis technology is a cutting-edge multimodal artificial intelligence technology that requires complex artificial intelligence models as support. Real-time multimodal language analysis technology requires a large number of computing resources and extremely low communication delays. Difficult to achieve in the current computing environment

At the same time, if the mobile environment is further considered, the current mobile terminals often have limited energy consumption, and the calculation of the multimodal artificial intelligence model in the real-time multimodal language analysis system requires a lot of energy consumption. Modal form, difficult to achieve real-time multimodal language analysis

Therefore, the design of a real-time multimodal language analysis system and method based on a mobile environment is a huge technical challenge.

Images