A real-time multimodal language analysis system and method based on mobile edge intelligence

Abstract

The invention discloses a real-time multimodal language analysis system and method based on mobile edge intelligence, including three types of mobile edge intelligent servers: mobile base station (MGS), unmanned vehicle (UGV) and unmanned aerial vehicle (UAV), three The computing resources of the user decrease in turn, and the flexibility of movement increases in turn; the real-time multi-modal language analysis system divides the user's language data into three modes: text, voice and image, according to the difficulty of calculation and analysis and the required It is necessary to calculate the size of the resource, and assign the calculation task to the appropriate MEI server for execution. The method of the present invention constructs a real-time multi-modal language analysis and calculation problem in a dynamic environment, and then generates a task offloading matrix and a resource allocation matrix through a deep learning online optimization method, and at the same time, automatically implements it according to the channel conditions and interference during communication The mobile path planning of the MEI server; the trained DNN can be applied to dynamic scenarios where the number of multi-modal computing tasks changes, and has strong practicability.

Description

technical field [0001] The invention belongs to the technical field of mobile edge computing, and in particular relates to a real-time multimodal language analysis system and method based on mobile edge intelligence. Background technique [0002] With the rapid development of artificial intelligence technology, speech recognition, image recognition, text recognition and other technologies have made breakthroughs in the field of computer human-computer interaction, but single-modal (single text, voice or image) language processing technology It is difficult to meet the complex scenes and environments in reality, but the analysis technology of multimodal language (combining multiple expressions such as text, voice and image) can allow the computer to hear and see clearly, and can better understand the meaning behind the language communication scene. Reducing misreading and misunderstanding in communication is a hot spot in the future analysis of language communication. [000...

AI Technical Summary

Problems solved by technology

[0002] With the rapid development of artificial intelligence technology, speech recognition, image recognition, text recognition and other technologies have made breakthroughs in the field of computer human-computer interaction, but single-modal (single text, voice or image) language processing technology It is difficult to meet the complex scenes and environments in reality, but the analysis technology of multimodal language (combining multiple expressions such as text, voice and image) can allow the computer to hear and see clearly, and can better understand the meaning behind the language communication scene. Reducing misreading and misunderstanding in communication is a hot spot in future language communication analysis

[0003] However, multimodal language analysis technology is a cutting-edge multimodal artificial intelligence technology that requires complex artificial intelligence models as support. Real-time multimodal language analysis technology requires a large number of computing resources and extremely low communication delays. Difficult to achieve in the current computing environment

At the same time, if the mobile environment is further considered, the current mobile terminals often have limited energy consumption, and the calculation of the multimodal artificial intelligence model in the real-time multimodal language analysis system requires a lot of energy consumption. Modal form, difficult to achieve real-time multimodal language analysis

Therefore, the design of a real-time multimodal language analysis system and method based on a mobile environment is a huge technical challenge.

Images