Multi-modal man-machine interaction system and control method thereof

Abstract

The invention discloses a multi-modal man-machine interaction system and a control method thereof. The main interaction process comprises: interaction information collection, interaction motion identification, interaction instruction mapping, logic processing unit, and interaction motion response. The interactive identification algorithm of voice, touch gesture and body feeling and gesture is broken through. The method comprises contact dynamic tracking and movement track coupling algorithm, physical parameter model coupling method, multi-user interaction queue mechanism. Touch control / body feeling gesture interaction mapping instruction and logic processing, cooperation / mutual exclusion access mechanism are set; the user control instruction is converted into service instruction of application system scheduling execution; multi-user interaction is supported; the system has strong anti-interference capability and identification efficiency.

Description

technical field [0001] The invention belongs to the technical field of artificial intelligence, and in particular relates to a multimodal human-computer interaction system and a control method thereof. Background technique [0002] With the development of emerging technologies such as sensor technology, intelligent computing technology, cloud computing, and big data, human-computer interaction continues to develop in the direction of people-oriented, enhanced perception, and user-friendly. Traditional mouse, keyboard and other interactive methods can no longer meet the needs of human-computer interaction. A new, efficient and convenient human-computer interaction environment needs to be established urgently to achieve natural and efficient interaction between humans and systems and accurate understanding of user intentions. [0003] At present, the relatively mature interaction methods are as follows: (1) voice interaction. The user sends instructions to the target program ...

AI Technical Summary

Problems solved by technology

See Document 8: S.Carrino, A.Péclat, E.Mugellini. Humans and smart environments: a novel multimodal interaction approach [A]. Proceedings of International Conference on Multimodal Interaction [C]. Alicante, Spain: Association for Computing Machinery, 2011 , 105-112. At present, the multimodal human-computer interaction integration method cannot take into account all the interaction methods, and the degree of freedom and naturalness of the user is restricted to a certain extent, and it cannot be independently selected and adjusted according to the user's habits or preferences.

Images