Multi-modal interaction-based multi-human-computer cooperative behavior monitoring system and control method thereof

Abstract

The invention relates to the field of human-computer interaction, provides a multi-modal interaction-based multi-human-computer cooperative behavior monitoring system and aims to solve the technical problems of how to obtain interactive behavior data of all users and apply the data to a multi-human cooperative command behavior. The multi-modal interaction-based multi-human cooperative behavior monitoring system provided by the invention comprises a cloud server and a plurality of subsystems, wherein the subsystems comprise image collecting devices for collecting current spatial images, controldevices for recognizing faces and carrying out semantic analysis, wake-up recognition and sound source positioning, power amplifying devices and a plurality of sound receiving devices for collectingsounds in space and judging sound source positions according to the collected sounds; the cloud server is used for obtaining recognition results and analysis results according to the subsystems and sending control instructions to the control devices of the subsystems; and the control devices are used for controlling the corresponding power amplifying devices to play response voices according to the control instructions. Through the system provided by the invention, the interaction behaviors of all the users can be obtained and the multi-human cooperative operation can be realized.

Description

technical field [0001] The invention relates to the technical field of human-computer interaction, in particular to a multi-modal interaction-based multi-person collaborative behavior monitoring system and a control method thereof. Background technique [0002] With the rapid development of artificial intelligence, the machine's multi-modal understanding of humans in the process of human-computer interaction plays an increasingly important role in the fields of education, command and control, and engineering manufacturing. With the development of society, the process of human-computer interaction requires the cooperation of multiple people to complete a task. Conventional applications are more about grafting multiple single technologies into products, traditionally including image application technology, speech recognition technology such as iFlytek's voice, face++'s image algorithm, etc. The starting point of artificial intelligence is the accurate processing of image reco...

AI Technical Summary

Problems solved by technology

Conventional human-computer interaction monitoring and monitoring methods, including speech recognition and action monitoring, are all run independently and without collaborative analysis, so it is inevitable that there will be a lack of correlation error analysis and accuracy analysis for event processing

Images