Voice visualization method based on integration characteristic and neural network

Abstract

The invention relates to a voice visualization method based on an integration characteristic and a neural network. The method is characterized by comprising the following eight steps of: preprocessing a voice signal, extracting characteristics, optimizing the characteristics, designing the neural network, mapping position information, encoding a main color, encoding pattern information and synthesizing an image. Different voice characteristics are integrated in an image to create a voice signal readable mode for a deaf mute, and images at different positions have different colors, so that the advantage that the deaf mute has higher color stimulated visual memory capacity is better utilized; moreover, tone characteristics are adopted to encode the pattern information in order to reduce a screen accommodating load and an observer memory burden; therefore, voices consisting of the same final and different tones are displayed at the same position. Compared with the conventional method, the voice visualization method has high robustness and classification positioning capacity, and has a good effect of assisting the learning of the deaf mute.

Description

technical field [0001] The invention relates to a visualization method of Mandarin Chinese, in particular to a speech visualization method based on integrated features and neural networks. Background technique [0002] Speech is the acoustic performance of language, the most natural, effective and convenient means for human to exchange information, and also a kind of support for human thinking. For deaf-mute people, language communication has become a difficult thing to achieve. Some deaf-mute people cannot speak because their auditory organs have been damaged and they cannot collect voice information to the brain. Studies have shown that the human auditory system and visual system are two different and complementary information systems. The visual system is a highly parallel information receiving and processing system. Millions of cone cells on the retina of the human eyeball pass through The fibrous nerve tissue is connected with the brain to form a highly parallel channe...

AI Technical Summary

Problems solved by technology

[0003] In 1947, R.K.Potter and G.A.Kopp et al. proposed a visualization method—the spectrogram, and then different speech research experts began to study and improve this speech visualization method. For example, in 1976, L.C.Stewart et al. proposed a chromatogram And in 1984, G.M.Kuhn et al. proposed a real-time spectrogram system for training deaf people, and P.E.Stern in 1986, F.Plante in 1998 and R.Steinberg in 2008 also proposed many spectrogram improvements. method, but the displayed spectrogram is very professional, and it is difficult to distinguish memory

Especially for the same person with different voices, or even the same voice for the same person, it may cause changes in the spectrogram, and its robustness is even worse for voice signals recorded in different environments

[0004] In addition, some scholars have realized speech visualization through the movement changes of human vocal organs and facial expressions, and effectively analyzed the human pronunciation process. However, in terms of speech intelligibility, it is still difficult to achieve the desired effect. Except for very few experts, it is difficult for people to directly perceive speech sounds directly by observing the movement of vocal organs and changes in facial expressions

Images