Time sequence mapping method for text to audio realized by computer

Abstract

The present invention introduces a device, method and computer reading media for mapping establishing text to audio time sequence. The present invention also introduces a device, method and computer reading media for playing audio text animation. A mapping agent (10) takes the text (12) and the corresponding audio record (11) as input, and makes begin and end time assign to the text unit (15). Theplayer (50) takes text (15), audio (17) and mapping (16) as input, and makes text animation and displays text (15) in-phase along with playing the audio (17). The present invention is used to endow vitality during the process of playing audio record; to instead the traditional playback control to control audio play; to play and display the note of voice recording; to implement the characteristicof flowing audio without using basic stream protocol.

Description

technical field [0001] The invention relates to the field of audio analysis, in particular audio such as speeches containing textual descriptions. More specifically, the corresponding process for creating text-to-audio mappings. Background technique [0002] The first technological advance in language-based was the development of simple vocalizations, which at the time could only convey meaning in temporal isolation. Later, people combined these initial vocalizations in time phase and sequential order to form streams of speech. Later, people invented drawing simple symbols or images on cave walls or other suitable surfaces, but they only communicated in isolation in space. Later generations linked these symbols or images with spoken language in time. Later, people combined these independent language-related graphics in a sequential order in space to form written language or "text". Specifically, our innovative ancestors began to order the sequential space of morphologica...

AI Technical Summary

Problems solved by technology

When playing a recorded speech while simultaneously presenting a written transcript of that speech (or presenting a text while simultaneously playing a transcribed version of it), the listening reader (or the audience who is reading) encounters several problems: The first Yes, how do you grasp where the words in the text have progressed in relation to what is being told? Previous techniques deal with this problem in two ways, and in the following we analyze their shortcomings

The second problem is that in speech-plus-text representations, the individual written words that make up the text can be made machine-searchable, annotatable, and interactive, while the individual spoken words of the audio part cannot.

Previous technologies, while aware of the correspondence between text and audio, have failed to make audio containing speech machine-searchable, annotatable, and interactive

A third issue is that interactive transmission of audio components requires the development of a streaming protocol

[0020] 1. The creation of this kind of image takes a long time and requires relevant personnel to have high skills

[0021] 2. Even if only text is displayed and audio is played, the creation of this kind of video will form a large-capacity data file

These large files consume correspondingly large amounts of bandwidth and data storage space, and thus impose many limitations on devices that can download speech-plus-text representations to programmable or dedicated digital computing devices

[0022] 3. Animation is fixed

[0023] 4. Usually animations are below word-level granularity

[0025] 6. Interaction with audio is limited to controlling the player

[0026] 7. Audio is not machine searchable or annotatable

[0029] 10. Cannot interact with the text itself

Images