Device and method for text to audio mapping, and animation of the text

Abstract

The present invention introduces a device, method and computer reading media for mapping establishing text to audio time sequence. The present invention also introduces a device, method and computer reading media for playing audio text animation. A mapping agent (10) takes the text (12) and the corresponding audio record (11) as input, and makes begin and end time assign to the text unit (15). The player (50) takes text (15), audio (17) and mapping (16) as input, and makes text animation and displays text (15) in-phase along with playing the audio (17). The present invention is used to endow vitality during the process of playing audio record; to instead the traditional playback control to control audio play; to play and display the note of voice recording; to implement the characteristic of flowing audio without using basic stream protocol.

Description

technical field [0001] The invention relates to the field of audio analysis, in particular audio such as speeches containing textual descriptions. More specifically, it relates to a device for creating text-to-audio mapping and corresponding processing, and a device for text animation synchronized with playing audio information. The text animation that is synchronized with the audio represents the mapping of text to audio, and the amount of information conveyed is far more than that expressed by audio or text itself, and it is also more than the information expressed by the combination of audio and static text quantity. [0002] Based on the first embodiment of the present invention, we provide a device ("Sound Text Mapper 10") and a method for creating text-to-audio mappings. [0003] Based on the second embodiment of the present invention, we provide a text animation device ("sound and text player 50") with audio playback. [0004] The mapper 10 and the player 50 of the p...

AI Technical Summary

Problems solved by technology

When playing a recorded speech while simultaneously presenting a written transcript of that speech (or presenting a text while simultaneously playing a transcribed version of it), the listening reader (or the audience who is reading) encounters several problems: The first Yes, how do you grasp where the words in the text have progressed in relation to what is being told? Previous techniques deal with this problem in two ways, and in the following we analyze their shortcomings

The second problem is that in speech-plus-text representations, the individual written words that make up the text can be made machine-searchable, annotatable, and interactive, while the individual spoken words of the audio part cannot.

Previous technologies, while aware of the correspondence between text and audio, have failed to make audio containing speech machine-searchable, annotatable, and interactive

A third issue is that interactive transmission of audio components requires the development of a streaming protocol

[0023] 1. The creation of this kind of image takes a long time and requires relevant personnel to have high skills

[0024] 2. Even if only text is displayed and audio is played, the creation of this kind of video will form a large-capacity data file

[0025] These large files consume a correspondingly large amount of bandwidth and data storage space, and thus impose limitations on devices that can download speech-plus-text representations to programmable or dedicated digital computing devices

[0026] 3. Animation is fixed

[0027] 4. Usually animations are below word level granularity

[0029] 6. Interaction with audio is limited to controlling the player

[0030] 7. Audio is not machine searchable or annotatable

[0033] 10. Cannot interact with the text itself

Images