Cognitive dynamic video summarization using cognitive analysis enriched feature set

Abstract

Accurate and concise summarization of a media production is achieved using cognitive analysis which groups segments of the production into clusters based on extracted features, selects a representative segment for each cluster, and combines the representative segments to form a summary. The production is separated into a video stream, a speech stream and an audio stream, from which the cognitive analysis extracts visual features, textual features, and aural features. The clustering groups segments together whose visual and textual features most closely match. Selection of the representative segments derives a score for each segment based on factors including a distance to a centroid of the cluster, an emotion level, an audio uniqueness, and a video uniqueness. Each of these factors can be weighted, and the weights can be adjusted in accordance with user input. The factors can have initial weights which are based on statistical attributes of historical media productions.

Description

BACKGROUND OF THE INVENTIONField of the Invention[0001]The present invention generally relates to multimedia productions, i.e., “videos”, and more particularly to an automated method of summarizing a long video.Description of the Related Art[0002]Advancements in a wide variety of technologies have led to an explosion in the production and distribution of multimedia productions. These advancements include improvements in recording technologies such as cellphone cameras which enable nearly anyone to become a producer / director, as well as enhanced distribution channels such as the Internet. One common example is the YouTube phenomenon by which ordinary users have the opportunity to view literally millions of video programs.[0003]When someone wants to view a multimedia production on a particular topic they may find there are hundreds or even thousands of available videos on the topic. Referring again to the YouTube example, that service offers a search function by which users can enter ...

AI Technical Summary

Benefits of technology

The present invention relates to a method for generating a summary of a media production by organizing the production into segments and conducting a cognitive analysis of each segment to extract features associated with it. The features are then grouped into clusters and a representative segment is selected for each cluster to form the media summary. The method can be used for video, audio, and speech flows, and can include extracting visual, textual, and audio features from the media production. The analysis is based on factors such as distance, emotion level, audio and video uniqueness, and user input. The invention enables efficient and automated generation of media summaries without requiring human input.

Problems solved by technology

The user may, however, be overwhelmed by the sheer number of videos presented.

It is difficult for the user to even begin sifting through the results, and time constraints of course make it impossible for the user to actually view all of the videos that may be potentially relevant.

Images