Multi-Modal Digital Video Library
(M. R. Lyu)

We will develop an automated system which can enable multimedia information capture, search, retrieval, summarization and reuse. We envision multiple information streams (newswires, television and radio broadcasts, archival collections) arriving in different media formats, which require segmentation, indexing, categorization, retrieval, summarization, and on-demand integrated presentation.

We build a Digital Video Library system to explore Internet video exchange concepts, technology and infrastructure. In essence, television and radio documents are indexed via the automatically transcribed audio track, which is processed much like text for retrieval and summarization purposes.

We will perform all or part of the following development tasks:

• Robust indexing and retrieval of audio and video documents. Indexing will be image-based derived from visual content, as well as text-based via connected speech recognition and new statistical natural language processing techniques.

• On-demand summarization of individual documents, or production of synthetic summaries combining information from multiple documents focusing on maximally query-relevant passages and reducing cross-document redundancy.

• Video segmentation and summarization, with tools for extraction, annotation, and reuse of designated content.