Semantic annotation and retrieval of documentary media objects

Abstract

This paper aims to propose a system for the semantic annotation of audio-visual media objects, which are provided in the documentary domain. It presents the system's architecture, a manual annotation tool, an authoring tool and a search engine for the documentary experts. The paper discusses the merits of a proposed approach of evolving semantic network as the basis for the audio-visual content description. Design/methodology/approach -- The author demonstrates how documentary media can be semantically annotated, and how this information can be used for the retrieval of the documentary media objects. Furthermore, the paper outlines the underlying XML schema-based content description structures of the proposed system. Findings -- Currently, a flexible organization of documentary media content description and the related media data is required. Such an organization requires the adaptable construction in the form of a semantic network. The proposed approach provides semantic structures with the capability to change and grow, allowing an ongoing task-specific process of inspection and interpretation of source material. The approach also provides technical memory structures (i.e. information nodes), which represent the size, duration, and technical format of the physical audio-visual material of any media type, such as audio, video and 3D animation. Originality/value -- The proposed approach (architecture) is generic and facilitates the dynamic use of audio-visual material using links, enabling the connection from multi-layered information nodes to data on a temporal, spatial and spatial-temporal level. It enables the semantic connection between information nodes using typed relations, thus structuring the information space on a semantic as well as syntactic level. Since the description of media content holds constant for the associated time interval, the proposed system can handle multiple content descriptions for the same media unit and also handle gaps. The results of this research will be valuable not only for documentary experts but for anyone with a need to manage dynamically audiovisual content in an intelligent way.