English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
  Generation and Grounding of Natural Language Descriptions for Visual Data

Rohrbach, A. (2017). Generation and Grounding of Natural Language Descriptions for Visual Data. PhD Thesis, Universität des Saarlandes, Saarbrücken. doi:10.22028/D291-26708.

Item is

Files

show Files

Locators

show
hide
Description:
-
Locator:
http://scidok.sulb.uni-saarland.de/doku/lic_ohne_pod.php?la=de (Copyright transfer agreement)
Description:
-

Creators

show
hide
 Creators:
Rohrbach, Anna1, 2, Author              
Schiele, Bernt1, Advisor              
Demberg, Vera3, Referee
Darrell, Trevor3, Referee
Affiliations:
1Computer Vision and Multimodal Computing, MPI for Informatics, Max Planck Society, ou_1116547              
2International Max Planck Research School, MPI for Informatics, Max Planck Society, ou_1116551              
3External Organizations, ou_persistent22              

Content

show
hide
Free keywords: -
 Abstract: Generating natural language descriptions for visual data links computer vision and computational linguistics. Being able to generate a concise and human-readable description of a video is a step towards visual understanding. At the same time, grounding natural language in visual data provides disambiguation for the linguistic concepts, necessary for many applications. This thesis focuses on both directions and tackles three specific problems. First, we develop recognition approaches to understand video of complex cooking activities. We propose an approach to generate coherent multi-sentence descriptions for our videos. Furthermore, we tackle the new task of describing videos at variable level of detail. Second, we present a large-scale dataset of movies and aligned professional descriptions. We propose an approach, which learns from videos and sentences to describe movie clips relying on robust recognition of visual semantic concepts. Third, we propose an approach to ground textual phrases in images with little or no localization supervision, which we further improve by introducing Multimodal Compact Bilinear Pooling for combining language and vision representations. Finally, we jointly address the task of describing videos and grounding the described people. To summarize, this thesis advances the state-of-the-art in automatic video description and visual grounding and also contributes large datasets for studying the intersection of computer vision and computational linguistics.

Details

show
hide
Language(s): eng - English
 Dates: 2017-05-162017-06-02
 Publication Status: Published online
 Pages: X, 215 p.
 Publishing info: Saarbrücken : Universität des Saarlandes
 Table of Contents: -
 Rev. Type: -
 Identifiers: BibTex Citekey: Rohrbachphd17
DOI: 10.22028/D291-26708
URN: urn:nbn:de:bsz:291-scidok-68749
 Degree: PhD

Event

show

Legal Case

show

Project information

show

Source

show