English
 
Help Privacy Policy Disclaimer
  Advanced SearchBrowse

Item

ITEM ACTIONSEXPORT
  Improving Semi-Supervised and Domain-Adaptive Semantic Segmentation with Self-Supervised Depth Estimation

Hoyer, L., Dai, D., Wang, Q., Chen, Y., & Van Gool, L. (2021). Improving Semi-Supervised and Domain-Adaptive Semantic Segmentation with Self-Supervised Depth Estimation. Retrieved from https://arxiv.org/abs/2108.12545.

Item is

Files

show Files
hide Files
:
arXiv:2108.12545.pdf (Preprint), 6MB
Name:
arXiv:2108.12545.pdf
Description:
File downloaded from arXiv at 2021-09-28 06:45 arXiv admin note: text overlap with arXiv:2012.10782
OA-Status:
Visibility:
Public
MIME-Type / Checksum:
application/pdf / [MD5]
Technical Metadata:
Copyright Date:
-
Copyright Info:
-

Locators

show

Creators

show
hide
 Creators:
Hoyer, Lukas1, Author
Dai, Dengxin2, Author           
Wang, Qin1, Author
Chen, Yuhua1, Author
Van Gool, Luc1, Author
Affiliations:
1External Organizations, ou_persistent22              
2Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society, ou_1116547              

Content

show
hide
Free keywords: Computer Science, Computer Vision and Pattern Recognition, cs.CV
 Abstract: Training deep networks for semantic segmentation requires large amounts of
labeled training data, which presents a major challenge in practice, as
labeling segmentation masks is a highly labor-intensive process. To address
this issue, we present a framework for semi-supervised and domain-adaptive
semantic segmentation, which is enhanced by self-supervised monocular depth
estimation (SDE) trained only on unlabeled image sequences.
In particular, we utilize SDE as an auxiliary task comprehensively across the
entire learning framework: First, we automatically select the most useful
samples to be annotated for semantic segmentation based on the correlation of
sample diversity and difficulty between SDE and semantic segmentation. Second,
we implement a strong data augmentation by mixing images and labels using the
geometry of the scene. Third, we transfer knowledge from features learned
during SDE to semantic segmentation by means of transfer and multi-task
learning. And fourth, we exploit additional labeled synthetic data with
Cross-Domain DepthMix and Matching Geometry Sampling to align synthetic and
real data.
We validate the proposed model on the Cityscapes dataset, where all four
contributions demonstrate significant performance gains, and achieve
state-of-the-art results for semi-supervised semantic segmentation as well as
for semi-supervised domain adaptation. In particular, with only 1/30 of the
Cityscapes labels, our method achieves 92% of the fully-supervised baseline
performance and even 97% when exploiting additional data from GTA. The source
code is available at
https://github.com/lhoyer/improving_segmentation_with_selfsupervised_depth.

Details

show
hide
Language(s): eng - English
 Dates: 2021-08-272021
 Publication Status: Published online
 Pages: 25 p.
 Publishing info: -
 Table of Contents: -
 Rev. Type: -
 Identifiers: arXiv: 2108.12545
BibTex Citekey: Hoyer2108.12545
URI: https://arxiv.org/abs/2108.12545
 Degree: -

Event

show

Legal Case

show

Project information

show

Source

show