TCDA: Temporally-Consistent Depth-Aware Unsupervised Domain Adaptation for Semantic Segmentation in Urban Scenes

• Main Author: Barbosa, Felipe Manfio
• Publication Date: 2025
• Format: Master thesis
• Language: eng
• Source: Biblioteca Digital de Teses e Dissertações da USP
• Download full: https://www.teses.usp.br/teses/disponiveis/55/55134/tde-07082025-113420/
• topic: Adaptação de domínio não supervisionada; ADAS; ADAS; Aprendizado auto-supervisionado; Aprendizado profundo; Autonomous vehicles; Consistência temporal; Deep learning; Depth; Multimodal perception; Percepção multimodal; Profundidade; Segmentação semântica; Self-supervised learning; Semantic segmentation; Temporal consistency; Unsupervised domain adaptation; Veículos autônomos

Visual perception plays a fundamental role in the correct localization and navigation of autonomous vehicles in urban environments. In this scenario, Semantic Segmentation is a valuable technique, usually derived from RGB images, for identifying elements and their relationships in a given scene. With the advent of Convolutional Neural Networks and Deep Learning, Semantic Segmentation methods reached a new level in terms of accuracy (90% in Cityscapes dataset). Nonetheless, this comes at the cost of heavy and complex network architectures that require high-cost embedded processing centers. Additionally, adapting these models to new operation domains remains an unsolved issue, mainly because fine-tuning for all possible target domains is impractical given the high costs of data annotation. Although simulation engines can provide large amounts of labeled data, the Domain shift between synthetic and real domains still hinders direct translation to application scenarios. In light of that, Unsupervised Domain Adaptation has emerged as a promising alternative by exploring unlabeled data and self-supervised learning as helper tools for Domain Adaptation. However, the majority of the literature builds upon single- frame RGB-only methods. Taking this into account, we argue that a huge opportunity for leveraging Temporal and Depth data as auxiliary sources of supervision is being disregarded. While Depth data has strong structural/geometric value and tend to be more robust to domain shifts, Temporal data provide large amounts of unlabeled data and useful temporal cues for helping in the adaptation process. Therefore, in this work we propose a synthetic-to-real Unsupervised Domain Adaptation method that builds upon three cornerstones: efficiency, depth and temporal awareness. The results demonstrate that the proposed method improves both segmentation quality and temporal consistency in the target domain. Specifically, it achieves an overall improvement of approximately 250% in the mean Intersection over Union (mIoU) for critical classesroad, sidewalk, car, and personcompared to the baseline (non-adapted) model.