Segformer++: efficient token-merging strategies for high-resolution semantic segmentation

  • Utilizing transformer architectures for semantic segmentation of high-resolution images is hindered by the attention’s quadratic computational complexity in the number of tokens. A solution to this challenge involves decreasing the number of tokens through token merging, which has exhibited remarkable enhancements in inference speed, training efficiency, and memory utilization for image classification tasks. In this paper, we explore various token merging strategies within the framework of the Segformer architecture and perform experiments on multiple semantic segmentation and human pose estimation datasets. Notably, without model re-training, we, for example, achieve an inference acceleration of 61% on the Cityscapes dataset while maintaining the mIoU performance. Consequently, this paper facilitates the deployment of transformer-based architectures on resource-constrained devices and in real-time applications.

Download full text files

  • 113157.pdfeng

    Postprint. © 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Export metadata


Number of document requests

Additional Services

Share in Twitter Search Google Scholar
Author:Daniel KienzleGND, Marco Kantonis, Robin SchönGND, Rainer LienhartGND
Frontdoor URL
Parent Title (English):IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) 2024, San Jose, CA, USA, August 7-9, 2024
Place of publication:Piscataway, NJ
Type:Conference Proceeding
Year of first Publication:2024
Publishing Institution:Universität Augsburg
Release Date:2024/05/23
Institutes:Fakultät für Angewandte Informatik
Fakultät für Angewandte Informatik / Institut für Informatik
Fakultät für Angewandte Informatik / Institut für Informatik / Lehrstuhl für Maschinelles Lernen und Maschinelles Sehen
Dewey Decimal Classification:0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 004 Datenverarbeitung; Informatik
Latest Publications (not yet published in print):Aktuelle Publikationen (noch nicht gedruckt erschienen)
Licence (German):Deutsches Urheberrecht