Predicting media memorability from a multimodal late fusion of self-attention and LSTM models
- This paper reports on the GTH-UPM team experience in the Predicting Media Memorability task at MediaEval 2020. Teams were requested to predict memorability scores at both short-term and long-term, understanding such score as a measure of whether a video was perdurable in a viewer’s memory or not. Our proposed system relies on a late fusion of the scores predicted by three sequential models, each trained over a different modality: video captions, aural embeddings and visual optical flow-based vectors. Whereas single-modality models show a low or zero Spearman correlation coefficient value, their combination considerably boosts performance over development data up to 0.2 in the short-term memorability prediction subtask and 0.19 in the long-term subtask. However, performance over test data drops to 0.016 and -0.041, respectively.
Download full text files
Statistics
Print On Demand
Additional Services
| Author: | Ricardo Kleinlein, Cristina Luna-JiménezORCiDGND, Zoraida Callejas, Fernando Fernández-Martínez |
|---|---|
| URN: | urn:nbn:de:bvb:384-opus4-1226902 |
| Frontdoor URL | https://opus.bibliothek.uni-augsburg.de/opus4/122690 |
| URL: | https://nbn-resolving.org/urn:nbn:de:0074-2882-4 |
| ISSN: | 1613-0073OPAC |
| Parent Title (English): | MediaEval 2020 - Multimedia Benchmark Workshop 2020: Working Notes proceedings of the MediaEval 2020 Workshop, online, 14-15 December 2020 |
| Publisher: | CEUR-WS |
| Place of publication: | Aachen |
| Editor: | Steven Hicks, Debesh Jha, Konstantin Pogorelov, Alba García Seco De Herrera, Dmitry Bogdanov, Pierre-Etienne Martin, Stelios Andreadis, Minh-Son Dao, Zhuoran Liu, José Vargas-Quirós, Benjamin Kille, Martha Larson |
| Type: | Conference Proceeding |
| Language: | English |
| Date of Publication (online): | 2025/06/04 |
| Year of first Publication: | 2020 |
| Publishing Institution: | Universität Augsburg |
| Release Date: | 2025/06/05 |
| First Page: | 61 |
| Series: | CEUR Workshop Proceedings ; 2882 |
| Institutes: | Fakultät für Angewandte Informatik |
| Fakultät für Angewandte Informatik / Institut für Informatik | |
| Fakultät für Angewandte Informatik / Institut für Informatik / Lehrstuhl für Menschzentrierte Künstliche Intelligenz | |
| Dewey Decimal Classification: | 0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 004 Datenverarbeitung; Informatik |
| Licence (German): |  CC-BY 4.0: Creative Commons: Namensnennung (mit Print on Demand) |
