- Short-form video (SFV) services, such as TikTok and Instagram Reels, have rapidly gained widespread popularity, accumulating billions of users. However, evaluating Quality of Experience (QoE) for these services poses challenges as they are typically consumed in mobile and interactive settings. In this paper, we introduce a novel QoE evaluation framework for SFV, which allows for a controlled presentation of stimuli and a reliable collection of valid QoE ratings in an unsupervised setting, while maintaining the authenticity of the mobile, interactive SFV experience. We use our framework to conduct two QoE studies on the impact of waiting times, i.e., initial delay and stalling, on the QoE of SFV via crowdsourcing. Our findings reveal that initial delay results in a three times higher probability that users swipe to the next video within the first ten seconds compared to stalling. In contrast, the Mean Opinion Score (MOS) of stalling is up to 0.4 lower than initial delay for the sameShort-form video (SFV) services, such as TikTok and Instagram Reels, have rapidly gained widespread popularity, accumulating billions of users. However, evaluating Quality of Experience (QoE) for these services poses challenges as they are typically consumed in mobile and interactive settings. In this paper, we introduce a novel QoE evaluation framework for SFV, which allows for a controlled presentation of stimuli and a reliable collection of valid QoE ratings in an unsupervised setting, while maintaining the authenticity of the mobile, interactive SFV experience. We use our framework to conduct two QoE studies on the impact of waiting times, i.e., initial delay and stalling, on the QoE of SFV via crowdsourcing. Our findings reveal that initial delay results in a three times higher probability that users swipe to the next video within the first ten seconds compared to stalling. In contrast, the Mean Opinion Score (MOS) of stalling is up to 0.4 lower than initial delay for the same waiting time conditions. These insights provide valuable guidelines for optimizing SFV content delivery to enhance user satisfaction, but also highlight the need for novel QoE models, which can describe not only perceived QoE but also resulting user engagement and behavior.…
