Usability evaluation for online professional search in the Dutch archaeology domain

  • This paper presents AGNES, the first information retrieval system for archaeological grey literature, allowing full-text search of these long archaeological documents. This search system has a web interface that allows archaeology professionals and scholars to search through a collection of over 60,000 Dutch excavation reports, totalling 361 million words. We conducted a user study for the evaluation of AGNES's search interface, with a small but diverse user group. The evaluation was done by screen capturing and a think aloud protocol, combined with a user interface feedback questionnaire. The evaluation covered both controlled use (completion of a pre-defined task) as well as free use (completion of a freely chosen task). The free use allows us to study the information needs of archaeologists, as well as their interactions with the search system. We conclude that: (1) the information needs of archaeologists are typically recall-oriented, often requiring a list of items as answer; (2)This paper presents AGNES, the first information retrieval system for archaeological grey literature, allowing full-text search of these long archaeological documents. This search system has a web interface that allows archaeology professionals and scholars to search through a collection of over 60,000 Dutch excavation reports, totalling 361 million words. We conducted a user study for the evaluation of AGNES's search interface, with a small but diverse user group. The evaluation was done by screen capturing and a think aloud protocol, combined with a user interface feedback questionnaire. The evaluation covered both controlled use (completion of a pre-defined task) as well as free use (completion of a freely chosen task). The free use allows us to study the information needs of archaeologists, as well as their interactions with the search system. We conclude that: (1) the information needs of archaeologists are typically recall-oriented, often requiring a list of items as answer; (2) the users prefer the use of free-text queries over metadata filters, confirming the value of a free-text search system; (3) the compilation of a diverse user group contributed to the collection of diverse issues as feedback for improving the system. We are currently refining AGNES's user interface and improving its precision for archaeological entities, so that AGNES will help archaeologists to answer their research questions more effectively and efficiently, leading to a more coherent narrative of the past.show moreshow less

Download full text files

Export metadata

Statistics

Number of document requests

Additional Services

Share in Twitter Search Google Scholar
Metadaten
Author:Alex Brandsen, Suzan Verberne, Karsten LambersORCiDGND, Milco Wansleeben
URN:urn:nbn:de:bvb:384-opus4-1242165
Frontdoor URLhttps://opus.bibliothek.uni-augsburg.de/opus4/124216
Parent Title (English):arXiv
Publisher:arXiv
Type:Report
Language:English
Date of Publication (online):2025/08/01
Year of first Publication:2021
Publishing Institution:Universität Augsburg
Release Date:2025/08/05
Page Number:21
DOI:https://doi.org/10.48550/arXiv.2103.04437
Institutes:Philologisch-Historische Fakultät
Philologisch-Historische Fakultät / Digital Humanities
Philologisch-Historische Fakultät / Digital Humanities / Lehrstuhl für Image Processing and Visualization in Digital Humanities
Dewey Decimal Classification:9 Geschichte und Geografie / 93 Geschichte des Altertums (bis ca. 499), Archäologie / 930 Geschichte des Altertums bis ca. 499, Archäologie
Licence (German):CC-BY 4.0: Creative Commons: Namensnennung