G-MACT at SemEval-2025 Task 8: exploring planning and tool use in question answering over tabular data
- This paper describes our system submitted to SemEval-2024 Task 8 “Question Answering over Tabular Data.”The shared task focuses on tackling real-life table question answering (TQA) involving extremely large tables with the additional challenges of interpreting complex questions. To address these issues, we leverage a framework of Multi-Agent Collaboration with Tool use (MACT), a method that combines planning and tool use. The planning module breaks down a complex question by designing a step-by-step plan. This plan is translated into Python code by a coding model, and a Python interpreter executes the code to generate an answer. Our system demonstrates competitive performance in the shared task and is ranked 5th out of 38 in the open-source model category. We provide a detailed analysis of our model, evaluating the effectiveness and the efficiency of each component, and identify common error patterns. Our paper offers essential insights and recommendations for future advancements inThis paper describes our system submitted to SemEval-2024 Task 8 “Question Answering over Tabular Data.”The shared task focuses on tackling real-life table question answering (TQA) involving extremely large tables with the additional challenges of interpreting complex questions. To address these issues, we leverage a framework of Multi-Agent Collaboration with Tool use (MACT), a method that combines planning and tool use. The planning module breaks down a complex question by designing a step-by-step plan. This plan is translated into Python code by a coding model, and a Python interpreter executes the code to generate an answer. Our system demonstrates competitive performance in the shared task and is ranked 5th out of 38 in the open-source model category. We provide a detailed analysis of our model, evaluating the effectiveness and the efficiency of each component, and identify common error patterns. Our paper offers essential insights and recommendations for future advancements in developing TQA systems.…
| Author: | Wei Zhou, Mohsen Mesgar, Annemarie FriedrichORCiDGND, Heike Adel |
|---|---|
| URN: | urn:nbn:de:bvb:384-opus4-1264073 |
| Frontdoor URL | https://opus.bibliothek.uni-augsburg.de/opus4/126407 |
| URL: | https://aclanthology.org/2025.semeval-1.100/ |
| URL: | http://979-8-89176-273-2 |
| Parent Title (English): | Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), 31 July - 1 August 2025, Vienna, Austria |
| Publisher: | Association for Computational Linguistics (ACL) |
| Place of publication: | Stroudsburg, PA |
| Editor: | Sara Rosenthal, Aiala Rosá, Debanjan Ghosh, Marcos Zampieri |
| Type: | Conference Proceeding |
| Language: | English |
| Date of Publication (online): | 2025/11/20 |
| Year of first Publication: | 2025 |
| Publishing Institution: | Universität Augsburg |
| Release Date: | 2025/11/26 |
| First Page: | 726 |
| Last Page: | 742 |
| Institutes: | Fakultät für Angewandte Informatik |
| Fakultät für Angewandte Informatik / Institut für Informatik | |
| Fakultät für Angewandte Informatik / Institut für Informatik / Lehrstuhl für Computerlinguistik | |
| Dewey Decimal Classification: | 0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 004 Datenverarbeitung; Informatik |
| Licence (German): |  CC-BY 4.0: Creative Commons: Namensnennung |
