Context Matters: An Empirical Study Of The Impact Of Contextual Information In Temporal Question Answering Systems

Schumacher Dan, Haji Fatemeh, Grey Tara, Bandlamudi Niharika, Karnik Nupoor, Kumar Gagana Uday, Chiang Jason Cho-yu, Rad Paul, Vishwamitra Nishant, Rios Anthony. Arxiv 2024

[Paper]    
Applications Security Training Techniques

Large language models (LLMs) often struggle with temporal reasoning, crucial for tasks like historical event analysis and time-sensitive information retrieval. Despite advancements, state-of-the-art models falter in handling temporal information, especially when faced with irrelevant or noisy contexts. This paper addresses this gap by empirically examining the robustness of temporal question-answering (TQA) systems trained on various context types, including relevant, irrelevant, slightly altered, and no context. Our findings indicate that training with a mix of these contexts enhances model robustness and accuracy. Additionally, we show that the position of context relative to the question significantly impacts performance, with question-first positioning yielding better results. We introduce two new context-rich TQA datasets, ContextAQA and ContextTQE, and provide comprehensive evaluations and guidelines for training robust TQA models. Our work lays the foundation for developing reliable and context-aware temporal QA systems, with broader implications for enhancing LLM robustness against diverse and potentially adversarial information.

Similar Work