Emulating Human Cognitive Processes For Expert-level Medical Question-answering With Large Language Models
Verma Khushboo, Moore Marina, Wottrich Stephanie, López Karla Robles, Aggarwal Nishant, Bhatt Zeel, Singh Aagamjit, Unroe Bradford, Basheer Salah, Sachdeva Nitish, Arora Prinka, Kaur Harmanjeet, Kaur Tanupreet, Hood Tevon, Marquez Anahi, Varshney Tushar, Deng Nanfu, Ramani Azaan, Ishwara Pawanraj, Saeed Maimoona, Peña Tatiana López Velarde, Barksdale Bryan, Guha Sushovan, Kumar Satwant. Arxiv 2023
[Paper]
In response to the pressing need for advanced clinical problem-solving tools in healthcare, we introduce BooksMed, a novel framework based on a Large Language Model (LLM). BooksMed uniquely emulates human cognitive processes to deliver evidence-based and reliable responses, utilizing the GRADE (Grading of Recommendations, Assessment, Development, and Evaluations) framework to effectively quantify evidence strength. For clinical decision-making to be appropriately assessed, an evaluation metric that is clinically aligned and validated is required. As a solution, we present ExpertMedQA, a multispecialty clinical benchmark comprised of open-ended, expert-level clinical questions, and validated by a diverse group of medical professionals. By demanding an in-depth understanding and critical appraisal of up-to-date clinical literature, ExpertMedQA rigorously evaluates LLM performance. BooksMed outperforms existing state-of-the-art models Med-PaLM 2, Almanac, and ChatGPT in a variety of medical scenarios. Therefore, a framework that mimics human cognitive stages could be a useful tool for providing reliable and evidence-based responses to clinical inquiries.
Similar Work