Large Language Models In Introductory Programming Education: Chatgpt's Performance And Implications For Assessments · The Large Language Model Bible Contribute to LLM-Bible

The Large Language Model Bible

Research on Large Language Models. Maintained by Sean Moran.

Contact Sean Moran about this survey or website. Made with Jekyll and Hyde.

Large Language Models In Introductory Programming Education: Chatgpt's Performance And Implications For Assessments

Kiesler Natalie, Schiffner Daniel. Arxiv 2023

[Paper]    
GPT Interpretability And Explainability Model Architecture

This paper investigates the performance of the Large Language Models (LLMs) ChatGPT-3.5 and GPT-4 in solving introductory programming tasks. Based on the performance, implications for didactic scenarios and assessment formats utilizing LLMs are derived. For the analysis, 72 Python tasks for novice programmers were selected from the free site CodingBat. Full task descriptions were used as input to the LLMs, while the generated replies were evaluated using CodingBat’s unit tests. In addition, the general availability of textual explanations and program code was analyzed. The results show high scores of 94.4 to 95.8% correct responses and reliable availability of textual explanations and program code, which opens new ways to incorporate LLMs into programming education and assessment.

Similar Work