Is Chatgpt A Good Translator? Yes With GPT-4 As The Engine
Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Xing Wang, Shuming Shi, Zhaopeng Tu. Arxiv 2023
[Paper]
[Code]
GPT
Has Code
Model Architecture
Prompting
Security
This report provides a preliminary evaluation of ChatGPT for machine
translation, including translation prompt, multilingual translation, and
translation robustness. We adopt the prompts advised by ChatGPT to trigger its
translation ability and find that the candidate prompts generally work well
with minor performance differences. By evaluating on a number of benchmark test
sets, we find that ChatGPT performs competitively with commercial translation
products (e.g., Google Translate) on high-resource European languages but lags
behind significantly on low-resource or distant languages. As for the
translation robustness, ChatGPT does not perform as well as the commercial
systems on biomedical abstracts or Reddit comments but exhibits good results on
spoken language. Further, we explore an interesting strategy named
for distant languages, which asks ChatGPT to
translate the source sentence into a high-resource pivot language before into
the target language, improving the translation performance noticeably. With the
launch of the GPT-4 engine, the translation performance of ChatGPT is
significantly boosted, becoming comparable to commercial translation products,
even for distant languages. Human analysis on Google Translate and ChatGPT
suggests that ChatGPT with GPT-3.5 tends to generate more hallucinations and
mis-translation errors while that with GPT-4 makes the least errors. In other
words, ChatGPT has already become a good translator. Please refer to our Github
project for more details:
https://github.com/wxjiao/Is-ChatGPT-A-Good-Translator
Similar Work