Sabi\'a-2: A New Generation Of Portuguese Large Language Models
Almeida Thales Sales, Abonizio Hugo, Nogueira Rodrigo, Pires Ramon. Arxiv 2024
[Paper]
We introduce Sabi'a-2, a family of large language models trained on Portuguese texts. The models are evaluated on a diverse range of exams, including entry-level tests for Brazilian universities, professional certification exams, and graduate-level exams for various disciplines such as accounting, economics, engineering, law and medicine. Our results reveal that our best model so far, Sabi'a-2 Medium, matches or surpasses GPT-4’s performance in 23 out of 64 exams and outperforms GPT-3.5 in 58 out of 64 exams. Notably, specialization has a significant impact on a model’s performance without the need to increase its size, allowing us to offer Sabi'a-2 Medium at a price per token that is 10 times cheaper than GPT-4. Finally, we identified that math and coding are key abilities that need improvement.
Similar Work