Study finds Korean LLMs trail global rivals even on home‑turf exam math

Summary

Korea JoongAng Daily reports that a Sogang University research team tested five Korean large language models against leading US and Chinese models on College Scholastic Ability Test (CSAT) questions and advanced essay‑style math problems, finding a stark performance gap. Foreign models like GPT‑5.1, Gemini 3 Pro Preview, Claude Opus 4.5, Grok 4.1 Fast and DeepSeek V3.2 scored between 76 and 92 points, while domestic systems such as Upstage’s Solar Pro‑2, LG AI Research’s Exaone 4.0.1, Naver’s HCX‑007, SK Telecom’s A.X 4.0 and NCSoft’s Llama Varco 8B mostly landed in the 20‑point range, with the weakest model scoring just 2 points. Even when models were allowed to use Python tools and given multiple attempts on a separate EntropyMath benchmark, non‑Korean systems still dominated, suggesting that Korea’s sovereign‑AI push has not yet closed the reasoning gap with frontier models. The authors say they plan to build an international math leaderboard and update tests as newer model versions arrive, but the results reinforce concerns that language‑localized LLMs can lag badly on deeper problem‑solving unless they match global players in both data and algorithmic sophistication.