Dental Benchmarks
With dental experts worldwide, we test how well large models reason in real dental practice—and how safe they are.
8,978 expert-verified questions from 88 countries and regions across 14 dental specialties, testing the clinical reasoning and safety risks of large models in dentistry.
Global Partners
Our partner network covers 10 countries and regions
16 universities, hospitals and research institutes, together with dental experts in Hong Kong, the Chinese mainland and overseas, take part in the development, benchmarking and clinical validation of FreedomAI's dental AI.

Base map: World Map (1:57,000,000, outline edition) from the Standard Map Service of the Ministry of Natural Resources of China Map approval No.: GS(2020)4392
- ChinaThe Chinese University of Hong Kong, Shenzhen, Peking University, National Biomedical Imaging Center, Peking University, Peking-Tsinghua Center for Life Sciences, Shenzhen Stomatology Hospital (Pingshan) of Southern Medical University
- Hong Kong, ChinaFaculty of Dentistry, The University of Hong Kong, The Prince Philip Dental Hospital
- United StatesMayo Clinic
- GermanyLMU University Hospital Munich
- SwitzerlandPartner country
- IndiaPartner country
- MalaysiaSEGi University & Colleges
- AustraliaPartner country
- NigeriaNational Hospital Abuja, College of Medicine, University of Lagos, Bayero University, Usmanu Danfodiyo University, Sokoto, University of Maiduguri
- EthiopiaMadda Walabu University
Why a global dental benchmark
Large language models now approach human experts on medical licensing exams, but real clinical work requires multi-step reasoning with incomplete and ambiguous information. Dentistry is especially demanding: care combines spatial analysis, procedural skills and long-term treatment planning, adapted to each patient. Existing dental LLM benchmarks are mostly limited to one country's licensing exams or individual subspecialties, rely on multiple-choice questions and test memorization more than case-based reasoning.
GlobalDentBench was built to close this gap. According to the paper, it is the first multinational dental benchmark: 8,978 expert-verified questions from 88 countries and regions across six continents, covering 14 dental specialties, 3 question types and 3 progressive reasoning levels. Questions come from official licensing exams, authoritative textbooks and peer-reviewed case reports; the benchmark is built by an automated agent pipeline and calibrated throughout by 6 senior dentists.
The team evaluated 12 frontier large models (6 closed-source, 6 open-source). The closer questions get to real cases, and the more they depend on the individual patient, the lower the scores; on real case questions, nearly one third of answers contained content that could harm patients. The paper concludes that today's large models are suited to supporting knowledge retrieval and synthesis under expert supervision, but cannot yet make dental clinical decisions on their own.
The study was completed by 25 authors from institutions including the Faculty of Dentistry at the University of Hong Kong, The Chinese University of Hong Kong, Shenzhen, Shenzhen Stomatology Hospital (Pingshan) of Southern Medical University, Peking University, Mayo Clinic in the United States and LMU University Hospital Munich in Germany, with members of the FreedomAI team taking part: COO Junjie Zhao and CTO Zhenyang Cai are co-first authors, Chief AI Scientist Benyou Wang and Chief Medical Scientist Junwen Wang are corresponding authors, and CEO Jianquan Li contributed to model inference and result collection. The construction and evaluation code is open-sourced on GitHub, and the open subset GlobalDentBench-OA is available on Hugging Face.
across six continents
Question sources
The paper counts question sources by the UN M49 standard. Below are the 73 countries and regions labeled in the open dataset GlobalDentBench-OA; those ingreensupplied the official dental licensing exams used for the multiple-choice questions.
Asia28
- Chinese mainland
- Hong Kong, China
- Taiwan, China
- India
- Japan
- South Korea
- Saudi Arabia
- Iran
- United Arab Emirates
- Thailand
- Indonesia
- Malaysia
- Türkiye
- Nepal
- Vietnam
- Cambodia
- Iraq
- Israel
- Jordan
- Philippines
- Qatar
- Syria
- Georgia
- Lebanon
- Pakistan
- Singapore
- Sri Lanka
- Palestinian Territories
Europe24
- United Kingdom
- Italy
- Spain
- Germany
- Switzerland
- Netherlands
- France
- Portugal
- Croatia
- Hungary
- Bulgaria
- Greece
- Russia
- Austria
- Belgium
- Ireland
- Sweden
- Albania
- Czechia
- Denmark
- Norway
- Romania
- Slovenia
- Ukraine
Africa7
- Tunisia
- Egypt
- South Africa
- Libya
- Morocco
- Nigeria
- Uganda
North America6
- Canada
- United States
- Mexico
- Costa Rica
- Guatemala
- Puerto Rico
South America6
- Brazil
- Colombia
- Ecuador
- Chile
- Peru
- Venezuela
Oceania2
- Australia
- New Zealand
Benchmark design: 3 reasoning levels × 3 question types × 14 specialties
Knowledge recall
Recalling established dental knowledge, without substantial case reasoning.
Average of 12 models: 74.01%
Routine reasoning
Reasoning mainly from core clinical findings, following a typical diagnostic or treatment path.
Average of 12 models: 55.64%
Individualized reasoning
A final judgment requires the patient's individual information, individual constraints or non-standard clinical factors.
Average of 12 models: 35.71%
Multiple choice MCQ
3,679 questions
Scoring: Answer matching
From officially recognized dental licensing and qualification exams in Australia, New Zealand, Canada, India, the United Kingdom and the United States, testing standardized knowledge.
Average of 12 models: 81.34%
Short answer SAQ
3,709 questions
Scoring: Graded by a judge model (calibrated by dentists)
From authoritative dental textbooks (such as Diagnosis and Treatment Planning in Dentistry); models answer in natural language.
Average of 12 models: 64.53%
Case-based CBQ
1,590 questions
Scoring: 5 key points scored one by one, plus safety risk assessment
Generated from peer-reviewed case reports in high-impact dental journals from 2020–2025, testing clinical reasoning in real scenarios.
Average of 12 models: 22.34%
Covers 14 dental specialties
Built by agents, calibrated by dentists
- 1Collecting authoritative sources
Under the guidance of senior dentists, sources are collected from official licensing and qualification exams, authoritative dental textbooks and peer-reviewed case reports from 2020–2025, with source countries and regions labeled by the UN M49 standard.
- 2Document normalization
The Reformat Agent uses OCR (DeepSeek-OCR2), BeautifulSoup, Pandoc and other tools to convert PDFs, images, Office documents, XML and web pages into Markdown, adding metadata such as titles, sources and authors.
- 3Building questions by type
The Extract Agent chunks first and then extracts: multiple-choice questions become stems, options and answers; short-answer Q&A pairs are extracted; case-based items get a question and 5 scoring key points. Each item goes through at most 3 rounds of self-correction and is discarded if it still fails.
- 4Unified tagging and final review
The Tag Agent labels each question's specialty and reasoning level: if two rounds of prediction agree, the label is accepted; otherwise a third round decides by majority vote. Items enter the benchmark only after self-correction and review by senior dentists.
- 5Type-specific evaluation
Multiple-choice answers are matched directly; short answers are judged by a grading model; case-based answers are scored against 5 key points (20 points each), and every answer is graded for safety: S0 safe, S1 possibly reversible harm, S2 possibly irreversible harm.
- 6Calibrated by dentists throughout
Six licensed senior dentists (average 6.8 years of clinical experience) reviewed sources, refined the workflow and calibrated grading from February 5 to March 25, 2026, contributing 297 person-hours.

Key findings
The closer to real cases, the lower the scores
Average scores of 12 models: 81.34% on multiple choice, 64.53% on short answer and only 22.34% on case-based questions.
Individualized reasoning is a shared weakness
Average scores fall from 74.01% at L1 to 55.64% at L2 and 35.71% at L3; every model scores below 50% at L3.
Pricier models are not necessarily better
Gemini-3.1-Pro-Preview has the highest overall score (63.27%), and GLM-5 leads the open-source models (56.43%). Differences in inference cost far exceed differences in scores—higher cost does not necessarily buy better performance.
Clear differences between specialties
Oral mucosal diseases have the highest average score (63.66%), while orthodontics (50.59%), pediatric dentistry (51.08%) and traditional prosthodontics (51.92%) are the lowest—a 13.07-point gap between highest and lowest.
Nearly one in three answers carries a safety risk
Of 19,080 answers to 1,590 case-based questions, 31.01% were judged unsafe, including 4.51% that could cause irreversible harm; orthodontics had the highest unsafe rate, at 44.30%.
Severe risks concentrate in sensitive specialties
Answers that could cause irreversible harm were most concentrated in “systemic health, pharmacology and safety” (14.15% on average across models), followed by anesthesia and medical emergencies (8.93%) and pediatric dentistry (6.33%).
Automatic grading closely matches dentists' judgment
Dentists agreed with 98.15% of the results from the grading model Gemini-3-Flash-Preview (inter-dentist agreement: 96.67%). Multiple-choice and short-answer items were fully reviewed by experts, with 99.98% accuracy; a sample of 523 case-based items showed 96.78% accuracy.
Overall scores and inference costs of representative models (6 of the 12 evaluated)
| Model | Type | Overall score (macro average) | Cost per 1,000 calls (USD) | Notes |
|---|---|---|---|---|
| Gemini-3.1-Pro-Preview | Closed-source | 63.27% | 18.45 | Highest overall score |
| Gemini-3-Flash-Preview | Closed-source | 61.59% | 3.72 | Best value among closed-source models; the grading model in this study |
| Claude-Sonnet-4.6 | Closed-source | 59.71% | 19.59 | Near the top, at a higher cost |
| Grok-4 | Closed-source | 57.77% | 38.14 | Highest inference cost |
| GLM-5 | Open-source | 56.43% | 5.48 | Highest overall score among open-source models |
| Kimi-K2.5 | Open-source | 55.49% | 1.37 | Outstanding value among open-source models |
Source: Main text of Section 2.3 of the paper (Figures 2C and 2E). The paper has no data table; scores of the other 6 models are given only in figures.



25 authors
Dentists, medical and AI researchers from Hong Kong, the Chinese mainland, the United States and Germany, including members of the FreedomAI team.
Affiliations
FreedomAI expert network
Led by renowned dental experts, specialists and institutions in China and abroad take part in annotation, professional evaluation and clinical validation of FreedomAI's dental AI. This is the company's partner network, not the author list of the paper; Professors Lijian Jin, Falk Schwendicke and Zuolin Jin are authors of GlobalDentBench.

Lijian Jin
Faculty of Dentistry, The University of Hong Kong Paper author
- Council Member, FDI World Dental Federation
- Board Member, International Association for Dental Research (IADR)
- President, IADR Asia Pacific Region
- President, Asian Pacific Society of Periodontology
- Honorary Council Member, Chinese Stomatological Association
- Vice Chair, Guangdong–Hong Kong–Macao Greater Bay Area Oral Health Alliance

Falk Schwendicke
LMU University Hospital Munich Paper author
- Lead of the dental topic group, World Health Organization (WHO) AI for Health
- Convenor, International Organization for Standardization (ISO) working group on AI in dentistry
- Recipient of the IADR Distinguished Scientist Award

Zuolin Jin
Shenzhen Stomatology Hospital (Pingshan) of Southern Medical University Paper author
- Chair, Committee of Orthodontics, Chinese Stomatological Association (COS)
- Council Member, Chinese Stomatological Association
- Executive Committee Member, Asian Pacific Orthodontic Society (APOS)
- Associate Editor, Chinese Journal of Orthodontics
- Member, World Federation of Orthodontists

Ming Zhong
School of Stomatology, China Medical University
- Chief Expert, School of Stomatology, China Medical University
- Recipient of the State Council Special Government Allowance
- Fellow, International College of Dentists
- Chair, Committee of Oral Pathology, Chinese Stomatological Association
- Editor-in-chief of the national standardized residency training textbook Oral Pathology (1st and 2nd editions)
Partner specialists
Bing Liu
Daheng Li
Dongfei Wang
Fangjie Zhou
Fen Liu
Fujia Kang
Hao Chunxiu
Jiamin Zhao
Jiaming Zhang
Jinwen Liu
Jinyan Chen
Junting Wang
Kuan Yang
Lei Shi
Lin Zhao
MAO Yilin
Menghong Li
Mengmeng Song
Min Huang
Minghui Wei
Mingxin Zhang
Patricia Ho
Peijuan Wang
Peng Chen
Rui Lu
Shiduo Sun
Shirong He
Shixin Zhang
Shuwei Huang
Siqi Qiu
Tai Pak Chuen Patrick
Wu Songying
XU Leyao
Xiangjie Jin
Xue Qiao
Yang Jiang
Yaqiu Zhang
Yefeng Wu
Yidan Guo
Yifan Zheng
Ying Wang
Yujia Wu
Zhenlai Zhu
Zhiqiang Deng
Zhuoying LiPartner institutions
The Chinese University of Hong Kong, ShenzhenChina
Faculty of Dentistry, The University of Hong KongHong Kong, China
Peking UniversityChina
National Biomedical Imaging Center, Peking UniversityChina
Peking-Tsinghua Center for Life SciencesChina
The Prince Philip Dental HospitalHong Kong, China
Mayo ClinicUnited States
LMU University Hospital MunichGermany
SEGi University & CollegesMalaysia
National Hospital AbujaNigeria
College of Medicine, University of LagosNigeria
Bayero UniversityNigeria
Usmanu Danfodiyo University, SokotoNigeria
University of MaiduguriNigeria
Madda Walabu UniversityEthiopiaPartner countries and regions
Citation
@article{zhao2026globaldentbench,
title={GlobalDentBench: A Multinational Benchmark for Evaluating LLM Clinical Reasoning in Dentistry with Expert Calibration},
author={Junjie Zhao and Jingyi Liang and Zhenyang Cai and Jiaming Zhang and Zhenwei Wen and Shuzhi Deng and Wenjing Yi and Chunfeng Luo and Hexian Zhang and Junying Chen and Tianrui Liu and Zhuhui Bai and Zixu Zhang and Pradeep Singh and Xiang Liu and Jianquan Li and Nhan L Tran and Falk Schwendicke and Zuolin Jin and Lijian Jin and Liangyi Chen and Wei-fa Yang and Benyou Wang and Junwen Wang and Shan Jiang},
journal={npj Digital Medicine},
year={2026},
note={Accepted},
url={https://arxiv.org/abs/2605.24636}
}




