Dental Benchmarks

With dental experts worldwide, we test how well large models reason in real dental practice—and how safe they are.

In progress

Where does expert-level dental reasoning go wrong, and what supervision is needed to correct it?

After GlobalDentBench, the team is building the more challenging OmniDentBench. Questions come from open-access case reports and research papers in PubMed Central; they are drafted by agents, filtered for difficulty by frontier models and reviewed one by one by dental experts in China and abroad, to test large models on expert-level clinical decisions and find out what kind of professional supervision they need most.

1,000+challenging case questions
12dental specialties
6ability and safety dimensions
8types of imaging evidence

Ability and safety dimensions

Clinical reasoningTreatment and managementPatient safetyEthical and legal boundariesProfessional judgmentCommunication quality

Question formats

  • Objective single-choice questions
  • Objective exact-answer questions (equivalent forms accepted)
  • Open-ended questions (weighted rubrics + safety red lines)

Imaging evidence

Intraoral photosPanoramic radiographsPeriapical radiographsCephalometric radiographsCBCT and CTMRIHistopathology imagesDigital dental models and scans
  1. 1
    Source tracing and multimodal evidence assembly

    Case reports and research papers from PubMed Central open access are screened for dental relevance and source eligibility; full texts and related images are extracted, keeping traceable multimodal evidence.

  2. 2
    Agent-drafted questions and difficulty screening

    GPT-5.4 drafts candidate questions from the retained text and images. After quality control with checking and correction, GPT-5.4, Gemini-3-Flash-Preview and Claude-Opus-4.6 screen them for difficulty, and only qualified questions move on to expert review.

  3. 3
    Expert review and inclusion

    Three committee members vote independently without seeing each other's votes, and at least two must approve. The final review involves a Chinese specialty committee chair and at least one overseas expert, and a question is included only if all final reviewers agree.

Preliminary results show that frontier models still have clear weaknesses in expert-level dental reasoning, and some answers cross safety red lines. Full results will be released with the paper.

OmniDentBench overview: 12 specialties, 6 ability and safety dimensions, 3 question formats and 8 types of imaging evidence (source: OmniDentBench poster)
OmniDentBench overview: 12 specialties, 6 ability and safety dimensions, 3 question formats and 8 types of imaging evidence (source: OmniDentBench poster)
Expert-governed three-stage construction pipeline (source: OmniDentBench poster)
Expert-governed three-stage construction pipeline (source: OmniDentBench poster)

ContributorsJunjie Zhao, Ziying Sheng, Jiajun You, Zuolin Jin, Junwen Wang, Benyou Wang, and dental experts in China and abroad from the OmniDentBench Consortium

InstitutionsThe University of Hong Kong, Beijing Institute of Collaborative Innovation, The Chinese University of Hong Kong, Shenzhen, Shenzhen Loop Area Institute, Shenzhen Stomatology Hospital (Pingshan) of Southern Medical University