[{"data":1,"prerenderedAt":23},["ShallowReactive",2],{"news-fr-huatuogpt-research-journey":3},{"slug":4,"lang":5,"title":6,"summary":7,"date":8,"category":9,"tags":10,"cover":15,"order":16,"author":17,"authorTitle":17,"authorSlug":17,"people":18,"bodyLang":21,"html":22},"huatuogpt-research-journey","fr","Parcours de recherche | Cinq générations de la série HuatuoGPT de la CUHK-Shenzhen : du dialogue médical à l'adaptation par apprentissage par renforcement seul, avec un modèle 27B qui atteint 70,1 sur HealthBench","Depuis 2023, des chercheurs de la CUHK-Shenzhen et leurs collaborateurs étudient les grands modèles médicaux à travers la série HuatuoGPT. Plusieurs membres clés de FreedomAI sont auteurs de ces articles, et ces travaux constituent le socle méthodologique de DentalGPT.","2026-11-16","技术",[11,12,13,14],"HuatuoGPT","grands modèles médicaux","apprentissage par renforcement","IA multimodale médicale","/images/news/huatuogpt-research-journey/cover.jpg",11,"",[19,20],"蔡镇阳","王本友","en","\u003Cp>Turning a general-purpose large model into a reliable medical assistant means clearing several hurdles: it has to talk like a doctor, master solid medical knowledge, understand medical images, and reason step by step through complex problems.\u003C/p>\n\u003Cp>Since 2023, researchers at The Chinese University of Hong Kong, Shenzhen and their collaborators have pursued medical large-model research through the HuatuoGPT series: from professional medical dialogue, medical knowledge adaptation and medical image–text understanding, all the way to complex medical reasoning and domain adaptation with reinforcement learning alone, building datasets such as Huatuo-26M and PubMedVision and the CMB benchmark along the way. Several FreedomAI core team members took part in many of these projects. This line of research also provides the methodological foundation in data construction, model training and professional evaluation for DentalGPT, FreedomAI&#39;s multimodal oral-health large model.\u003C/p>\n\u003Ctable>\n\u003Cthead>\n\u003Ctr>\n\u003Cth>First released\u003C/th>\n\u003Cth>Work\u003C/th>\n\u003Cth>Focus\u003C/th>\n\u003Cth>Published in\u003C/th>\n\u003C/tr>\n\u003C/thead>\n\u003Ctbody>\u003Ctr>\n\u003Ctd>May 2023\u003C/td>\n\u003Ctd>HuatuoGPT\u003C/td>\n\u003Ctd>Professional medical dialogue\u003C/td>\n\u003Ctd>Findings of EMNLP 2023\u003C/td>\n\u003C/tr>\n\u003Ctr>\n\u003Ctd>November 2023\u003C/td>\n\u003Ctd>HuatuoGPT-II\u003C/td>\n\u003Ctd>Medical knowledge adaptation\u003C/td>\n\u003Ctd>COLM 2024\u003C/td>\n\u003C/tr>\n\u003Ctr>\n\u003Ctd>June 2024\u003C/td>\n\u003Ctd>HuatuoGPT-Vision\u003C/td>\n\u003Ctd>Medical image–text understanding\u003C/td>\n\u003Ctd>EMNLP 2024\u003C/td>\n\u003C/tr>\n\u003Ctr>\n\u003Ctd>December 2024\u003C/td>\n\u003Ctd>HuatuoGPT-o1\u003C/td>\n\u003Ctd>Complex medical reasoning\u003C/td>\n\u003Ctd>Findings of ACL 2025\u003C/td>\n\u003C/tr>\n\u003Ctr>\n\u003Ctd>October 2026\u003C/td>\n\u003Ctd>HuatuoGPT-3\u003C/td>\n\u003Ctd>Domain adaptation with reinforcement learning\u003C/td>\n\u003Ctd>arXiv preprint (2026)\u003C/td>\n\u003C/tr>\n\u003C/tbody>\u003C/table>\n\u003Cp>Table: Timeline of the HuatuoGPT series. Dates are when each paper was first made public; the last column gives the conference or preprint platform where it was published.\u003C/p>\n\u003Ch2>2023: from answering questions to communicating like a doctor\u003C/h2>\n\u003Ch3>HuatuoGPT: combining real doctors&#39; data with distilled data\u003C/h3>\n\u003Cp>HuatuoGPT is a large model for medical consultation. Its core idea is to use two kinds of data in supervised fine-tuning: data distilled from ChatGPT, with detailed and well-formed answers, and real data from doctors, closer to how doctors actually take histories and make diagnoses. The team then trained a reward model and used RLAIF (Reinforcement Learning from AI Feedback) to align the strengths of both kinds of data in the model.\u003C/p>\n\u003Cp>The paper reports that HuatuoGPT achieved the best medical consultation performance among open-source models at the time on GPT-4 evaluation, human evaluation and medical benchmarks, and in most cases outperformed its &quot;teacher&quot;, ChatGPT. It was published in Findings of EMNLP 2023.\u003C/p>\n\u003Ch3>HuatuoGPT-II: medical adaptation in a single stage\u003C/h3>\n\u003Cp>The usual way to turn a general model into a medical expert is a two-step process of &quot;continued pre-training + supervised fine-tuning&quot;, but the data in the two stages differs greatly in language, genre and format. HuatuoGPT-II converts the heterogeneous data from both stages into simple &quot;input–output&quot; pairs and completes medical domain adaptation in a single stage.\u003C/p>\n\u003Cp>The team validated this approach in fields where ChatGPT was relatively weak, such as traditional Chinese medicine: HuatuoGPT-II led on several Chinese medical benchmarks, including medical licensing exams, and achieved the best result on a brand-new national medical licensing exam. It was published at COLM 2024.\u003C/p>\n\u003Ch3>Data and evaluation: Huatuo-26M and CMB\u003C/h3>\n\u003Cp>In the same period, the team released \u003Cstrong>Huatuo-26M\u003C/strong>, a Chinese medical question-answering dataset of about \u003Cstrong>26 million\u003C/strong> question–answer pairs. It can be used to evaluate retrieval and generation, as external knowledge for retrieval-augmented generation, and for continued training (the paper was later published in Findings of NAACL 2025).\u003C/p>\n\u003Cp>On evaluation, the team proposed \u003Cstrong>CMB\u003C/strong> (Comprehensive Medical Benchmark in Chinese). The authors argue that directly translating English medical benchmarks fits poorly with the local healthcare context, so CMB is designed entirely around the Chinese language and cultural setting; traditional Chinese medicine is an important part of it, but not the whole. It was published at NAACL 2024.\u003C/p>\n\u003Ch2>2024: understanding images and learning to reason\u003C/h2>\n\u003Ch3>HuatuoGPT-Vision: 1.3 million samples of medical visual knowledge\u003C/h3>\n\u003Cp>Constrained by privacy and annotation costs, high-quality medical image–text data has long been scarce. The team refined medical image–text pairs from PubMed and had GPT-4V, which can see the images, denoise and reorganize the data, building \u003Cstrong>PubMedVision\u003C/strong> with \u003Cstrong>1.3 million\u003C/strong> medical visual question-answering samples.\u003C/p>\n\u003Cp>Experiments show that PubMedVision significantly improves the medical abilities of existing multimodal models, including on the Health &amp; Medicine track of MMMU, and manual checks by medical experts confirmed the data quality. HuatuoGPT-Vision, a 34B medical multimodal model trained on this data, outperformed other open-source models in medical multimodal scenarios. It was published at EMNLP 2024.\u003C/p>\n\u003Ch3>HuatuoGPT-o1: training complex reasoning with verifiable problems\u003C/h3>\n\u003Cp>Math problems can be checked against an answer; medical reasoning is hard to verify. HuatuoGPT-o1 rewrites challenging multiple-choice questions from medical exams into open-ended &quot;verifiable medical problems&quot; with a single answer that can be checked objectively, and introduces a medical verifier to check whether the model&#39;s output is correct.\u003C/p>\n\u003Cp>Training has two steps (Figure 1): first, guided by the verifier, the model searches for correct complex reasoning trajectories, which are used for fine-tuning; then reinforcement learning (PPO) with rewards from the verifier further improves complex reasoning. The paper shows that with only \u003Cstrong>40,000\u003C/strong> verifiable medical problems, HuatuoGPT-o1 outperforms general and medical open-source baselines; complex reasoning improves medical problem solving and benefits more from reinforcement learning. It was published in Findings of ACL 2025 as \u003Cem>Towards Medical Complex Reasoning with LLMs through Medical Verifiable Problems\u003C/em>.\u003C/p>\n\u003Cp>\u003Cspan class=\"news-figure\">\u003Cimg src=\"/images/news/huatuogpt-research-journey/fig1.png\" alt=\"Figure 1: HuatuoGPT-o1's two-stage training. Left: searching for complex reasoning trajectories guided by the verifier and using them for fine-tuning; right: reinforcement learning with verifier rewards. Source: the paper\" loading=\"lazy\" />\u003Cspan class=\"news-caption\">Figure 1: HuatuoGPT-o1's two-stage training. Left: searching for complex reasoning trajectories guided by the verifier and using them for fine-tuning; right: reinforcement learning with verifier rewards. Source: the paper\u003C/span>\u003C/span>\u003C/p>\n\u003Ch2>2026: HuatuoGPT-3, adapting with reinforcement learning and no supervised fine-tuning\u003C/h2>\n\u003Cp>&quot;Supervised fine-tuning (SFT) first, then reinforcement learning (RL)&quot; is the mainstream pipeline for domain adaptation. It makes cold starts easy but can reduce the model&#39;s exploratory diversity, and multi-stage optimization adds complexity. Pure on-policy reinforcement learning, on the other hand, struggles with cold starts. Mixed-policy reinforcement learning, which mixes a teacher model&#39;s outputs into training, has two problems of its own: early in training, valuable tokens in the teacher&#39;s outputs are learned too slowly, which the authors call &quot;Gradient Starvation&quot;; late in training, outdated teacher outputs pull the model backwards, which they call &quot;Teacher-Distribution Anchoring&quot;.\u003C/p>\n\u003Cp>To solve this, the team proposes \u003Cstrong>OnePO\u003C/strong> (One-stage Policy Optimization), which treats teacher outputs as a &quot;temporary guide&quot; (Figure 2):\u003C/p>\n\u003Cul>\n\u003Cli>\u003Cstrong>Adaptive Objective Evolution\u003C/strong>: strengthens the learning signal for informative teacher tokens to which the current model assigns low probability, easing gradient starvation;\u003C/li>\n\u003Cli>\u003Cstrong>Teacher Retirement\u003C/strong>: a teacher output is kept in training only when its reward is strictly higher than the best reward among the current model&#39;s outputs in the same group; once the model catches up, the teacher output automatically &quot;retires&quot; and training returns to standard on-policy reinforcement learning.\u003C/li>\n\u003C/ul>\n\u003Cp>\u003Cspan class=\"news-figure\">\u003Cimg src=\"/images/news/huatuogpt-research-journey/fig2.png\" alt=\"Figure 2: Overview of OnePO. Early in training, teacher outputs speed up the learning of domain knowledge; the model then gradually returns to exploring on its own. Source: the paper\" loading=\"lazy\" />\u003Cspan class=\"news-caption\">Figure 2: Overview of OnePO. Early in training, teacher outputs speed up the learning of domain knowledge; the model then gradually returns to exploring on its own. Source: the paper\u003C/span>\u003C/span>\u003C/p>\n\u003Cp>In the medical domain, OnePO reaches 67.2 on HealthBench (Total) with only 20,000 training samples, 2.7 points above SFT+RL and 7.4 points above pure RL. With more training data, the team trained the open-source medical model series \u003Cstrong>HuatuoGPT-3\u003C/strong>, in 9B and 27B versions based on Qwen3.5-9B and Qwen3.8-27B. The 27B version scores \u003Cstrong>70.1\u003C/strong> on HealthBench (Total) and \u003Cstrong>71.4\u003C/strong> on HealthBench Professional, above the 57.8 and 62.5 of GPT-6 Astra (high), the comparison model in the paper (Figure 3).\u003C/p>\n\u003Cp>\u003Cspan class=\"news-figure\">\u003Cimg src=\"/images/news/huatuogpt-research-journey/fig3.png\" alt=\"Figure 3: HealthBench Professional scores versus model size. Source: the paper\" loading=\"lazy\" />\u003Cspan class=\"news-caption\">Figure 3: HealthBench Professional scores versus model size. Source: the paper\u003C/span>\u003C/span>\u003C/p>\n\u003Cp>The HuatuoGPT-3 technical report was released on arXiv in October 2026 as an extended version of the ICML 2026 paper on OnePO, and the models and code are open source.\u003C/p>\n\u003Ch2>From HuatuoGPT to DentalGPT\u003C/h2>\n\u003Cp>Dentistry is a field that relies heavily on imaging and requires highly specialized knowledge, and the methods developed in the HuatuoGPT series carry over to it:\u003C/p>\n\u003Cul>\n\u003Cli>\u003Cstrong>How reasoning data is built\u003C/strong>: DentalGPT builds its complex reasoning (chain-of-thought) data directly with the HuatuoGPT-o1 method.\u003C/li>\n\u003Cli>\u003Cstrong>From general medical data to specialist data\u003C/strong>: the DentalGPT preprint found that images involving teeth make up only about 0.3% of PubMedVision, so the team built a dedicated dataset of more than 120,000 dental images.\u003C/li>\n\u003Cli>\u003Cstrong>Specialist evaluation\u003C/strong>: in DentalGPT&#39;s comparative experiments, the medical multimodal model HuatuoGPT-Vision averages 47.0% accuracy on five dental benchmarks, while DentalGPT, a 7B model trained on dental data with reinforcement learning, reaches 67.1%. The DentalGPT paper concludes that domain-specific data and training strategies are essential for dental AI.\u003C/li>\n\u003Cli>\u003Cstrong>Reinforcement learning\u003C/strong>: from PPO in HuatuoGPT-o1 to GRPO in DentalGPT and OnePO in HuatuoGPT-3, reinforcement learning runs through the second half of this line of research.\u003C/li>\n\u003C/ul>\n\u003Ch2>Team and impact\u003C/h2>\n\u003Cp>These results come from researchers at The Chinese University of Hong Kong, Shenzhen and their collaborators, with FreedomAI core team members taking part in many of them. FreedomAI Chief AI Scientist Benyou Wang is one of the authors of the papers above; FreedomAI CTO Zhenyang Cai is the second author of HuatuoGPT-o1 and one of the authors of HuatuoGPT-Vision. The first author of HuatuoGPT-II, HuatuoGPT-Vision, HuatuoGPT-o1 and HuatuoGPT-3 is Junying Chen.\u003C/p>\n\u003Ch2>Papers and resources\u003C/h2>\n\u003Cul>\n\u003Cli>HuatuoGPT (Findings of EMNLP 2023): \u003Ca href=\"https://aclanthology.org/2023.findings-emnlp.725/\" target=\"_blank\" rel=\"noopener noreferrer\">https://aclanthology.org/2023.findings-emnlp.725/\u003C/a>\u003C/li>\n\u003Cli>HuatuoGPT-II (COLM 2024): \u003Ca href=\"https://openreview.net/forum?id=eJ3cHNu7ss\" target=\"_blank\" rel=\"noopener noreferrer\">https://openreview.net/forum?id=eJ3cHNu7ss\u003C/a>\u003C/li>\n\u003Cli>HuatuoGPT-Vision (EMNLP 2024): \u003Ca href=\"https://aclanthology.org/2024.emnlp-main.418/\" target=\"_blank\" rel=\"noopener noreferrer\">https://aclanthology.org/2024.emnlp-main.418/\u003C/a>\u003C/li>\n\u003Cli>HuatuoGPT-o1 (Findings of ACL 2025): \u003Ca href=\"https://aclanthology.org/2025.findings-acl.751/\" target=\"_blank\" rel=\"noopener noreferrer\">https://aclanthology.org/2025.findings-acl.751/\u003C/a>\u003C/li>\n\u003Cli>HuatuoGPT-3 (arXiv 2026): \u003Ca href=\"https://arxiv.org/abs/2610.05966\" target=\"_blank\" rel=\"noopener noreferrer\">https://arxiv.org/abs/2610.05966\u003C/a>\u003C/li>\n\u003Cli>Huatuo-26M (Findings of NAACL 2025): \u003Ca href=\"https://aclanthology.org/2025.findings-naacl.211/\" target=\"_blank\" rel=\"noopener noreferrer\">https://aclanthology.org/2025.findings-naacl.211/\u003C/a>\u003C/li>\n\u003Cli>CMB (NAACL 2024): \u003Ca href=\"https://aclanthology.org/2024.naacl-long.343/\" target=\"_blank\" rel=\"noopener noreferrer\">https://aclanthology.org/2024.naacl-long.343/\u003C/a>\u003C/li>\n\u003Cli>Code and models: \u003Ca href=\"https://github.com/FreedomIntelligence\" target=\"_blank\" rel=\"noopener noreferrer\">https://github.com/FreedomIntelligence\u003C/a>\u003C/li>\n\u003C/ul>\n",1794841747814]