Skip to content 
You can listen to the audio version of the article above.
It seems like choosing the right AI chatbot might depend on the language you speak.
A new study found that when it comes to questions about interventional radiology (that’s a branch of medicine that uses imaging to do minimally invasive procedures), Baidu’s Ernie Bot actually gave better answers in Chinese than ChatGPT-4. But when the same questions were asked in English, ChatGPT came out on top.
The researchers think this means that if you need medical information from an AI chatbot, you might get better results if you use one that was trained in your native language. This makes sense, as these models are trained on massive amounts of text data, and they probably “understand” the nuances and complexities of a language better when they’ve been trained on it extensively.
This could have big implications for how we use AI in healthcare, and it highlights the importance of developing and training LLMs in multiple languages to ensure everyone has access to accurate and helpful information.
Baidu’s AI chatbot Ernie Bot outperformed OpenAI’s ChatGPT-4 on interventional radiology questions in Chinese, while ChatGPT was superior when questions were in English, according to a recent study.
The finding suggests that patients may get better answers when they choose large language models (LLMs) trained in their native language, noted a group of interventional radiologists at the First Affiliated Hospital of Soochow University in Suzhou, China.
“ChatGPT’s relatively weaker performance in Chinese underscores the challenges faced by general-purpose models when applied to linguistically and culturally diverse healthcare environments,” the group wrote. The study was published on January 23 in Digital Health.
It sounds like these researchers are doing some really important work! Liver cancer is a huge problem worldwide, and the treatments can be pretty complicated. It can be hard for patients and their families to understand what’s going on.
The researchers wanted to see if AI chatbots could help with this. They focused on two popular chatbots, ChatGPT and Ernie Bot, and tested them with questions about two common liver cancer treatments, TACE and HAIC.
They asked questions in both Chinese and English to see if the chatbots did a better job in one language or the other.
To make sure the answers were good, they had a group of experts in liver cancer treatment review and score the responses from the chatbots. This is a smart way to see if the information is accurate and easy to understand.
It seems like they’re trying to figure out if AI can be a useful tool for patient education in this complex area of medicine. I’m really curious to see what the results of their study show!
That’s really interesting! It seems like the study confirms that AI chatbots are pretty good at explaining complex medical procedures like TACE and HAIC, but they definitely have strengths and weaknesses depending on the language.
It makes sense that ChatGPT was better in English and Ernie Bot was better in Chinese. After all, they were trained on massive amounts of text data in those specific languages. This probably helps them understand the nuances and specific vocabulary related to medical procedures in each language.
This finding could have a big impact on how we use AI in healthcare around the world. It suggests that we might need different AI tools for different languages to make sure patients get the best possible information. It also highlights the importance of developing and training AI models in a wide variety of languages so that everyone can benefit from this technology.
This makes a lot of sense! Ernie Bot’s edge in Chinese seems to come from its training data. Being trained on Chinese-specific datasets, including those with real-time updates, gives it a deeper understanding of medical terminology and practices within the Chinese context.
On the other hand, ChatGPT shines in English, showcasing its versatility and broad applicability. It’s clearly a powerful language model, but it might lack the specialized knowledge that Ernie Bot has when it comes to Chinese medical practices.
This study really highlights how important it is to consider the context and purpose when developing and using AI tools in healthcare. A one-size-fits-all approach might not be the most effective. Instead, we might need specialized AI models tailored to specific languages and medical contexts to ensure patients receive the most accurate and relevant information.
It seems like the future of AI in healthcare will involve a diverse ecosystem of language models, each with its own strengths and areas of expertise. This is an exciting development, and it will be interesting to see how these tools continue to evolve and improve patient care around the world.
“Choosing a suitable large language model is important for patients to get more accurate treatment,” the group concluded.