What can an Artificial Intelligence chatbot tell us about orthodontic treatment?
A recent development has been Chat GPT. This is an Artificial Intelligence chatbot that condenses information and provides text responses. It is becoming popular and is described as making Artificial Intelligence mainstream. It provides intelligent-sounding text and appears to be plausible. This may have implications for writing scientific papers, essays and blogs.
I came across the use of Chat GPT in a post on the Orthodontic Controversies Facebook group. As a result, the concept of this post has been introduced previously. But I also wanted to take the opportunity to ask several questions.
What did I do?
I asked it to describe the following.
- What are the benefits of orthodontic treatment?
- Can we expedite orthodontic tooth movement?
- Do orthodontic extractions cause harm?
- Is there a relationship between orthodontics and childhood breathing?
- What are orthodontic self-ligating brackets?
- What is orthotropics?
- Who is Kevin O’Brien, the orthodontist?
What did I find?
It was straightforward to find the site and register. Once registered, I was in a queue, probably due to the number of users. Nevertheless, after approximately 10 minutes, I could start asking questions. This was very straightforward; I typed the question into a text box. The chatbot generated a response to each question within a few seconds. Each response was about 300 words, clear and well-written. I analysed the answers using an arbitrary score out of 10 for accuracy.
These are the main points from the responses that I received.
Benefits of orthodontic treatment
- Improved function
- Improved appearance
- Better oral health concerning caries and periodontal disease
- Reduced chance of trauma
- Saving costs by reducing the likelihood of dental treatment later in life
Accuracy score: 5
Speeding up tooth movement?
Response:
Orthodontists may use the following methods to reduce the length of orthodontic treatment.
- Self Ligating brackets
- Vibration
- Piezocison
- Propel MOPs
- Clear aligners
I also asked another question about the evidence supporting these methods. The response was that the overall evidence base needed to be improved. The bot added the rider that people should work closely with their orthodontist.
Accuracy score: 0
Do orthodontic extractions cause harm?
Response:
- Extractions may be necessary to correct crowding, protrusion or some bite issues.
- They are a last resort.
- Orthodontists will carefully evaluate their needs.
Accuracy score 10
Orthodontics and breathing
Response:
Extractions may worsen breathing by reducing the airway’s size and affecting the tongue’s and other tissues’ position.
Accuracy 2
Self-ligation.
The use of SLBs results in
- Less friction
- Reduced treatment time
- Easy to clean
- Fewer adjustments and fewer visits
Accuracy 0
How do functional appliances work?
Response:
- Functional appliances can guide the jaws and teeth into healthier, more functional positions.
Accuracy 3
Who is Kevin O’Brien, the orthodontist?
First response:
“Kevin O’Brien is a highly respected and influential figure in the field of orthodontics, known for his expertise in evidence-based practice, appliance development, and education”.
I’ll take that. It then went on to state.
“Dr O’Brien has also been involved in developing several orthodontic appliances, including the Damon System self-ligating bracket system and the Invisalign aligner system”.
Overall accuracy 2.
What did I think?
This was a fascinating experience. The responses of the chatbot were interesting. They were undoubtedly written clearly and confidently. This gave the impression that the information was accurate. However, it did not reflect the current scientific knowledge we know is available.
I tried to find out the source of information that the Chatbot uses. I discovered that it is not directly connected to the internet. Nevertheless, it gains knowledge from a massive text dataset derived from books, articles and websites. Therefore, it produces responses that need to be more accurate in orthodontics. In effect, it reflects a combination of science and the erroneous claims made in advertising and websites.
Is this a problem? We need to remember that this is not a search engine. It appears to condense readily available information. However, the chatbot’s responses must also be inaccurate if the source information is inaccurate. We need to remember this if our patients and practitioners use ChatGPT.
Finally, it is the early days of this new technology, representing an exciting development. I will return to this again in 12 months to see if it is more refined then,
Emeritus Professor of Orthodontics, University of Manchester, UK.
This nascent technology has a long way to go but also has enormous potential. Right now, the output is based on any available retrospective data, which may not have been validated for accuracy. In the near future, predictive outcomes e.g. virtual orthodontic treatment planning may become viable as AI will assess the data dispassionately in terms of risks, benefits and alternatives –
If you are interested in a primer for understanding how chatGPT and large language models work, S Wolfram wrote an excellent article I highly recommend.
https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/
Garbage in, garbage out. Bias in, bias out. Politicization in, politicization out.
A potentially dangerous way of controlling speech and thought, absolutely. I’d be careful with this dangerous tool.
what was the methodology for assigning accuracy scores?
The Real Person!
The Real Person!
I gave them marks out of 10! not very scientific I know
This is very interesting. I suggest trying this again with Bing Chat, Microsoft’s AI search engine. Bing Chat integrates ChatGPT and the internet via the Bing search engine. I have found that accuracy still varies when searching for orthodontic related material on Bing Chat, but if I ask it to search PubMed then I have found results to be more accurate. Bing Chat will also list the sources and specific research articles where it gathers information.
For example, I asked Bing Chat “Please search pubmed for what are self-ligating brackets”.
Bing Chat Response:
“Some studies have compared self-ligating braces with conventional braces in terms of efficiency, effectiveness, and stability of treatment. The results are mixed and inconclusive. Some studies suggest that self-ligating braces may have some advantages over conventional braces, such as shorter treatment time, less friction, less pain, better oral hygiene, and better aesthetics. However, other studies do not find significant differences between the two types of braces or even report some disadvantages of self-ligating braces, such as more root resorption, more anchorage loss, more interproximal reduction.
Therefore, more high-quality research is needed to determine the therapeutic efficacy of self-ligating brackets. You should consult with your orthodontist about whether they are suitable for you.”
AI is as good as the data it is fed . Suffers the same bias as it’s creator (data)