Can orthodontics be evidence-based?
I receive many comments on my blog and individual posts. These vary from praise to extreme personal criticism. Recently, Jean-Marc E. Choufani, a first-year resident from a programme in New England, made some tremendous perceptive comments following my recent post on MSE. I thought it would be good to publish these in full along with my response.
Introduction
As much as I admire your systematic and thorough approach to evaluating the articles you review, I sometimes wonder whether certain conclusions regarding quality of evidence and risk of bias are genuinely worth the assigned weights in day-day practice.
After reviewing the book “evidence-based orthodontics” by Huang et al. – I found it quite entertaining how very few procedures and clinical decisions in orthodontics are truly evidence-based.
Research evidence
Does this absence of evidence mean it doesn’t work? Should we focus more on evaluating the cost-benefit ratio for each patient when proposing treatment plans? Of course, it should be considered incompetence to suggest something sure to cause harm to the patient. However, how do we assess this in orthodontics?
When looking at some of the biological effects of orthodontic treatment, many patients are arguably ‘healthier’ before orthodontic treatment, especially when side-effects of treatment and mechanotherapy are not anticipated.
Although the implications of these comments may seem to defeat the purpose of carrying out studies as the scientific method dictates, I’d like to mention an article published by Straus and Sackett in 1999 titled “Applying evidence to the individual patient” that provided me with some clarity when thinking about this post.
Non-lethal nature of orthodontics
Due to the non-lethal nature of orthodontic treatment, the ‘medical’ approach to evidence-based practice may not be as crucial as in other medical specialties. In theory, offering an approach that might be more invasive and expensive, for example, the techniques advocated by Wilmes or De Clerk) maybe logical. Particularly if the treatment has the potential to provide a result that may surpass conventional treatment. We may also eliminate the need for appliances such as extra-oral headgears and facemasks.
However, this doesn’t mean one should advocate trying everything and anything available in the market. Some may argue the need for statistical and clinical significance evaluated in a 6-10mm profession may be unwarranted when considering all the factors involved in certain types of orthodontic treatment. Might it not be more appropriate to develop a system of guidelines that can help gear practitioners make better decisions, not based on experience, but on viable and effective options?
Popular methods
Besides statistical significance, we should consider why specific methods are gaining more traction other than being novel. Although I’m taking the stance of a ‘devil’s advocate,’ I wonder about certain conclusions made about clinical studies in a field that tackles so much biological variability where ‘all roads lead to Rome’, so to speak. I understand that there is a dire need for a scientific approach to orthodontic treatment. Indeed, we may realign the focus to address the most purposeful and effective techniques for tackling the problem. However, this may not be enough. Certainly, patient psychology and quality of life should always determine why one treatment could be selected over another. Perhaps, we need to develop these outcomes further?
Final reflections
I would like to share a disclaimer that I am currently an orthodontist in training. My comments reflect the limited information I’ve gained over the past few months in my residency. I’ve shared my reflections to stir up discussion. I’m aware that this may lead to criticism, and it is welcomed! However, can we truly move forward and evolve unless we share our philosophies and put our ideas to the test? I’ll leave that for you to decide.
What did I think?
I thought these were all great points, and I would like to respond as best I can. So a good place to start would be my first post on my blog. I outlined the aims of this blog. It is worth revisiting. This post was published on September 1st, 2013, which seems long ago. I stated
“Over the past ten years there has been a marked increase in the number of high quality prospective trials that have been published. They have provided clear evidence on the effectiveness of our treatment and delivery of care. However, at the same time there has been greater changes to practice that occurred because of the effect of statements of so-called “gurus” and advertising. Orthodontists have accepted many of these claims and we have to consider if we are losing our way as an evidenced based specialty”?
Evidence base
This, in effect, is the challenge that Jean-Marc raised. His first point was about whether we have an evidence base because most of our trials are deemed to be at risk of bias and that there is generally an absence of evidence. My response is simple, I agree. Many of our trials are small, and they are at high risk of bias according to most assessment tools used. There are two reasons for this. Firstly, we are still gaining experience in trials, as this is still a relatively new technique for orthodontists. Furthermore, the risk of bias tools are very unforgiving when applied to orthodontics. For example, we can’t blind the operators and patients to the interventions.
We need to consider how to deal with this criticism. I feel that we need to interpret all these classifications in light of the nature of orthodontics and come to our own conclusions. A great example of this has been the recent discussion on intercepting impacted canines. Contributors made comments pointing out that removing a primary canine is a very low-risk procedure. As a result, should continue this practice, even if the evidence for its success is not strong. In effect, we are trying to do good with minimal intervention. The most important thing that we can do is interpret the literature using our critical appraisal skills.
This, of course, is very different from the promotion of methods that are more costly or likely to cause harm to our patients. This is the realm of the few extreme KOLs and snake oil salespeople who blight our speciality. I do not feel that they can hide behind the non-lethal nature of orthodontics.
Popular methods and all roads lead to Rome.
Like many thinking orthodontists, I am heavily influenced by Lysle Johnston. One of his most compelling quotes is that “everything works”. This is entirely correct. I agree with this concept when applied to morphological change from treatment. It is rare to see an intervention that does not do some “good”. However, we do not know the effects of our therapy on patient values. Again, I agree with Jean-Marc about the need to measure quality of life to make conclusions about arguably the most important effects of treatment.
Finally, for the benefit and safety of our patients,we do need to put our ideas to the test. We can only progress by carrying out the optimum research design, increasing education in critical appraisal, applying this to our publications, and finally stopping making unsubstantiated claims on the effect of treatment. It is only then that we can call orthodontic treatment an evidence-based speciality. We are on the way to achieving this, and great comments like this help us think about the way forwards.
Emeritus Professor of Orthodontics, University of Manchester, UK.
Excellent letter from Dr. Choufani. I share his concerns.
First, my hat off to Jean-Marc E. Choufani for the conviction and extreme bravery to open our eyes. Sometimes I do not dare to state my opinion after 30 years of clinical practice and Jean-Marc is doing that only a few months into residency. Bravo! What a wonderful future for all of us if we have residents like Jean-Marc.
To add to this healthy debate I would suggest that a missing link is the complexity of the informed consent process in orthodontics. When, and if available, the certainty of the available evidence is extremely complex to be properly articulated to our prospective patients. Minors, mature minors, legal guardians all should receive the same basic information but each of these subgroups requires carefully crafting of the message. If we struggle to properly synthesize what is known about management option X imagine how the message gets reshaped when explaining it to our prospective patients. How can they make a real well-informed decision? Lots to be properly understood in the critical step when delivering elective medical care. This is even before we discuss the need for continuous informed consent in orthodontics.
I’m flattered by your comments but I have to warn you that I do not have skin in the game as the rest of you do. Residency feels like a lucid dream. The purpose of my reflections was to elicit open dialogue to learn from those of you who are in practice and dealing with the real world. I’m eager to hear and learn from all who chose to engage and share their thoughts.
I read the evidence-based orthodontics review books by Huang and by Miles.
There are few treatments that orrhodontists offer with strong scientific basis.
But the reason is that orthodontists keep re-defining the rules of science. We keep moving the goalposts on what we accept as “strong” science.
This means we reject what we previously thought was scientifically evidenced.
Can anyone even give a universal and enduring definition of “science” that is just a few words, which the ordinary person can understand and remember?
Actually, science is a thought technology that, like all technologies, gets replaced by evolved improvements.
With this perspective, Lysle Johnston was aware,
When he said “everything works”, this is probably true when assessed within the relevant historical context.
We all know it’s a mistake to judge a value system outside if its hisrtorical context. Science is no different.
After 39 years of practice I am dumbfounded by the focus on appliances. The teeth do not know how the force was applied. They know direction, duration, and degree of force. It is irrelevant how the force is applied, When I was in my residency the focus was on treatment objectives. Once the objectives were determined a treatment plan that delivered the proper force system was developed. Now in our woke world the focus is on how the patient feels. Hopefully, the nobility of the profession will be resurrected by the application of evidence based therapy.
Jean-Marc E. Choufani has made a wonderful start in the specialty. Being skeptical and questioning everything provides a great foundation to build on.
Even though our knowledge may be incomplete and the evidence may be imperfect, we are aware that some treatments are ineffective or are a waste of time.
We know so much more today than we did a decade or two ago. Our research designs are far better than they have ever been and the evidence they provide will only continue to improve.
Some of us are aware that the snake oil treatments promoted by KOL’s are only effective in lining their pockets and those of the companies they represent.
Lysle Johnson provided a compass to allow us to escape from the guru-based orthodontics of Angle, Tweed, Begg, Ricketts and more recently Damon. His studies, by today’s standards, may be less than ideal, but his conclusions have been correct. We know that we cannot grow mandibles. The view from the shoulder of this great man is clear and bright.
Some treatments do not require evidence to show that they are patently illogical. The perpetual motion machines, or boot strap treatments, such as the pendulum appliance and the Clark trombone appliance are good examples. The Carriere appliance could be considered an expensive method of using Class ll elastics with considerable undesirable side effects.
We may not know everything, but really do know so much about what to do, and more importantly, what not to do. Everything works to a degree, some things are better and a few are best.
Wonderful discussion and I come to realization that the journey is the destination. I would like to recall my guest editorial in The Angle Orthodontist titled ” An Orthodontist’s Data”
[Angle Orthod (2018) 88 (2): 246. https://doi.org/10.2319/0003-3219-88.2.246%5D
“In contemporary perspective, evidence-based patient management has become the cynosure of all eyes in medicine and dentistry. Many medical treatments, including drug therapeutics fall into this scheme but I find the complexity of orthodontic management hard to align with the concept of “one size fits all” at this moment. Most recently, for many medical and pharmaceutical treatments, it is now just becoming recognized that individual, and sometimes genetic, susceptibility should be considered and analyzed when formulating the most effective care strategy. Under the pervasive biological and mechanical limitations faced during orthodontic care, we should be equally (or perhaps even more) prepared to acknowledge that individual characteristics need to guide treatment decisions. It is naïve to think that “Clear evidence shows…” will ever be able to define a “best” treatment pathway for all patients.
At this moment, I assert that we need to move beyond the simplified “evidence-based” ideology that is being developed to define treatment decision-making. There are many individual, as of yet undefined, patient characteristics that potentially …….”
What an excellent article, especially from an orthodontist in training. How times have changed! Critical thinking as opposed to training in my day when certain “gurus” of a different kind told us what was right. Remember the saying…”extract four 4’s…if not why not”! I am sure it rings a few bells.Which trainee in those days would dare to argue!
Danny, you don’t know me but I have been known to enjoy stirring up the pot. I only do so because I find the best conversations to take place when someone’s emotional reaction elicits further introspection, leading to comments and reflections they didn’t even know they had..
I have no qualm with convention, because I really believe experience can be crucial – the concept of skin in the game is very important to me. I just think that the value of experience is definitely overexaggerated and abused.
I would like to refer you to a short but wonderful piece titled Ideas and Idealism –
https://meridian.allenpress.com/angle-orthodontist/article/1/1/14/51401/IDEALS-AND-IDEALISM
What a great discussion !
My compliments to Dr Choufani for his perceptive points. I have been a sceptic of evidence-based treatment particularly the validity of systematic reviews precisely because of the flaws of orthodontic treatment research to date.
Patient biological variability and treatment cooperation plus the lack of true controls question the validity of comparisons. Wouldn’t it be great if we could do true double blind random trials. But as Dr O’Brien mentions “we can’t blind operators and patients” Twenty-four (+/-) month treatments make it even more difficult.
What has not been mentioned is the long- term stability of orthodontic results.
Lysle Johnstone was quoted “everything works”. But does it ???
The evidence has great limitations between bias, number of samples, time limitations, non-homogeneous samples, etc., they have great advantages and disadvantages as any tool created by man has, that is why we must know how, when and where to use them.
Hence, in theory it can only give recommendations and not dictate laws or rules, we all know that the evidence is only part of decision making, the Doctor’s experience and patient’s decisions are as important as the first, perhaps even in many cases more important if we take into account that only a fraction of professional knowledge is supported by evidence.
With respect to health, today the evidence mostly tells us that orthodontics does not damage tissues, it does not produce musculoskeletal problems such as TMJD, however, clinicians most think otherwise.
And there is more, a little evidence points to the opposite on this controversial issue, but it seems that here in this field the voice of the majority counts, in the end, making evidence becomes just a democratic exercise, however with quite often history has shown us that in science the majority has been miserably wrong.
And contrary to what most might think, including Choufani, the evidence is not really based on the scientific method since the data is obtained directly from the computer monitor and not from raw data, from data extracted from the field, from the action same as required by the classical method, where is direct observation, field experimentation? the scientific method has a 100% experiential character.
For the same reason, the evidence can only be a dance of numbers without a context far from reality, there are voices of alarm that already give an account of it, we do not understand our biological framework in the first place, that is why there are more and more invasive procedures, as you rightly mention Choufani, and it is mistakenly believed that there are no consequences, so how can we do science!?
And I’m not just talking about new and invasive treatments, Orthodontics has many accepted procedures that are true Iatrogenic, for that reason I don’t agree with Jhonston’s “everything works”, it works in relation to what? For example, multiple treatments that are completely different from each other can “correct” class II, but they do it in different ways, they affect the form and function in different ways, that is, they affect the biology, health and profile of the patient in different ways.
Great discussion! However, I believe what Dr. Johnston said was “If everything works; nothing matters.” So he was really asking us to discern the efficacy of treatment effects with evidence, and to always be critical of the snakeoil promises.
I have to agree with Dr McCray. What Dr Johnston meant when he said “When everything works nothing matters”, runs somewhat parallel to Dr Choufani’s statement on the “Non-lethal effects of Orthodontic treatment”. When nobody gets hurt by a treatment outcome it is all fun and games, who is bothered about evidence. Dr Johnston goes on to say, the buck stops at the chairside (which is basically us), nobody would know how well you have finished the case. The patient with severe crowding extremely happy with their teeth alignment after the first 6 weekly appointment with an 0.014″ archwire wondering and asking how much longer and why it should take 18-24 more months or a dentist trying out his hand at braces having the same excitement when he sees the same result, both not knowing exactly what more needs to be done (and thus the inception of 6 months braces), the extent to which one can pursue perfection in orthodontics. Both go around sharing their pictures in excitement on social media platforms. It is because orthodontics rarely causes any lethal ill effects to the human body (and pays the bills), he makes the statement ” when everything works nothing matters”, Dr Johnston goes on to to say (in favour of evidence and RCT’s) that ” Is there any answer to any question that would cause you to change the way you practice” and if the answer is “No” then “Turn off the lights the party is over”. The search for unbiased evidence maintains orthodontics as a science and keeps companies and their KOL’s from taking over our profession. I would like to end by quoting Yeats “The best lack all conviction while the worst are full of passionate intensity”. Dr Kevin O’ Brien thank you for having the conviction to pursue the truth, through evidence in orthodontics.
To know exactly what Dr Lysle Johnston meant by his statement you could listen to a lecture given by him titled ” Through a Glass Darkly: Orthodontics in the 21st century” AAO May 3rd 2016, a must hear !
I am delighted to see the highly perceptive comments by a courageous first year resident, Jean-Marc E. Choufani. This will be someone to watch out for in our profession!
At the other end of the scale, after 55 years in orthodontic practice I find myself also questioning some of the basic tenets of our profession. Does it not seem strange that with all the research and observations that have been carried out over the past decades we still have no answers to many of the fundamental questions that Orthodontists face on a daily basis? In fact, much of the research tends to bring up conflicting conclusions and leave the clinician in a quandary, hence the increasing reliance on KOLs and Orthodontic gurus to whom the task of decision making is delegated. Many fundamental questions still have no real answers after 60 years or more of enquiry, such as:-
What is the optimal force to move a tooth in the mouth?
Why does it still take 2 years to treat an extraction case well?
What is the real relationship between occlusion and TMD?
Is there an optimum position of teeth for long term stability?
Is long term unsupported stability a reality?
Can the growth of the jaws be influenced by our treatments?
Is interventional treatment worthwhile for the patient?
Is it better to extract or not extract?
Why do the the same standardised treatments carried out for different patients often provide good results?
Why do apparently similar patients respond quite unfavourably to treatment?
How much has high quality research helped to advance the day to day practice of Orthodontics vis a vis the contributions from the manufacturers?
How can Orthodontists draw really meaningful conclusions from statistical analyses involving at best a few dozens of patients when biological variation is so large, particularly in growing patients? Medical research generally includes thousands of patients to obtain real clinical significance.
Although Science does play a part in the highly complex work of the Orthodontist, there are many other factors at play, not the least of which are the artistry and sensitivity of the clinician. Without wishing to denigrate our profession in any way, in many respects it bears more resemblance to Arts and Crafts, with a strong dash of Surgery and Psychology added in the mix, than true Science.
Neville, thank you for sharing your reflections after such a long, and fruitful I can imagine, career. This is exactly why I agreed to post my thoughts. As with anything that requires thought, I believe this will take a long time to come to light, regardless, I look forward to the discussions to be had with those who decide to engage in this conversation and contribute their thoughts to those of us who enjoy a healthy dose of skepticism about whatever gets thrown at us.
I would love to be able to discuss the points you mentioned, I’ve reached out to you via facebook, if you’d agree to spare some of your time.
Thanks again for sharing your experience.