Do I still believe in evidence based orthodontics?
Do I still believe in evidence based orthodontics?
Every now and then a paper comes along that makes me stop and think about how I practice and what I have taught over the past thirty years. Professor Trisha Greenhalgh and colleagues published one of these papers in the British Medical Journal several weeks ago. I have followed her work for a while because she adopts a brilliant questioning approach to medical research. She also runs a really interesting Twitter account @trishgreenhalgh and has written a great book on interpreting research called “how to read a paper”.
They based their paper on the content and discussions of a symposium into a reappraisal of evidence based medicine. It is an open access paper, so youshould be able to access it.
Evidence based medicine: a movement in crisis
T Greenhalgh et al
British Medical Journal 2014:348:g3725
doi: http://dx.doi.org/10.1136/bmj.g3725
Over the years I have spent a large amount of time carrying out trials, agonising over the details of producing systematic reviews and speaking about evidence based orthodontics. I have really done my best to help develop my orthodontic research to a higher level. As a result, when I saw the title of this paper on Twitter, I was concerned that all our efforts had been misdirected. However, when I read the paper, I was reassured that we were heading in the right direction. Nevertheless, some of the issues that they raised were very relevant to evidence based orthodontics and I would like to discuss them in this post.
Firstly, not all of the paper, particularly the section on industry sponsored trials, is relevant to orthodontics. This is because the orthodontic industry does not bother carrying out trials, they just release the product, advertise the new paradigm and we all buy it! (see this post)
I also read this paper with the knowledge that medicine is light years ahead of orthodontics in trials, evidence and guidelines and the authors spent some time considering these areas. However, other sections of the paper are relevant to the current status of evidence based orthodontics. I shall highlight some of these and make suggestions on what you can do to ensure that we strike the right balance between evidence based orthodontics and the characteristics and values of our individual patients.
The application of evidence based orthodontics
One of their central arguments is that “real EBM has the care of the individual patients as a top priority”. As a result, we should synthesise research evidence that is relevant to the individual patient and explain this to them in terms that they can readily understand. In other words, we should not be so rigid that we simply apply the results of the most recent relevant systematic review to their condition. For example, we have good evidence from trials and systematic reviews on the most effective methods and optimum timing for the treatment of a child with prominent teeth. Nevertheless, we should not simply treat every child with prominent teeth with a functional appliance. We must go back to basic diagnosis and identify the aetiology of the problem. Furthermore, it would be wrong to suggest treatment for all these patients in order to reduce the chances of incisal trauma, without a discussion with our patients of the relative risk and numbers needed to treat.
Another area of our work that is relevant to this discussion is the renaissance of screening children at a young age to detect and intercept developing malocclusion. While these recommendations are laudable, we currently have no strong evidence on the effectiveness of “interceptive” treatment or even how many children we need to screen to intercept one orthodontic problem. As a result, we need to appraise the evidence for this new movement.
You should also be more critical of guidelines and their sources of evidence and not necessarily take them at face value. For example the Royal College of Surgeons guideline on the management of impacted canines still recommends that we should remove a primary canine to encourage eruption of the permanent canine. But it also state in the same section that ” Further well-reported randomised controlled trials are required to assess the full effectiveness of this clinical intervention”. This means we do not know if this intervention is effective, so why do we include it in a guideline?
Training to ask the questions of research.
The authors of the paper point out that this is essential. I suggest that we all need to learn to interpret research. I cannot help feeling that we are all too concerned about data presented as means of only a few millimetres or degrees. Consequently, we do not consider the effect size and confidence intervals. Unlike most high quality medical journals, I am not sure if any of our journals make the reporting of confidence intervals a requirement of publication. As a result, we tend to take the numbers at face value without an assessment of the small “differences” that excite only us along with an evaluation of the level of uncertainty. We simply cannot practice evidence based care if we do not take these into account.
Interestingly, they also emphasise that we need to pay more attention to the individual patient and we can learn from case discussions. One way for us to do this is by asking journal editors to publish discussions on “real world” cases. This would take us away from marveling at the perfectly treated “collections of precious things” in our journals that simply signify that the operator was brilliant or simply got lucky!
Publishing
I think that we all should ask the publishers to raise the bar and raise their standards. I wonder if the journals are still too full of retrospective studies and cephalometric tables that result in confusing low quality evidence that fails to answer the “so what” question. It would be a major step if we could persuade the journal editors to only publish high quality trials and systematic reviews in a form that was understandable to both our patients and us. But if we took this step, would we have too few papers to read and a few less journals? But would this be a bad thing? We are interested in a small, but important, part of the human body, but we have many journals reporting studies of mixed quality and doubtful relevance. This simply adds to the confusion of evidence based orthodontics.
Get the researchers to be more ambitious and imaginative
Their final point was that more imaginative research is needed. This was a great section of the paper. Essentially, we need to move away from our values, which mean very little to anyone but orthodontists, towards outcomes that are relevant to patients. For example, we are wide open to the use of qualitative research techniques which we have always ignored because you cannot interview plaster models and radiographs. I can guarantee that if a young academic is starting their research career and they concentrated on introducing qualitative methods into their research, they would achieve a high profile and make a difference to patient care very quickly.
Summary
In summary, borrowing very heavily from the paper. I still have a great belief in evidence based orthodontics, but at this stage of our development (miles behind medicine), I think that we need to do the following.
- Interpret our limited high quality literature carefully to avoid treating our patients as a set of teeth characterised by features that we place into categories and treat accordingly.
- Learn to interpret the literature and continuously question the recommendations of both Professors and industry sponsored advocates spouting “evidence”.
- Encourage our journals to raise their quality and not simply concentrate on filling up the space.
- Be prepared to ask our researchers to be more imaginative, and no matter who they are they should be able to answer the “so what” question (me included).
Have a look at the paper and join in any discussion below.
Emeritus Professor of Orthodontics, University of Manchester, UK.
I read this paper sometime ago when it was first published – what I also ‘took home’ was not to discount well constructed retrospective, serial etc studies, as although the evidence may not be ‘ideal’ it should not be dismissed, neither should clinical experience.
After all the best we can do for our patients is to interpret the evidence, and apply both it and our clinical experience to the individual
Thanks Ross, yes I agree we should not discount well conducted retrospective studies, if this is all the evidence that we have got. However, the important issue is that many of the retrospective studies that are currently published are not well conducted.
In effect, what we need to do is make an assessment of the quality of the study, it may be a trial or retrospective study and then decide how much uncertainty is reduced by the findings, if any?
Thanks again for the comment
My particular bugbear are laboratory studies, particularly in regard to demineralisation around brackets, but also those attempting to measure debond rates and friction. They tell us very little (if anything) about what is actually going on in the mouth. I believe that there is no longer any justification for publishing the results of these studies in an academic orthodontic journal, but unfortunately journal editors often do not agree with me!
Studies involving qualitative research designs are excellent for understand the patient’s perspective and for generating theories, particularly concerning patterns of behaviour. This has mostly been ignored in orthodontic research in the past, which is essentially atheoretical.
I think that useful data could also be obtained from large, prospective, cohort studies. A number of longitudinal studies have been undertaken in the UK, following children from birth into adulthood. If we could get involved, even as a small part, in one of these studies, I think we could address a number of research questions that might be difficult to answer using a RCT design.
Matthew 7:3
King James Version (KJV)
And why beholdest thou the mote that is in thy brother’s eye, but considerest not the beam that is in thine own eye?
Dear Kevin,
I found this tucked away in a corner of my computer. I can’t remember whether or not I sent this to you. It may have been intended for someone else named Kevin. In any event….
In Boston Society, it is said that the Lowells speak only to the Cabots, the Cabots speak only to the Lodges and the Lodges speak only to God. So it is with the orthodontic RCT folks—they talk only to and about one another. People think that I am a foe of the RCT. Nothing could be further from the truth. I am not a Luddite; I know that the RCT is the gold standard. Unfortunately, the most important orthodontic RCTs are at best a considerably adulterated gold standard. Moreover, the problems are not theoretical. They are real and obvious.
The participants at all levels are aware that they are participating in a trial. The patients are worth thousands each and thus are apprised of the seriousness of their role in the trial. The practitioners know that their work will be scrutinized, measured, and reported. Blinding? That’s for sugar pills. Lest the World be unaware of the putative significance of the ongoing process, a bit of data-peeking here and there whets the specialty’s appetite. It is the Hawthorne Effect writ large. O.K., perhaps our RCTs are as good as they need to be; live and let live. Unfortunately, that’s not the way things go.
The problem as I see it is the promulgation of the idea that the RCT is the only valid source of data. What this does is ensure that the most important questions will never be resolved. In other words, it grants perpetual license to practice any way you might want, as long as the craziness is accompanied by a vow to change if and when an RCT weighs in. You can randomize slot size or bracket prescription, but what about, say, extraction versus nonextraction? Adult orthodontics versus Surgery? An ethical trial must press a full-informed potential participant for a preference. Who in his/her right mind would prefer surgery if there were true between-treatment equipoise? We flipped a coin; it came up surgery; sign here. Really? Sheldon Baumrind came up with an interesting way of randomizing the un-randomizable. I’m not sure his approach was ever tried or whether or not it would pass muster with an IRB in 2015. There is a way, however, to approach the problem statistically—confounder summarization with discriminant analysis (Miettinen).
Given patients treated some time ago, discriminant analysis can be used to describe sub-samples that could have been treated either way. The purpose is not to explore a “borderline,” but rather to define subsamples that were similar at the time of treatment. These patients can be recalled long after treatment to estimate the between-treatments differences. No Hawthorne effect, given that nobody knew that they would ever be in a study; however, no randomization. Some sort of unknown bias could be present. Unknown—and perhaps unlikely—versus known, obvious, and probably significant observational distortions. Clearly, both approaches are a bit below the peak of the pyramid of evidence; however, I’ll bet that the orthodontic RCT is less powerful and of greatly limited scope. Given that this assertion can never be proved, I suppose it’s time to quote and invoke the words of the famous philosopher, Rodney King: “Can’t we all just get along?”
Lysle Johnston
Dr. Johnston
About 18 years ago I was nervously applying to Orthodontics programs. I asked a lot of question about programs. Unfortunately most of the information I received was misinformation. I took Michigan off my list. After my residency I practiced in a very sheltered environment …. the military. After separating from the military I began to realize that there were some big differences between what I learned and how was practicing, and what was actually going on out in the wild. After hearing you speak at a few AAO meetings I realized that the reason that some people felt uncomfortable around you was simply because you told the truth. You’re the guy that says the stuff that a lot of other people are thinking, but would never say themselves.
Unfortunately I have been extremely burnt out and depressed about the future of Orthodontics. This specialty,which I used to love, which I worked so hard to become apart of, is starting to fall apart. With so many members of our specialty bending the evidence ( when they ought to be bending archwires) around self-serving motives, and many other members participating in order to remain competitive ( A smarmy character at a meeting once told me he used Damon brackets only because his competitors were using them. “If you can’t beat them join them”). It seems like the specialty is mostly interested in automatizing every aspect of orthodontic treatment in order to be more efficient…..to see more patients and increase production. The economy has exposed some embarrassing realities about our sense of entitlement. There are members of the specialty who are practicing differently today not because it is serving the patients best interest, but rather because their production has fallen.
I’m wondering if you can give me some potential solutions. I’m looking for solutions that I could help implement. And I don’t want to polarize or divide the specialty…quite the opposite.
Or maybe I’m just imagining this and I need to shut my mouth. I’m sure there are plenty of people that are going to tell me that. I’m almost ready to give up.
Regards, J
Dear Jared don’t give up! There are others who, like you, have felt the need to find a new approach to the way we practise orthodontics. Dr Barry Raohael is one of them and he started Airway Orientated Orthodontics which now has quite a following. Then there is Dr Bill Hang who runs his Orthotropic courses and more recently Myobrace has been introduced into the U.S. Dr Derek Mahony, based in Australia but an international lecturer has interesting material on his website: http://www.fullfaceorthodontics.com.
I have many more on my blog roll on http://www.connectingheads.com so never feel that you are alone. The increasing number of patients suffering from airway related problems must be reaching the point where medicine & dentistry need to be working in an integrated way to best serve these patients. “Questioning is the first step to enlightenment” Lauren’s van de Post.
Very interesting thoughts