Evidence based orthodontic treatment-no good deed goes unpunished. Part 2: Let them eat cake
Let Them Eat Cake
This is the second part of our joint lecture at the recent British Orthodontic Conference. Lysle Johnston has based this post on his presentation. This was about the value of alternatives to the orthodontic randomised controlled trial. It is the first guest post on my blog.
In Boston Society, it is said that the Lowells speak only to the Cabots and the Cabots speak only to God. From my standpoint, this is an elitist ethos shared by the RCT folks, who seem to speak only to and about each other and their RCTs. If the masses want evidence, let them wait for an RCT! (Qu’ils mangent de la brioche). As someone who has toiled for over 50 years in the vineyards of orthodontic research, I feel somewhat insulted to find that my research generally is considered too far down the pyramid of truth to be considered or even discussed.
Do not misinterpret this lament—I agree that systematic reviews are at the top; however, as I see it, there is a problem in the orthodontic application of the RCT. Orthodontic treatment options aren’t just pills of different colors. To randomize treatments, there not only has to be true equipoise, but also, and perhaps more problematic, the investigators must seek to elicit a preference from each fully-informed participant. Does the following scenario seem realistic?
“Mrs. Jones, we think you can be treated equally well either by adult orthodontics or by surgery. We are truly uncertain and would like you to participate in a study designed to compare the two options. We flipped a coin, and it came up surgery. Sign here”.
The orthodontic treatment options to which a subject might reasonably be indifferent are limited and may be of minimal interest to the topic of evidence-based orthodontic practice. This problem consigns many of our most difficult questions to the dust bin of the un-answerable. The RCT-or-nothing attitude has the unwanted (or ignored) side effect of granting license to any treatment, no matter how lunatic, until a never-to-be-executed 
RCT signs in. The Cowardly Lion became brave when given a medal; apparently, the orthodontic researcher becomes truly “medical” when he or she has conducted an RCT. Alas, the cognoscenti may be good at discerning the mote in the eye of a lesser investigator, but quite willing to ignore the log in their own (Matthew 7:3). Hawthorne effect? Data peeking? Lack of blinding? No long term data? Alas, it is probable that both the question and the researchers will be dead when the time comes to look at the surprisingly small percentage of subject who might be willing to return for records. (In my experience with recalls, the cooperation rate is about 1 in 10). The important thing seems to be the secret handshake.
My message is that there are lesser, more timely strategies, many of which may be quite good at minimizing the various biases that proliferate unchecked in today’s orthodontic marketplace.
In the American South, there is an old saying: “Never try to teach a pig to sing; it wastes your time and annoys the pig.” My visit to the British Conference at Brighton left me with the impression that the rank and file may not be annoyed, but clearly they have a limited interest in finding out whether popular, salable treatments get the RCT seal of approval. As long as everything works and nobody dies, proof may be beside the point.
Emeritus Professor of Orthodontics, University of Manchester, UK.
It was great to read the thoughts on orthodontic research from both Drs O’ Brien and Dr Johnston.
I would like to say that I agree with most of the comments, if not all, Dr. Johnston is making and that I would like to stress out that all research is valuable and should be considered1.
I think that the key issue is to be able to understand the applications and limitations of each study design so that we conduct, report and interpret our research correctly.
I think that the problem with orthodontics and dentistry is that sometimes comments like the ones posted by Dr Johnston are misinterpreted as there is no need for RCTs leading to complacency, to no effort to improve our research methodology and to recycling of the old problems. For example, it is common in orthodontics that non-randomised studies are analysed using simple analyses without accounting for any confounders and this is clearly a problem.2
Most of the issues that are integral to an RCT design and reporting are also applicable to non-randomised studies and considering them can only help no matter what study design we are using. Obviously, there are many published non-randomised studies that are excellent and many RCTs which are either no RCTs and/or are of low quality.3, 4
Bias is not something we can measure and knowing, thinking and doing something about it at the design and/or analysis stage increases the chance of reducing it and increases the chance of producing a better and more useful study regardless of design.
If the ultimate aim is for the existing evidence to be combined in a systematic review knowing how to sort out the different issues inherent in each study is more important than the nomenclature. Cochrane also uses a risk of bias tool for non-randomised studies 5 and GRADE when assessing the quality of the evidence from systematic reviews considers also non-randomised studies. 6
References
1. Pandis N, Tu Y-K, Fleming PS, Polychronopoulou A. Randomized and nonrandomized studies: complementary or competing? Am. J. Orthod. Dentofac. Orthop. 2014;146(5):633–40.
2. Spanou A, Koletsi D, Fleming PS, Polychronopoulou A, Pandis N. Statistical analysis in orthodontic journals: are we ignoring confounding? Eur. J. Orthod. 2015.
3. Pandis N, Polychronopoulou A, Madianos P, Makou M, Eliades T. Reporting of research quality characteristics of studies published in 6 major clinical dental specialty journals. J. Evid.-Based Dent. Pract. 2011;11(2):75–83.
4. Koletsi D, Pandis N, Polychronopoulou A, Eliades T. What’s in a title? An assessment of whether randomized controlled trial in a title means that it is one. Am. J. Orthod. Dentofac. Orthop. 2012;141(6):679–85.
5. Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016;355:i4919.
6. Balshem H, Helfand M, Schünemann HJ, et al. GRADE guidelines: 3. Rating the quality of evidence. J. Clin. Epidemiol. 2011;64(4):401–6.
Lysle exercised his most acute, earnest and articulate style in this essay that brings our attention to his common-sense observations about dental research generally and orthodontic research specifically.It always refreshes me to hear Lysle whether in print or personally. Thanks, Kevin for including this one.
Kevin:
It’s a smart move to include Dr Johnston’s viewpoint. He won’t remember the conversation I had with him at the French Orthodontic Meeting about 20 years ago, but his last remark “As long as everything works and nobody dies, proof may be beside the point” is salient. How could “orthodontic treatment” possibly lead to the death of a patient? In a recent study on premolar extractions for orthodontic reasons, Larsen et al. (1) showed that there was no difference in OSA risk for the extraction and non-extraction groups. But the prevalence of OSA in both groups appeared to be some 2-10 times as high as the general pediatric population (2). My fear is that there may be a ‘life-threatening’ issue that needs to be brought to the attention of our orthodontic colleagues.
References
1. Larsen AJ, Rindal DB, Hatch JP, Kane S, Asche SE, Carvalho C,
Rugh J. Evidence Supports No Relationship between Obstructive Sleep Apnea and Premolar Extraction: An Electronic Health Records Review. J Clin Sleep Med. 2015;11(12):1443-8.
2. Lumeng JC, Chervin RD. Epidemiology of pediatric obstructive sleep apnea. Proc Am Thorac Soc. 2008;5(2):242-52.
Dr. Singh
As you’d indicated, Dr. Johnston might not recall saying to you, “As long as everything works and nobody dies, proof may be beside the point”; I’m curious, do you recall what your question was to which he was responding, and if so, will you please share it on this forum?
Hello professor Johnston and O´Brien.
As a long time follower of this blog (albeit never yet commenting) I wish to congratulate you on the great favour you do to the orthodontic speciality with this blog.
I believe scientific articles are essential for our improvement as clinicians, but I also believe the discussion posts like the ones on this blog are of great value for us.
Thank you for your selfless dedication and contribution.
By the way, I heard you might be coming to Brazil next year. If it is true, I am looking forward to it.
My best regards,
Alexandre Veiga Jardim