Clear aligners are not as good as fixed appliances? A systematic review
This new systematic review suggests that treatment with clear aligners is associated with worse treatment than fixed appliances. This is an important conclusion. But does it stand up to close scrutiny? Read on….
Clear aligner treatment is now a well-established method of delivery of orthodontic care. The aligner companies and their paid Key Opinion Leaders make many claims for its effectiveness. However, for such a popular and costly method of care, there is surprisingly little high-quality research. I was, therefore, very interested to see this new systematic review that the EJO has just published. A team based in Zurich, Athens and Finland did the review.
Spyridon N. Papageorgiou et al
EJO advanced access. doi:10.1093/ejo/cjz094
What did they ask?
They did the review to answer this question.
“Is there a difference in the treatment outcome with clear aligners compared to fixed appliances for comprehensive orthodontic treatment”?
What did they do?
They carried out an extensive and ambitious systematic review. They followed the standard methods of electronic search, screening the titles, paper selection, assessment of bias, data extraction and meta-analysis.
Importantly, they included randomised trials and non-randomised retrospective studies. I will return to this later. The PICO was
Participants: Orthodontic patients needing comprehensive treatment.
Intervention: Clear aligners
Comparator: Fixed appliances
Outcome: Occlusal index scores using the ABO discrepancy index or the Peer Assessment Rating.
They registered the protocol before the study started.
They measured the risk of bias for the RCTs with the Cochrane Risk of Bias tool. The ROBINS-I tool was used to measure the risk of bias for the non-randomised studies.
What did they find?
They found 11 studies that satisfied their inclusion criteria. These were divided into 4 randomised trials and 7 retrospective non-randomised trials.
When they looked at the risk of bias, they decided that two of the trials were at high risk of bias. This was because of problems with randomisation, deviations from protocol, missing outcome data and outcome measurements. They also reported that the non randomised studies were at a higher risk of bias. For example, five were a moderate risk, one of them serious and one of them was at critical risk of bias. However, when I looked at the supplementary data on the EJO website, I was confused because this stated that 6 of the studies were at serious risk of bias, and one was critical. These shortcomings were lack of control of confounders, selection bias, deviations from protocol, outcome measurement and selective reporting.
The authors presented a large number of meta-analyses. I do not have the space to go into these here. As a result, I shall just concentrate on the data concerning the ABO scores. They reported that treatment with aligners was associated with worse ABO scores compared to fixed appliances. The difference was 9.9 (95% CI=3.6-16.2). This was clinically significant.
This meta-analysis reported on a total of 148 patients who had been treated with aligners and 148 fixed appliance treatments. The heterogeneity was 84% which is high. This data was derived from two non-randomised trials that were at serious risk of bias and one RCT with some concerns.
Their overall conclusion was:
“Orthodontic treatment with clear aligners is associated with worse treatment outcomes compared to fixed appliances in adult patients. Current evidence does not support the clinical use of aligners as a treatment modality that is equally effective to the gold standard of braces”.
What did I think?
Firstly, I thought that these conclusions were very bold. I feel that it is essential that when findings of this nature are made that the paper should be robust. I am afraid that I think that this is not the case for this paper. I have the following problems.
On first reading, it appears to be useful. However, the analyses include non-randomised retrospective studies. We must assume that most retrospective studies are subject to selection bias. As a result, when authors include retrospective studies in a systematic review, then its conclusions must be weakened. This is why Cochrane Reviews do not usually include this type of study.
I also thought that the reporting in the paper was not consistent with the data that they provided. All the non-randomised retrospective studies were at serious or critical risk of bias. Again, this reduces the strength of the evidence.
Finally, I had a close look at the papers that they included in the meta-analysis on the ABO scores. They based this meta-analysis on one trial and two retrospective studies. The International Journal of Clinical and Experimental Medicine published the trial. Importantly, the e-century publishing corporation publishes this journal and this is on Beall’s List of predatory journals. Furthermore, the Journal has retracted the paper because of plagiarism. The AJO published one of the NRS studies. The final study was an unpublished Masters thesis from the University of Montreal. These are not high levels of evidence.
Unfortunately, I cannot help feeling that the papers that they included in their paper cannot support the conclusion they published. I would like to point out that this is simply my academic opinion.
Emeritus Professor of Orthodontics, University of Manchester, UK.
Really? Why not just take out a full page ad stating that the authors dislike very much aligners and save more than just precious resources?
Thank-you Kevin for pointing out some of the flaws of this kind of study. It is indicative of the bias and perhaps hatred that exists when faced with this disruptive technology – not discounting myself. There are additional limitations that make the methodology and this conclusion – containing an unfounded clinical message no less – “bold” as you politely worded it. The most striking to me is the ambitious, broad question, to be addressed with poor quality data and tools that are not valid for corroborating the conclusion. Assessment of treatment outcome is an immense undertaking and whilst the ABO and PAR are some of the most objective and accepted tools we have to measure ALIGNMENT AND OCCLUSION, they have their own scientific limitations and do not come close to measuring and quantifying treatment outcome. Any Occlusal Index alone does not provide the type nor strength of evidence to support the final clinical message. Change of semantics could have saved much of this report.
The “bold” conclusion is rife with bias and inaccuracy:
“Orthodontic treatment with clear aligners is associated with worse (an inappropriate term to use in an objective “scientific” conclusion) treatment outcomes (where is the assessment of treatment time, soft tissue change, iatrogenic damage, patient satisfaction to name but a few variables that combine to provide treatment outcome) compared to fixed appliances in adult patients. Current evidence (sorry but most published literature is not current when measuring comprehensive case outcome -1.5-2 years of clinical treatment followed by data collection and analysis prior to submission- with a rapidly evolving digital tool) does not support the clinical use of aligners as a treatment modality that is equally effective to the gold standard of braces”).
…..Well hate to point it out, but your study does not support your conclusion.
Last point and thanks for indulging my bias, is that many of the “outcome” studies are performed in university clinics and / or mentored by clinicians unfamiliar with the technique being assessed, be it fixed appliances or computer programmed aligners (Thanks to Dr Baumrind for pointing out this important design flaw many moons ago). One such paper concluded that aligners were poor at correcting A-P discrepancy – but clinicians never ran any A-P mechanics in association with aligners. That’s like placing brackets and wires only (no elastics, no other mechanics) and expecting comprehensive Class II correction! Computer programmed aligner mechanics overcome many of the negative physical effects we need to control with fixed appliances, but they are not automatic.
Are we to dismiss novel mechanical potential due to increasing corporate involvement in our profession? If so, every clinician’s prerogative; let’s do it accurately. In a similar vein to 2 posts ago, we need these investigations, but keep it tight – or it becomes just as biased and incredible as much of the aligner marketing we are all faced with.
“…with worse treatment outcome compared to fixed appliances in adult patients.”
I think that VV has put forth an eloquent critique to this paper. However, even if the conclusions drawn by the authors were valid, what recommendations are there for the clinician if any? Only offer fixed therapy? Perhaps but looking at this from the perspective of improving adult oral health and we all agree that reducing malocclusion improves oral health, acceptance of treatment is the vital first step. I am acutely aware (especially as a former adult patient) that prior to the development of sophisticated clear aligner therapy adult acceptance of orthodontic treatment was very low. Not to be snarky but without acceptance of treatment we have 0% improvement in oral health.
Some may say that we should only offer the “best” treatment but this is in conflict with medical treatment at least in the United States. For example, orthopedic surgeons offer different treatment options for degenerative knee conditions or vascular surgeons offer different treatment options for lower leg circulatory problems. They usually quote percentage improvement with each proposed treatment option all meeting current standards of care. I think it’s reasonable to say that you can offer the very best with fixed therapy but also have a reasonable alternative.
I understand that there is much frustration with the introduction of clear aligner therapy but it is not going away. In fact, it will only get better. How so? Digital tele-dentistry is a potential smart system with continuous feedback from the clinician to the respective company. As each case progresses and scans are sent back to the respective company either for more aligners or retainers, precise comparisons can be made with planned and actual results. I think that it is a fair assumption that this leads to an improved product. Is there this continual feedback with any of the fixed systems?
Lastly, the past few months I am finding more and more experienced non-orthodontic (zero cases) restorative dentists and periodontists recommending aligner orthodontic consultations for their own families as well as their patients….very powerful. I think that we have entered an era of what I term “aligner first mentality” at least in my little corner of the world.
Whoa whoa whoa…”we all agree that improving a malocclusion improves oral health” ??? I need some evidence of this.
As a disease or disorder, malocclusion is a dento-facial anomaly. I think a fair logical argument is that reducing malocclusion is therefore beneficial. I grant you that some would argue that malocclusion is only a cosmetic problem. Thus, implying that it is not significant and not that important. In fact, this is one of the arguments presented by the high-powered national law firm that is representing Smile Direct Club in front of United States Federal District Court Judge R. David Proctor, Northern District of Alabama in a lawsuit against the Dental Board of Alabama. Incidentally this lawsuit can have national implications in the USA since it is in a federal court. I stand by my argument and not give an inch.
Unpublished thesis ? Plagiarism?Is this the new standard of “faulty “ research we have to be keeping a watchful eye out for ?
Thank God for your blog and abilities.I am not well educated in this area and rely on folks like Dr.O,Brien.I am starting to lose faith in a deal of university research .especially when quoted and referred to many years after the study publication date.
From a purely practical and anecdotal point of view ;my day to day practice would be non sustainable if one accepted this analysis,s results.Pts./parents only pay for results that are as advertised !This is a major factor ,whatever a study and or an analysis states.
Has this been dismissed as a relevant piece of data in academia ??
As a parallel thread to the discussion, regardless of the level of evidence, it seems like measuring treatment outcome using ABO or PAR scores have very little clinical relevance.
At a time when the world is moving toward value-based care and patient centred outcomes, orthodontic studies continue to use arbritrary, orthodontist-defined metrics to determine the quality of orthodontic care, when there is little evidence linking such scores to either dental health outcomes, treatment stability, or patient satisfaction.
https://www.ncbi.nlm.nih.gov/pubmed/1628684
How can the opinion of 74 orthodontists in 1992 be the basis of judging the quality of outcomes for millions of orthodontic cases, treated with appliances that did not even exist when the PAR index was defined?
Dear Prof. O’Brien, thanks for your interest and recent blog (1) on our study (2).
Regarding the inclusion of non-randomized studies, we would like to point out that we had already included in our original publication a sensitivity analysis including only randomized trials that showed consistent results with the original analysis (Supplementary Table 9).
We have to disagree with the notion that non-randomized studies are de facto excluded from systematic reviews. Their potential value in evidence-based data synthesis has been acknowledged and specific approaches have been developed (3,4). What we did in our systematic review was on par with the guidance from the newly-updated Cochrane Handbook (5) and we followed specific Cochrane guidance on combining randomized and non-randomized studies with GRADE (6).
We do very much welcome your pointing out that the study from Liu et al. (7) had been retracted, which had evaded our notice. We searched in PubMed and extracted hits in Endnote, which also found the fulltext .pdf, so we didn’t land on the paper’s own PubMed entry and weren’t able to see it was retracted. We would like however to point out that this study has been retracted from plagiarism and not for data falsification, and therefore its impact on our conclusions is not straightforward. We have now nevertheless added another sensitivity analysis that excludes the randomized trial of Liu et al. (7) to see how this would impact our data. The updated Tables and Figure from the paper can be accessed openly from everyone interested from here (http://bit.ly/2Leb9f6) and have also been submitted in the European Journal of Orthodontics as correction.
It seems that excluding the rather conservative trial of Liu et al. (7) paints an even more bleak image for aligners. The difference in occlusal outcome with the ABO tool has moved from 9.9 points to 13.4 points, while the absolute risk increase for patients with unacceptable finishing quality (ABO-OGS score>30 pts) had moved from 21.7% more to 30.2% more. Therefore, excluding this retracting paper still doesn’t make any big difference to our conclusions. This has also reduced the heterogeneity from 84% to 0% (though, as explained heterogeneity was not a problem even at the start).
Finally, regarding the comment about the inclusion of the unpublished dissertation of Robitaille (8) we would like to point out that again we have followed the formal guidelines of the Cochrane Handbook in including grey literature: ”Review authors should generally search sources such as dissertations and conference abstracts” (9)
Many thanks again on behalf of the research team for your interest in our study and for the opportunity to clear these issues.
Best regards,
Spyros
References
1. https://kevinobrienorthoblog.com/clear-aligners-v-fixed-appliances-a-systematic-review/?fbclid=IwAR0VC92Rx_qNPGTVnqtb6e8HxQbVEpIG03hWz0UW4RcUnhsFS3pfMtlIlrw. Last accessed December 2, 2019.
2. Papageorgiou SN, Koletsi D, Iliadi A, et al. Treatment outcome with orthodontic aligners and fixed appliances: a systematic review with meta-analyses. Eur J Orthod. 2019 Nov 23. pii: cjz094. doi: 10.1093/ejo/cjz094. [Epub ahead of print].
3. Cameron C, Fireman B, Hutton B, et al. Network meta-analysis incorporating randomized controlled trials and non-randomized comparative cohort studies for assessing the safety and effectiveness of medical treatments: challenges and opportunities. Syst Rev. 2015 Nov 5;4:147.
4. Efthimiou O, Mavridis D, Debray TP, et al. Combining randomized and non-randomized evidence in network meta-analysis. Stat Med. 2017 Apr 15;36(8):1210-1226.
5. Reeves BC, Deeks JJ Higgins JPT, et al. Chapter 24: Including non-randomized studies on intervention effects. In: Higgins JPT, Thomas J, Chandler J, et al (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019). Cochrane, 2019. Available from http://www.training.cochrane.org/handbook.
6. Schünemann HJ, Cuello C, Akl EA, et al. GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence. J Clin Epidemiol. 2019;111:105-114.
6. Schünemann HJ, Cuello C, Akl EA, et al. GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence. J Clin Epidemiol. 2019;111:105-114.
7. Li W, Wang S, Zhang Y. The effectiveness of the Invisalign appliance in extraction cases using the the ABO model grading system: a multicenter randomized controlled trial. Int J Clin Exp Med. 2015;8(5):8276-82.
8. Robitaille P. Traitement combiné d’orthodontie et de chirurgie orthognatique avec Invisalign® : revue de la durée de traitement et des résultats obtenus. MSc Thesis University of Montreal, 2016.
9. Lefebvre C, Glanville J, Briscoe S, et al. Chapter 4: Searching for and selecting studies. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019). Cochrane, 2019. Available from http://www.training.cochrane.org/handbook.
Apologies for the wrong link to the correction—here is the correct one: http://bit.ly/2qdZDZZ.
The Real Person!
The Real Person!
Hi and thanks for the comments. I am afraid that I am going to have to disagree. When we consider the inclusion of NRS in review, this is the Cochrane advice.
“The decision of whether non-randomized studies (and what type) will be included is decided alongside the formulation of the review PICO. The main drivers that may lead to the inclusion of non-randomized studies include: (i) when randomized trials are unable to address the effects of the intervention on harm and long-term outcomes or in specific populations or settings; or (ii) for interventions that cannot be randomized (e.g. policy change introduced in a single or small number of jurisdictions) (see Chapter 24).”
Neither of these are relevant to your review.
When I think about the plagiarised paper that you included, whlle the paper was withdrawn for plagiarism, you cannot simply state that all is fine because the data can be treated differently. You cannot include any of this information in your review. I have also looked at your new meta analysis, this includes data from two non randomised trials that you classified as being of serious risk of bias. They also are very small and report data from a total of 143 patients. I have looked at the unpublished thesis and this is confined to orthognathic patients. I am not sure that this is relevant to the review and again it must introduce further uncertainty into the review.
As a result, I think that we can only conclude that this review is not sufficiently strong to come up with the very robust conclusions that you have made. I am sorry to be so critical, but it is important to justify robust conclusions from our literature.
Whoa whoa whoa…”we all agree that improving a malocclusion improves oral health” ??? I need some evidence of this.
I think we need to clear up these ” improving maloccn improves oral health” and “it’s all cosmetic” issues since we don’t, as a profession, appear to be able to agree on this, most basic, of reasons why we do stuff. As far as I can tell from reading the literature etc the position is as follows:
1: malocclusion is not, in itself, a disorder, it is in fact the norm. Almost everyone has some deviation from what we classify as a “perfect” occlusion. The idea of a malocclusion that needs correcting simply by being a, so-called, malocclusion is incorrect.
2: There is no relationship between what we call malocclusion and caries, perio disease or TMD. Some patients, few in number, can have true damaging occlusions (the most common being an anterior cross-bite) or functional problems (such as anterior open bite). The majority of patients treated have no disease or functional issues and cannot be predicted to develop any.
3: This means that most orthodontics is carried out to improve the appearance and can be classed as cosmetic by definition.
4: This does not mean that it is not important since presumably it is to the patient concerned. Neither does it imply that it can be carried out with impunity by anyone, trained or untrained. The reason for the treatment is not connected to its simplicity or complexity. Many cosmetic surgical procedures are highly complex and are not carried out by untrained professionals. If teeth are to be moved around then this is an intervention with corresponding side-effects that must be understood and mitigated against. Ignoring such side effects will inevitably result in damage and/or wasted time and money.
Very well and eruditely said.Thanks.
If there is evidence of better outcomes why are we not seeing it? Why are the “good aligner doctors” not publishing their data in quality refereed scientific journals? Why has every one of the 6 reviews on aligner published in a variety of “respected” journals since 2017 reached conclusions widely at variance with corporate claims? Those other reviews did not derive “robust” conclusions, it is true, and it is almost expected that reviews will publish muted conclusions. I am not taking issue here with the current review in question or Kevin O’Brien’s blog about the article. I merely am asking where is the evidence for “good” treatment outcomes. We hear a lot about it but the lack of anything other than anecdote is disturbing. And then it too will not be “current” for reasons VV outlined. Research has flaws, personalities, interpretations. I know this is a blog regarding a specific “robust” conclusion and how it was reached. However, for me the deeper question is what the actual truth is. At some point we cannot keep moving the goal posts to avoid accepting evidence and accepting claims of treatment simply because the appliance evolves so rapidly. There are many things that have NOT evolved even in Align – ie things not affected by attachments or tooth movement protocols. I would like to see evidence of excellence, even if it is not excellence of evidence. At the moment we seem to have evidence of non-excellence and non-excellent evidence