Are commercially marketed orthodontic products tested before promotion and release?
Most orthodontic developments are commercially driven. This includes self-ligated brackets, methods to speed up tooth movement, temporary anchorage devices and clear aligners. While this evolution may have improved our speciality. Unfortunately, it seems that products are released onto the market without being adequately tested in trials. Importantly, when these trials are done, they reveal that many of the claims made for the treatment are false. The best examples of this are self-ligating brackets, class II correctors and orthodontic vibrators. However, we do not know how many products are promoted without testing. This new paper provides us with this information, but it does not make comfortable reading.
A team from London, UK, did this study. Progress in Orthodontics published the paper. It is open access, so you can read it yourself without being a member of an orthodontic society.
Almaha Alhussain, Martyn T. Cobourne,Nikolaos Pandis and Jadbinder Seehra
Progress in Orthodontics: Advance access online. https://doi.org/10.1186/s40510-023-00487-y
I want to declare a conflict as I know most of the members of this team well, and I have worked with them in the past.
What did they ask?
They did the study to answer this question.
“What is the prevalence of clinical trials in orthodontics evaluating commercially marketed and non-marketed products”?
“Is there a presence of associations between the direction of the results of these trials in declaration of conflicts of interest and industry sponsorship”?
What did they do?
They did a meta-epidemiological study of orthodontic RCTs from 1st January 2017 to 31st December 2017.
I did not know what a meta-epidemiological study was, so I looked it up. I found this definition.
“Meta-epidemiological studies adopt a systematic review or meta-analysis approach to examine the impact of certain characteristics of clinical studies on the observed effect. The unit of analysis is the study not the patient”.
In other words, this looks at the characteristics of the studies to see if they influence the results and conclusions. This seems sensible to me.
They followed standard systematic review methods of electronic data search for papers, filtering the publications, and data extraction. They extracted a large amount of data from the papers. I thought that, of these, the most important were a declaration of conflicts of interest, source of funding, type of intervention, and direction of the intervention effect.
What did they find?
They identified 196 RCTs. The highest number of trials were published in the Angle Orthodontist (18.4%), AJO-DDO (14.8%), and EJO (13.3%).
- 65.3% of the trials assessed marketed products after their introduction.
- 44.9% found a positive effect of the intervention
- 47.4% found no difference.
- In 66.3%, no conflict of interest was declared. However, in 26%, this was not clear.
- For RCTs assessing marketed products, 45.3% reported a positive effect, and 47% reported no effect or equivalence between interventions.
- For marketed products, there was no association between industry funding and direction of effect.
- There was no association between declared conflict of interest and treatment effect.
What did I think?
This paper looked at a meaningful and relevant subject. The team used the correct methodology to answer the study question. The article was written clearly and was easy to understand.
The authors described a large amount of data; it is worth getting a copy of the paper and looking at it closely. I learned a lot from this study and looked at how the findings impact our knowledge of new innovations.
Importantly, it appears that most published trials assessed the effect of the innovation after its introduction onto the market. This emphasises that orthodontic companies do not carry out research into their products. They and their KOLs then make unfounded claims for their effects. I have written about these many times before. However, this is so important that it is worth repeating. The bottom line here is that we should not believe any of the claims that are made in the absence of research. This paper nicely illustrates the extent of this problem.
Secondly, when studies were done, while it was good to see that 44% found a positive effect of the intervention, it was disappointing that in 47% of trials, there was no evidence of an intervention. When I think back, these were likely to be the trials of self-ligation and some methods to speed up treatment. Again, this represents the unethical approach taken by some companies.
The study team also pointed out that researchers should be involved in the development process if we want to reduce research waste and ensure that products are correctly assessed.
We also need to consider the author’s conflict of interest declaration. This is a crucial step in the publication process. Again, it was disappointing that apparent conflicts were not declared in 26% of studies. I found this surprising because I thought that as part of the submission process of a paper, it was essential for authors to declare conflicts. As a result, I cannot understand why the journals are not insisting on this step.
The good news is that the direction of effect was not related to whether the trial was industry-funded. This means that those companies that funded trials did not influence the results. This represents a good ethical collaboration between industry and researchers.
Final thoughts.
This paper got me thinking, and I was disappointed to find a limited relationship between the orthodontic industry and clinical researchers. The companies appear to ignore the important step of researching their products. Instead, they introduce them onto the market with the help of their KOLs. This is a sorry state.
Emeritus Professor of Orthodontics, University of Manchester, UK.
Thanks for sharing this interesting meta-regression. Almost 50% of the products/approaches tested did not generate meaningful differences. As always, we must remember that the external validity of study results could be quite limited.
The fundamental problem with declaring a conflict of interest is that there is no simple way to determine whether the claim is correct. It would be better to declare when in doubt, but I think sometimes it may not be declared when in doubt. Even if no conflict is declared, there may actually be a conflict.
A previous meta-regression tested the impact of sponsorship bias in dental RCTs (Saltaji H, Armijo-Olivo S, da Costa BR, Major P, Cummings G, Amin M, Flores-Mir C. Influence of Sponsorship Bias on Treatment Effect Size Estimates in Randomized Trials of Oral Health Interventions: A Meta-epidemiological Study. J Evid-based Dent Pract 2021; 21: 101544. Doi: 10.1016/j.jebdp.2021.101544.). The treatment effect size estimates were 0.10 larger in trials with higher or unclear risk of sponsorship bias. So, an effect will likely exist, but the impact is not necessarily as large as we may think. An increase in effect size of 0.10 may or may not be clinically relevant.
Thank you for sharing this and for exposing so many other truths in our beloved specialty.
Back when all the fake claims of advantages of self ligation began to appear, I and many of my colleagues would refer to these marketing pitches as “snake oil” orthodontics. It has long been unsettling that so many certified orthodontic specialist (and programs) promoted and took advantage of the marketing claims that many twin bracket users knew were so false.
Was it that for many, offering anything “new” might result in a competitive advantage? The unspoken cost of capitalism?
And, the false beliefs have been so indoctrinated, they remain to this day.
Good call, Kevin:
In the US clinical, including orthodontic, products are classified according to the perceived level of risk, according to the FDA. If I’m not mistaken, most orthodontic products come under Class I (low risk) and do not require pre-market clearance (Class II). Perhaps others with more experience with ISO could add some further details, regarding the protocols in Europe, etc.?
This article and blog post appear to have perfectly summarized an important component of what is responsible for much of the degradation in orthodontic information. What is now needed is a solution.
When we have marketing and sales of an orthodontic product before research to support the claims, we are placing the cart before the horse. Just as when a cart is placed before a horse, the inevitable result is traveling in misguided and uncontrolled directions, wasted energy, and even crashes. For healthcare, the analogy holds even to the point that a crash can result in bodily harm to our patients.
So how do we resolve this issue and stop this vicious cycle in the degradation of orthodontic information, marketing, and sales of a product before evidence, cart before the horse?
It would seem that there are three components necessary for this system to persist as it has been. The first is the companies to develop, market, and sell products, without first having valid research to justify their claims. The second is KOLs to aid in the marketing part, pushing sales of products that do not have research to justify their claims. The third is us, orthodontists, to purchase these products and use them on our patients.
The Companies:
It would seem that asking the companies to change would be an exercise in futility. Yes, there are orthodontic companies that have maintained nearly 100% of their integrity. However, there are also those who have made literally hundreds of millions of dollars off of claims now known and proven to be false. These latter companies, seem unlikely to place any walls between what has worked in the past and their cash flow. They have acquired a taste for the riches unproven claims can provide.
The KOLs:
It seems unlikely that the KOLs pushing these unproven claims will stop by themselves either. Yes, there are many KOLs out there who only speak for products proven to do what they claim, only work for reputable companies, and are quite helpful to us, and by extension are quite helpful to our patients and our profession.
However, there are also KOLs who have demonstrated their willingness to say and do whatever their corporate backers tell them to, regardless of evidence or proof. With some corporations’ payments to individual orthodontic KOLs approaching a million dollars per year, those who have already compromised their integrity for profit also seem unlikely to place any moral walls between them and their cash flow. Change seems unlikely to come from them.
Us:
While maybe not, I will take an extreme position and say that I do think that we might possibly be excused the very first time that we trust a claim made by a company or a KOL. However, the statement, Fool me once shame on you fool me twice shame on me, seems very appropriate here. Once a corporation has shown us they are willing to make unproven or false claims, and once a KOL has shown they are willing to compromise their moral integrity for those corporate payments, it becomes our responsibility to not trust them after. To borrow from Maya Angelou “When someone shows you who they are, believe them the first time.”
*It becomes our responsibility to make sure that a product has research to justify its marketing claims before we use it.
*It is our responsibility to not trust corporations who have a history of making claims without research. *As some KOLs are moral and helpful, it is our responsibility to remember those KOLs who have pushed unproven and false claims on us.
For these KOLs, at best, we have an obligation to not trust anything they say or sell, without seeing proper research for it. At worst, we might even automatically presume that anything they say or sell is bogus in some way because it is them saying and selling it. In their defense, the former seems safer. For the latter, even a broken moral compass might be compared to a broken clock. It is accurate at least twice per day. Maybe they are occasionally correct.
In any event, as all other systems seem to currently be failing, the burden of responsibility would seem to fall on us, orthodontists, to discern when products are sold without evidence and avoid those products, corporations, and KOLs until evidence says otherwise.
IMO, this was a spectacular article and blog post, to shed light on what is very important to our profession. Thank you for sharing!
We (orthodontists) are not alone!! – the rest of dentistry and medicine are in the same boat
New products being launched at frequent intervals and often before the ‘old’ product has been used enough OR there is enough evidence to see if it works or is better than the existing product
Unfortunately too many journals don’t want to publish what they see as material/ equipment / technique studies – so we end up relying on rather poor manufacturers data. Maybe if more clinical academics under took relevant clinical research rather than than examining some obscure gene sequence (or similar non clinically related studies) we would have useful information to assist our clinical decisions??