How to get through the mire of orthodontic publications?
Every month, there is a vast number of orthodontic publications. As a result, It is easy to get overawed by the sheer volume of information that journals publish. This brings me to question whether there are too many papers. If this is the case, how do I identify the important ones? Read on for how I try to do this.
I came across this paper by an Ireland-based team, and this prompted me to think about the torrent of papers. The EJO published their paper.
Is orthodontic research falling prey to obscure and predatory journals? A bibliometric study
Michael P. Maroulakos et al
EJO advance access: https://doi.org/10.1093/ejo/cjae039
I would like to declare a conflict. I know several members of this team well. Padhraig Fleming is also a co-writer of this blog with me.
What did they ask?
They wanted to
“Evaluate where orthodontic research papers are published and to explore potential relationships between the journal and the characteristic of the study and the authors”.
What did they do?
They did a bibliometric study of 29 databases to search for orthodontic publications published between 1 January 2022-31 December 2023.
One researcher extracted the data from the journals. Using this data they classified the journals as legitimate, uncertain and predatory.
They also divided the journals into established and less established orthodontic journals.
The team identified 38,648 articles. They then analysed these further and identified a final sample of 9020 articles. Next, they randomly selected 1500 articles. The final subset was 753 orthodontic articles. These were from 246 unique journals associated with 117 publishers. This illustrates an incredible number of publications from one dental speciality.
What did they find?
The authors provided a large amount of data and very detailed methods for classifying the journals and papers. I do not have space to go into these here, but I felt that these were the main findings.
- The established journals were AJO-DDO, Angle, EJO, Journal of Orthodontics, Orthodontics and Craniofacial Research and Progress in Orthodontics
- Surprisingly, only 37% of the articles were from journals with orthodontic titles.
- 21% were published in other dental journals.
- 48% of the papers in orthodontic journals were published in the more established journals.
- 78.8% of papers were published in legitimate journals, 8.8% in journals of uncertain legitimacy, and 12.4% in predatory journals. Therefore, 1 in 5 papers were published in predatory or uncertain legitimacy journals.
- When they looked at the type of study. They found that 21% were cross-sectional, 11.9% in vitro, 10.9% were retrospective cohorts, 10.4% were case reports and 8.1% were validation studies. Just under half of the cohort studies did not have controls.
- 67% of the studies were classified as low quality. Only 18% were high quality (RCTs, systematic reviews, non-randomised controlled clinical trials, and prospective cohorts).
- Established orthodontic journals had higher odds of publishing higher-quality studies.
Their overall conclusion was.
“Most orthodontic articles were published in non-orthodontic journals. 1 in 5 were in predatory or uncertain legitimacy publications”.
“Higher quality studies were published in the five “established” orthodontic journals.
What did I think?
This was an interesting paper relevant to all practitioners interested in clinical research. It raises some critical questions. The most important is that a large amount of research is published in non-orthodontic journals. But should we be concerned about this?
In considering this topic, I have thought about my own experiences as a paper author. First and foremost, for those aiming for an academic career, publishing is crucial. There is a strong emphasis on producing as many papers as possible to enhance one’s curriculum vitae. This pressure can lead to a proliferation of papers in non-orthodontic and predatory journals.
When I began my career, I focused on improving the quality of my papers. However, I also aimed to publish as many as possible to impress promotion panels. As a result, some of my early papers may not have been of the highest quality. It was only in the last 20 years, once I was established, that I became very selective and focused on publishing only in the AJO. So there is no doubt that I fell into the “publish or perish” trap.
The current situation is becoming more complicated with the rise of predatory journals which offer fast reviews and publication for a fee. Unfortunately, the quality of the papers published in these journals is consistently low. For this reason, serious researchers should avoid publishing in these journals. Additionally, predatory publications may be linked to potential academic misconduct.
As a result of these factors, we may need to accept that the issues mentioned in this paper are here to stay. As clinician scientists, this necessitates careful consideration of our reading material. This brings me to a series of tips in reading the orthodontic research literature.
Applying a filter?
This is how I select papers to discuss in this blog. These are.
- Concentrate on the “established” journals.
- Scan the titles and abstracts to identify higher-level studies and read these critically.
- Primarily focus on studies concerning clinical research
- Pay some attention to any retrospective study. It may be interesting but it will have selection bias. You need to bear this in mind when you read the conclusions.
- Do not read anything in a predatory journal. These papers are written by authors who are simply trying to get anything published.
I hope that by doing this, I find the most valuable papers. However, I miss some and tend to pick these up by joining social media groups and streams on orthodontic research.
The Real Person!
The Real Person!
Thanks Kevin for raising an interesting topic. Misconduct such as falsified data has been a concern of editors of specialist journals for several years now and it will intensify with tools such as AI. The link is an attachment to a discussion on fraud in medical specialist journals https://www.youtube.com/watch?v=tPhoPTk49JE
Several approaches to improve veracity include:
using a journal with resources available to the Editor (to routinely scan for plagiarism, duplication, fraud and other misbehavior) and perhaps journals that show their determination to support good research practice by actively withdrawing articles if they are demonstrated to be suspect.
Using Open Access journals such as Angle Orthodontist and Australasian Orthodontic Journal as these don’t have paywalls and a commercial bias to publish articles that are ‘novel’.
Journals which foster replication and meta-analyses.
A reduction or elimination of NHST (look up if you don’t know), p-values and their associated claims of proof / significance.
For efficiency we do need tools which maximise the expense of our time in reading, hence weighting our reading to quality journals.
The Real Person!
The Real Person!
The problem is even the supposedly “best” ortho publications have far from good track records of publishing good science or sharing anything that is a true contribution to clinical treatment.
The Real Person!
The Real Person!
Science only ever approaches truth. Journals are never perfect but we collectively need to continually improve our literature.
I think it has always been difficult for journals to filter and then publish, now it is more difficult than ever for us professionals to maintain our grasp on the state of the science. We need to use our ‘bullshitometers’ at lectures even moreso.
The Real Person!
The Real Person!
Excellent tips. Things have certainly moved on from my day!
The Real Person!
The Real Person!
18% were considered high quality studies; however, nearly 7% of these were systematic reviews, which invariably conclude there is insufficient high-quality evidence to inform clinical decision making. Therefore, only about 1 published orthodontic study report in 10 contributes anything to our knowledge and understanding of the effectiveness of orthodontic clinical practice.
The Real Person!
The Real Person!
Hi Kevin,
I get a journal alert throughout Elsevier, ‘Practice Update’. Although handy for a precis, I always wonder at the validity of claims. A recent example is:
Assessing the Impact of AI-Enhanced Diagnostic Tools on the Treatment Planning of Orthodontic Cases
J Pharm Bioallied Sci; 2024 Apr 01; 16 (Suppl 2)S1798-S1800; P Gaonkar, I Mohammed, M Ribin, et al
The precis reads:
TAKE-HOME MESSAGE
The integration of AI into orthodontic treatment planning is a growing area of interest owing to its potential to improve diagnostic precision and treatment outcomes. This randomized controlled trial compared the effectiveness of AI-enhanced diagnostic tools with that of traditional diagnostic methods in 100 orthodontic cases. AI-powered software was used in the AI group for automated cephalometric analysis, 3D model evaluation, and treatment suggestions, whereas the traditional group relied on conventional assessments made by orthodontists. The AI-enhanced group had significantly more accurate treatment plans, fewer appointments, and shorter treatment times than the traditional group. Additionally, patients in the AI group reported higher satisfaction, even though the treatment cost was marginally higher. Although AI improved treatment efficiency and accuracy, the potential bias from clinicians and the AI model was not clearly discussed.
AI-enhanced diagnostic tools can significantly improve treatment outcomes in orthodontics, but clinicians must consider cost implications and ever-evolving advancement when integrating these technologies into practice.
I will of course need to access the original article [hoping it’s not behind a paywall] to ascertain if it is worthwhile information or not. On quick reading one could be excused for thinking we should adopt this technology NOW! Of course if you would like to try this topic that also would be great.
Love you stuff – thank you