Short steps on how to read a paper: Part 10. Outcomes
This short step is about the outcomes of a study. As with other sections, understanding how to interpret outcomes is essential to the reading and interpretation of a study.
The correct selection of outcomes is crucial to a research project. If the investigators choose to measure outcomes that are not relevant. Then the findings of the paper are not useful to anyone. This is a problem on many levels. Not least because clinical research is often expensive and arduous. Selection of inappropriate outcomes is a leading cause of waste in research.
The use of irrelevant and obscure outcomes is sadly very true for orthodontic research.
Regular readers of my blog will know that I am not a fan of some traditional orthodontic measures. For example, cephalometric analysis. This is because these measurements are often misused. This is done when investigators generate many related measurements and run simple univariate tests along all of them. They then spot the ones that are significant and base the paper on these. This can distort the interpretation of study findings very significantly. Furthermore, investigators can get excited about differences of less than 1 degree, and this is clinically meaningless.
This approach may have been acceptable when orthodontic research was in its early development. In the present time, these papers are not useful to anyone, apart from the person building their CV. Unfortunately, this practice is getting worse with the development of CBCT, which enables investigators to measure ad infinitum to the smallest degree.
What should you look for?
So, when you are reading a paper, I think that you should look for these properties of the outcomes.
- Is the outcome relevant to my clinical practice? For example, in a study looking at the effectiveness of a new bracket do the authors investigate the duration of treatment. Other clinically relevant outcomes are occlusal index scores, harms due to treatment, socio-psychological measures and appearance of the teeth and face,
- Does the outcome have meaning to our patients? For example, discomfort, the appearance of appliances and effects on self-esteem. Ideally, all outcomes should have value to patients. If they do not, we may be missing some of the critical impacts of treatment.
- Is the effect size that the authors report clinically relevant? This is a significant problem in orthodontics. It does not matter what outcome the authors select if there is no indication of the clinical relevance of the effect size, the paper does not help us.
In summary, if the outcomes lack relevance to your practice or to your patients or we are measuring minute differences, you should start to question the value of the paper.
Emeritus Professor of Orthodontics, University of Manchester, UK.