Can we reduce uncertainty in orthodontic treatment?

 Sense about Science: Reducing uncertainty in orthodontic treatment

I spend a large amount of time browsing websites looking for interesting articles on research that is relevant or even not relevant to orthodontics.  One of the most interesting  sites is Sense about Science (http://www.senseaboutscience.org).  Their twitter account is @senseaboutsci. This is a charitable trust that has the aim of helping people make sense of science and evidence. It is directed at lay and clinical people.  I suggest that you have a look at the website.

While having a look at their work, I came across the report of a symposium called “Making sense of Uncertainty” (http://www.senseaboutscience.org/pages/uncertainty.html).  I felt that this was a really interesting document as it explains the role of uncertainty in research, media and public policy.  When I read this it became clear to me that most of their discussion was relevant to orthodontic research and I thought that I should address this in this post and this is my orthodontic interpretation of some of the issues that they raised.

The paper starts of by pointing out that there is rarely such a thing as 100% certainty and scientific investigation aims to  reduce uncertainty as much as possible. When we consider orthodontics, there is no doubt that there is a large amount of uncertainty in almost everything that we do. This is reinforced by the common experience that 10 orthodontists will come up with 10 different treatment plans for one patient! There have also been several research projects that have been carried out to illustrate the diversity of clinical opinion when examining records of patients. For example, the papers by Han et al, and Ribarevski et at.

Uncertainty and confidence

A very nice quote is from Michael Rawlins (who is the Chair of Nice) he states that “Uncertainty is fraught with misinterpretation and he prefers to consider our level of confidence in a finding or decision”.  I felt that this was a clinically relevant way of addressing the problem and we should consider the level of confidence that we have in our “evidence based ” decisions.  This brings us to the interpretation of research papers and the statistics that indicate levels of confidence (or uncertainty).  Importantly, they indicate whether a finding is not simply a random event.

One method of evaluating confidence is to interpret confidence intervals and we are only just getting round to including these in orthodontic research.  One way that I can explain these is to consider a study in which we want to identify the average overjet of 11 year old children in the UK. We cannot make this measurement on all the children, so we select a sample and come up with a mean overjet measurement.  Because this is a sample we are uncertain on the accuracy of this measurement and we calculate the 95% confidence interval. This will indicate the range of values that we would expect the overjet to fall within for 95 out of 100 repeats of the data collection.  The narrower the confidence interval the less the uncertainty and this helps us come to conclusions the level of confidence  that we have in a piece of research.

I would like to illustrate this with the results from a recent systematic review into methods of moving molars distally. I reviewed this in an earlier post (Distal movement of molars: A case of swings and roundabouts?).  This is an open access paper (http://goo.gl/0wVqTC) and I will reproduce a table illustrating the amount of distal movement achieved with different distalising appliances.

If we evaluate this table, we can see that four studies  have been included in a meta analysis and this shows that for a total sample of 75 patients that the intra oral appliance is more effective than headgear in moving molars distally by 1.45 mm. It is clear that this difference is small and not very exciting, nevertheless, we also need to look at the confidence intervals.  These range from -2.74 to -0.15. This means that if we repeated this study 100 times then 95 times out of 100 then the “true mean”  will fall between -2.74 to -0.15. We can interpret this as representing a high degree of uncertainty in this area of our treatment. This is because the values represent a wide range from nearly 3.0 which is clinically significant to 0.15 which is of no value.  I can, therefore, conclude that the mean difference between treatments is not great and the finding has a high level of uncertainty.  In reality, we do not know much about the comparative effectiveness of distalising appliances and headgear. As a result, our decisions should be based on other factors such as our tolerance of risk in providing headgear with its inherent serious but rare risks.

Can we tolerate uncertainty?

So to move on from statistics, if we accept that we cannot eliminate uncertainty then we have to consider if we know enough or do we need to carry our additional research using larger and larger samples and complex studies that are designed to eliminate bias and reduce uncertainty?

This really depends on how we assess the  the quality of the evidence and consider the effects of our treatment.  We could  argue, that if we are providing a “low risk” treatment then we can operate with relatively large levels of uncertainty.  An example of this could be a course of simple Class I non-extraction treatment in which all we need to achieve is alignment of the incisors with no change in buccal segment relationship.  This type of treatment could be provided on a non extraction basis with any fixed appliance system.  The next level of “risk” could be more complex treatment involving irreversible decisions, for example, the extraction of teeth.   Finally, the highest level of risk is probably orthognathic surgery when the incorrect treatment decisions could result in major problems for our patients.  Ironically, when we evaluate orthodontic research, it appears we have concentrated on reducing the uncertainty about various treatment mechanics, bracket selection and the use of screws to help maintain anchorage. We have not, yet, addressed the areas in which there is maximum uncertainty and risk.  Clearly, we need to move on….

I have just heard that our  Cochrane review into Class II treatment has just been published (http://goo.gl/W2YyZ). I will return to this subject and use the review to illustrate our confidence in the research into the treatment of Class II problems.