CBCT is not reliable for airway assessment.
I have seen many case reports that use CBCT to measure the airway. However, I am not too sure about the accuracy of this technique? This new study gives us interesting findings. This is also relevant to the previous posts on OSA and orthodontics.
There is increasing interest in the use of orthodontic treatment to treat obstructive sleep apnoea in children. Frequently, the effects of treatment on the airway are evaluated by CBCT imaging. Surprisingly, we do not know if CBCT is a reliable measurement tool for any airway changes.
A team from Vancouver evaluated this question in this paper. The European Journal of orthodontics published it.
Reliability of upper airway assessment using CBCT
.
Jason N. Zimmerman
EJO: advanced access. doi:10.1093/ejo/cjy058
They did a really nice clear literature review, and I suggest anybody interested in this area reads this paper. They pointed out that when someone measures the airway on CBCT, they need to correctly orientate the image and select the threshold sensitivity. This introduces a level of subjectivity into the measurement. As a result, they suggested that it is necessary to evaluate the reliability of this technique.
What did they ask?
They asked this simple question;
“What is the intra and inter-examiner reliability of the assessment of volumetric and cross-sectional area of the airway using CBCT”.
What did they do?
Their first step was to do a sample size calculation that showed that they needed to review the images of 9 patients.
They then obtained CBCT images of 10 adult patients that had been taken by one operator using the same tomograph machine.
Then an oral maxillofacial radiologist, an academic orthodontist and academic orthodontist with additional study in sleep medicine, a private practice orthodontist, a senior orthodontic resident and a junior orthodontic resident were trained and calibrated in the assessment of the airway using CBCT images.
The examiners then did an airway analysis for each of the 10 patients. They used the following protocol.
- A manual orientation of the 3D image in the coronal, sagittal and transverse planes.
- Selected the mid-sagittal plane.
- Traced upper pharyngeal airway.
- Manually select and adjust the threshold value so that the software completely filled in the airspace.
They then measured the following
- Minimal cross-sectional area
- Total upper airway volume
- Nasopharyngeal airway volume
- Oro-pharyngeal airway volume
- Hypopharyngeal airway volume
Finally, they calculated the within and between examiner reliability using the Intra-class correlation coefficient (ICC).
What did they find?
They found the following relevant information
- Threshold sensitivity showed poor intra and inter-examiner liability
- The minimal cross-sectional area had poor intra and inter-examiner reliability
- Total airway volume showed good intra and poor inter-examiner reliability.
- Nasopharyngeal airway volume was good intra, but inter-examiner reliability was poor.
- Oropharyngeal volume had excellent intra and inter-examiner reliability.
- Hypopharyngeal volume had moderate intra and inter-examiner reliability.
They also showed that the reliability of the examiner varied with education and experience.
When we consider the effect of these findings, the most critical factor is between examiner values. Interestingly, when they calculated the percentage error between the examiners, they found that the highest values were for the minimal cross-sectional area (27.2%), the airway volume (15%), nasopharyngeal volume (12.8%).
They concluded that
“Further studies are needed before CBCT can be advocated for valid and reliable comparison of upper airway dimensions for our patients”.
What did I think?
I rarely get excited about measurement studies. However, I thought that this was an excellent study that provided us with realistic information about this new technique. I was surprised that similar studies had not been done.
I thought that they wrote a good discussion and they made the following important points.
- The variation in the threshold may not have a significant effect, but when combined with manual orientation and mid slice selection errors, this then reduces the reliability. Particularly with less experienced examiners.
- The percentage errors are so significant that they may be misinterpreted as a “treatment effect”.
- These results raise questions about whether the airway volume and/or minimum cross-sectional area can be directly related to an individual’s obstructive sleep apnoea.
- Furthermore, when we measure the airway in an upright and awake position, the scan may have no correlation with how the airway functions when a person is asleep.
It is also essential to consider that if someone who is measuring a CBCT is aware of whether the view is pre or post-treatment. Then additional biases, albeit subconscious, may influence their measurement.
The ultimate “take home” message to me, was that this technology, while producing nice graphics and other information, is not as accurate as we are led to believe. Are the smoking mirrors around the airway and orthodontics getting more obscured?
Blog update
I would like to give a blog update. We transferred the website to the new servers and this has led to greater stability and speed. I have a new logo and images, so I am not breaking copyright etc. The new website is almost written and will go live in a couple of weeks. Finally, while some people regularly used the apps, the uptake was not great. It is costing £1,000 a year for the app and I have decided that I cannot justify the cost. As a result, I have reluctantly decided to stop the apps, I hope that this is OK.
Emeritus Professor of Orthodontics, University of Manchester, UK.
In this report the patients were positioned in an upright position as it happens in almost all similar investigations. However, OSA happens during sleep time and, therefore, CBCT scans taken in the supine position are more appropriate. Unfortunately the number of reports with patients registered in this position is limited.
The onus of proof lies with those who put forth the theory. Although this research confirms what many of us have been saying for quite some time, we should be challenging the proponents of “Airway Orthodontics” to produce their peer-reviewed data. Here are some questions that I would ask them:
https://orthopundit.com/pediatric-airway-orthodontics-keep-rolling-the-dise/
Great post! I agree current airway studies with CBCT’s are interesting but limited in practical and diagnostic value. Obstructive Sleep Apnea usually happens when patients are sleeping. Current CBCT units captures images with patients sitting or standing up. I wonder how different this images would look with the patient in supine position, allowing for all tissues involved to accommodate to the change in position from a vertically taken image. To understand this we would need studies with hospital CT images, where the patients are actually lying down. The radiation values would be drastically higher, and such a risk would be questionable. A sample of patients that had hospital CT images taken for other reasons, could be gathered and then take an immediate CBCT in vertical position, and see what changes occur in shape and volume, if the DICOM data from different units and technology could be homologated. A few weeks ago, at my Angle Society component, I saw three CBCT images from the same patient (“normal”, breathing and swallowing) and they all looked and measured different… Reliable for diagnosis? I believe not!
What I understand from Kevin’s succinct summary of this study is that there was a lot of intra- and inter-examiner variability. It appears to me that the level of calibration might have been a tad deficient. In practice, radiologic techs do these measurements on a daily basis, and this combined experience might help with intra-examiner reliability. I once heard the phrase ‘a poor workman blames his tools’? So, in addition, if a good edge detection algorithm is used then this might help with the lack of manual intra-examiner reliability, especially when combined with experience. For alignment, one might need to move away from manual adjustments and use geometric morphometrics, such as Procrustes superimposition.
The idea that ‘The percentage errors are so significant that they may be misinterpreted as a treatment effect’ in untenable. In practice, six (6) different operators don’t measure six (6) different subjects in a study, the idea is to only have one variable, and minimize confounding factors.
The idea that ‘These results raise questions about whether the airway volume and/or minimum cross-sectional area can be directly related to an individual’s obstructive sleep apnea’ is non sequitur. From what I understand from Kevin’s succinct summary of this study, the authors made no assessment of the stage of respiration when these values were calculated, and none were done during sleep when airway behavior changes significantly. This was not an aim or objective of this study.
The idea that ‘Furthermore, when we measure the airway in an upright and awake position, the scan may have no correlation with how the airway functions when a person is asleep’ is non sequitur. First, basic sleep physiology dictates that a deficient airway during wakefulness will not get any better during sleep. I will not go into details here, but this was not an aim or objective of this study. Next, CBCT data cannot be used to diagnose a sleep disorder. However, a finding of a deficient airway during wakefulness with a CBCT scan, especially when allied with a sleep questionnaire, etc. might provide sufficient grounds for a referral to a sleep physician.
But, let’s face it, the airway is just hot air as far as orthodontics is concerned – right? The cosmetic value of having, say, 24 straight teeth is the gold standard – and consumers are now beginning to see the sequence of events, as follows;
1. See an Orthodontist who says teeth need to be safely removed without causing airway, breathing or sleep issues
2. See a General Dentist who extracts up to 4 premolar teeth
3. See a General Dentist who does clear aligners and improves the irregularity of the remaining teeth since that’s less expensive and less invasive than braces
4. See an Oral Surgeon who removes up to 4 wisdom teeth
Given the above scenario, I see orthodontic specialists doing only a limited number of complex cases in the future, and perhaps a different group of dentists that will be recognized as having the temerity to suggest that there may be medical necessity to assess risks, benefits and alternatives.
Reference
Celenk M, Farrell ML, Eren H, Kumar K, Singh GD, Lozanoff S. Upper airway detection in cone beam images. J Xray Sci Technol. 18(2):121-135, 2010.
A different group of dentists who basically ignore the evidence and concoct their own set of putative “benefits” based on less than nothing? Refusal to accept facts should not be conflated with temerity. No need for another spurious “specialty” based on nonscientific beliefs. Time to leave medical science to the “real” specialists and not let enthusiastic, but ill-informed dilettantes appropriate a serious medical condition.
I am an avid reader and admirer of this blog.It is commendable and a terrific resource.
In reading the blog over a period of years ,I have 3 points /questions that could be replied to and or discussed.
1~All the studies and discussions that occurred lead me to the conclusion that the only viable rationale for carrying out ortho.therapy is esthetic improvement(personally I disagree !)
2~Many of the studies are based on ,to my mind ,totally outdated ,philosophies ,modalities ~in my view.I cannot believe that the bulk of clinical techniques I was taught ,40 yrs.ago ,has not been improved /changed in that time span .I doubt any other field has stood still ,like this.
3~My criteria for efficient /successful treatment outcomes includes portions of the tenets of this blog.A much more significant criterion ,in my view ,is whether pts ./parents are willing to pay substantial fees for the clinical changes I provide . .If they are not ,I go broke .Its that simple and no amount of academic discourse will alter that.
It is my view that a total reliance on data (stress ~total )is throwing the baby out with the bath water.No study ,however sound ,is definitive .Money talks much louder in my world of fee for service.
Thankyou for allowing input.
OSA specially in children is a huge problem. CBCT might be useful If taken in suppine position. Further investigation should include this patient position for a more accurate conclussion
I think in terms of “airway” treatments (whether orthodontic, dental, or surgical) one needs to question what are the important metrics in order to gauge success of the treatment *over a period of time*, rather than the inter-rater margin of error of a single point in time?
To make a very simplistic analogy, if on January 1st I decide I’m making a New Year’s resolution to lose weight … I would step on my scale to get a reading. But, maybe I’m not sure if the reading is accurate, so I walk over to my neighbor’s house and step on their scale and get a slightly different reading. Concerned by the discrepancy, I then start going door-to-door until I’ve stood on everyone’s bathroom scale … and now I have a dozen readings that are maybe all within 10-15% of each other, and I just don’t know what to make of it. Does this mean scales are inaccurate?
Unfortunately, that’s asking the wrong question. None of the discrepancies between my scale and my neigbors’ scales (the intra-rater scoring) matter when considering airway *results over time* (IMO). I’ve seen CBCTs from oral surgeons performing MMA procedures showing increases in airway space of over 100%. I’ve seen CBCTs from dental appliances that have done the same. When a start vs. finish change shows 100% improvement … whether each reading is off by 10% becomes largely irrelevant. If I was “more or less” 190 pounds on my bathroom scale on January 31st, and I’m 160 by Christmas … does it matter if my neighbor’s scale says I’m actually 162? Not in the least (IMO). So while I don’t discount the accuracy of what the authors claimed in their research, I feel that the research is often inappropriately applied to infer that 3D airway modeling simply can’t be done or isn’t reliable. Like the losing weight example, individual readings may have a margin of error … but readings over time (if airway expansion ends up successful) will have such dramatic gains that the point-in-time discrepancies within the margin of error are are irrelevant compared to the size and scale of improvements received in successful patient cases.