Skip to main content
Erschienen in: Oral Radiology 3/2023

08.02.2023 | Original Article

Detection of aspiration from images of a videofluoroscopic swallowing study adopting deep learning

verfasst von: Yukihiro Iida, Janne Näppi, Tomoya Kitano, Toru Hironaka, Akitoshi Katsumata, Hiroyuki Yoshida

Erschienen in: Oral Radiology | Ausgabe 3/2023

Einloggen, um Zugang zu erhalten

Abstract

Objectives

A videofluoroscopic swallowing study (VFSS) is conducted to detect aspiration. However, aspiration occurs within a short time and is difficult to detect. If deep learning can detect aspirations with high accuracy, clinicians can focus on the diagnosis of the detected aspirations. Whether VFSS aspirations can be classified using rapid-prototyping deep-learning tools was studied.

Methods

VFSS videos were separated into individual image frames. A region of interest was defined on the pharynx. Three convolutional neural networks (CNNs), namely a Simple-Layer CNN, Multiple-Layer CNN, and Modified LeNet, were designed for the classification. The performance results of the CNNs were compared in terms of the areas under their receiver-operating characteristic curves (AUCs).

Results

A total of 18,333 images obtained through data augmentation were selected for the evaluation. The different CNNs yielded sensitivities of 78.8%–87.6%, specificities of 91.9%–98.1%, and overall accuracies of 85.8%–91.7%. The AUC of 0.974 obtained for the Simple-Layer CNN and Modified LeNet was significantly higher than that obtained for the Multiple-Layer CNN (AUC of 0.936) (p < 0.001).

Conclusions

The results of this study show that deep learning has potential for detecting aspiration with high accuracy.
Literatur
1.
Zurück zum Zitat Belafsky PC, Kuhn MA. The Clinician’s guide to swallowing fluoroscopy. New York: Springer; 2014.CrossRef Belafsky PC, Kuhn MA. The Clinician’s guide to swallowing fluoroscopy. New York: Springer; 2014.CrossRef
2.
Zurück zum Zitat Shaker R, Belafsky PC, Easterling C, Postma GN. Manual of diagnostic and therapeutic techniques for disorders of deglutition. 1st ed. New York: Springer-Verlag; 2013.CrossRef Shaker R, Belafsky PC, Easterling C, Postma GN. Manual of diagnostic and therapeutic techniques for disorders of deglutition. 1st ed. New York: Springer-Verlag; 2013.CrossRef
4.
Zurück zum Zitat Jaffer NM, Edmund D, Au FW-F, Steele CM. Fluoroscopic evaluation of oropharyngeal dysphagia: anatomic, technical, and common etiologic factors. Am J Roentgenol. 2015;204:49–58.CrossRef Jaffer NM, Edmund D, Au FW-F, Steele CM. Fluoroscopic evaluation of oropharyngeal dysphagia: anatomic, technical, and common etiologic factors. Am J Roentgenol. 2015;204:49–58.CrossRef
6.
Zurück zum Zitat Langmore SE, Krisciunas GP, Warner H, White SD, Dvorkin D, Fink D, et al. Abnormalities of aspiration and swallowing function in survivors of acute respiratory failure. Dysphagia. 2020;36:831–41.PubMedCrossRef Langmore SE, Krisciunas GP, Warner H, White SD, Dvorkin D, Fink D, et al. Abnormalities of aspiration and swallowing function in survivors of acute respiratory failure. Dysphagia. 2020;36:831–41.PubMedCrossRef
7.
Zurück zum Zitat Ito M, Kawakami M, Ohara E, Muraoka K, Liu M. Predictors for achieving oral intake in older patients with aspiration pneumonia: Videofluoroscopic evaluation of swallowing function. Geriatr Gerontol Int. 2018;18:1469–73.PubMedCrossRef Ito M, Kawakami M, Ohara E, Muraoka K, Liu M. Predictors for achieving oral intake in older patients with aspiration pneumonia: Videofluoroscopic evaluation of swallowing function. Geriatr Gerontol Int. 2018;18:1469–73.PubMedCrossRef
8.
Zurück zum Zitat Ekberg O, Aksglaede K. Radiology of the pharynx and the esophagus. Berlin: Springer; 2004.CrossRef Ekberg O, Aksglaede K. Radiology of the pharynx and the esophagus. Berlin: Springer; 2004.CrossRef
9.
Zurück zum Zitat Garon BR, Engle M, Ormiston C. Silent aspiration: results of 1,000 videofluoroscopic swallow evaluations. Neurorehabil Neural Repair. 1996;10:121–6.CrossRef Garon BR, Engle M, Ormiston C. Silent aspiration: results of 1,000 videofluoroscopic swallow evaluations. Neurorehabil Neural Repair. 1996;10:121–6.CrossRef
10.
Zurück zum Zitat Homer J, Massey EW. Silent aspiration following stroke. Neurology. 1988;38:317–9.CrossRef Homer J, Massey EW. Silent aspiration following stroke. Neurology. 1988;38:317–9.CrossRef
11.
Zurück zum Zitat Pikus L, Levine MS, Yang YX, Rubesin SE, Katzka DA, Laufer I, et al. Videofluoroscopic studies of swallowing dysfunction and the relative risk of pneumonia. Am J Roentgenol. 2003;180:1613–6.CrossRef Pikus L, Levine MS, Yang YX, Rubesin SE, Katzka DA, Laufer I, et al. Videofluoroscopic studies of swallowing dysfunction and the relative risk of pneumonia. Am J Roentgenol. 2003;180:1613–6.CrossRef
12.
Zurück zum Zitat Bock JM, Varadarajan V, Brawley MC, Blumin JH. Evaluation of the natural history of patients who aspirate. Laryngoscope. 2017;127(Suppl 8):S1-10.PubMedPubMedCentralCrossRef Bock JM, Varadarajan V, Brawley MC, Blumin JH. Evaluation of the natural history of patients who aspirate. Laryngoscope. 2017;127(Suppl 8):S1-10.PubMedPubMedCentralCrossRef
13.
Zurück zum Zitat Ramsey D, Smithard D, Kalra L. Silent aspiration: what do we know? Dysphagia. 2005;20:218–25.PubMedCrossRef Ramsey D, Smithard D, Kalra L. Silent aspiration: what do we know? Dysphagia. 2005;20:218–25.PubMedCrossRef
14.
Zurück zum Zitat Logemann JA. Evaluation and treatment of swallowing disorders. San Diego: College Hill Press; 1983. Logemann JA. Evaluation and treatment of swallowing disorders. San Diego: College Hill Press; 1983.
15.
Zurück zum Zitat Groher ME, Crary MA. Dysphagia: Clinical Management in Adults and Children. 3rd ed. St. Louis: Mosby; 2021. Groher ME, Crary MA. Dysphagia: Clinical Management in Adults and Children. 3rd ed. St. Louis: Mosby; 2021.
16.
Zurück zum Zitat Kim J, Oh BM, Kim JY, Lee GJ, Lee SA, Han TR. Validation of the videofluoroscopic dysphagia scale in various etiologies. Dysphagia. 2014;29:438–43.PubMedCrossRef Kim J, Oh BM, Kim JY, Lee GJ, Lee SA, Han TR. Validation of the videofluoroscopic dysphagia scale in various etiologies. Dysphagia. 2014;29:438–43.PubMedCrossRef
17.
Zurück zum Zitat Martin-Harris B, Brodsky MB, Michel Y, Castell DO, Schleicher M, Sandidge J, et al. MBS measurement tool for swallow impairment-MBSimp. Dysphagia. 2008;23:392–405.PubMedPubMedCentralCrossRef Martin-Harris B, Brodsky MB, Michel Y, Castell DO, Schleicher M, Sandidge J, et al. MBS measurement tool for swallow impairment-MBSimp. Dysphagia. 2008;23:392–405.PubMedPubMedCentralCrossRef
18.
Zurück zum Zitat Han TR, Paik NJ, Park JW, Kwon BS. The prediction of persistent dysphagia beyond six months after stroke. Dysphagia. 2008;23:59–64.PubMedCrossRef Han TR, Paik NJ, Park JW, Kwon BS. The prediction of persistent dysphagia beyond six months after stroke. Dysphagia. 2008;23:59–64.PubMedCrossRef
19.
Zurück zum Zitat Kendall KA, McKenzie S, Leonard RJ, Gonçalves MI, Walker A. Timing of events in normal swallowing: a videofluoroscopic study. Dysphagia. 2000;15:74–83.PubMedCrossRef Kendall KA, McKenzie S, Leonard RJ, Gonçalves MI, Walker A. Timing of events in normal swallowing: a videofluoroscopic study. Dysphagia. 2000;15:74–83.PubMedCrossRef
20.
Zurück zum Zitat Premakumar Y, Griffin MF, Szarko M. Morphometric characterisation of human tracheas: focus on cartilaginous ring variation. BMC Res Notes. 2018;11:32.PubMedPubMedCentralCrossRef Premakumar Y, Griffin MF, Szarko M. Morphometric characterisation of human tracheas: focus on cartilaginous ring variation. BMC Res Notes. 2018;11:32.PubMedPubMedCentralCrossRef
21.
Zurück zum Zitat Sperrin M, Winder J. Scientific basis of the royal college of radiologists fellowship (2nd Edition). Bistro: IOP Publishing; 2019.CrossRef Sperrin M, Winder J. Scientific basis of the royal college of radiologists fellowship (2nd Edition). Bistro: IOP Publishing; 2019.CrossRef
22.
Zurück zum Zitat Kim DH, Choi KH, Kim HM, Koo JH, Kim BR, Kim TW, et al. Inter-rater reliability of Videofluoroscopic dysphagia scale. Ann Rehabil Med. 2012;36:791–6.PubMedPubMedCentralCrossRef Kim DH, Choi KH, Kim HM, Koo JH, Kim BR, Kim TW, et al. Inter-rater reliability of Videofluoroscopic dysphagia scale. Ann Rehabil Med. 2012;36:791–6.PubMedPubMedCentralCrossRef
23.
Zurück zum Zitat Chang MC, Lee C, Park D. Validation and inter-rater reliability of the modified videofluoroscopic dysphagia scale (Mvds) in dysphagic patients with multiple etiologies. J Clin Med. 2021;10(13):2990.PubMedPubMedCentralCrossRef Chang MC, Lee C, Park D. Validation and inter-rater reliability of the modified videofluoroscopic dysphagia scale (Mvds) in dysphagic patients with multiple etiologies. J Clin Med. 2021;10(13):2990.PubMedPubMedCentralCrossRef
24.
Zurück zum Zitat Lee JW, Randall DR, Evangelista LM, Kuhn MA, Belafsky PC. Subjective assessment of videofluoroscopic swallow studies. Otolaryngol Head Neck Surg. 2017;156:901–5.PubMedCrossRef Lee JW, Randall DR, Evangelista LM, Kuhn MA, Belafsky PC. Subjective assessment of videofluoroscopic swallow studies. Otolaryngol Head Neck Surg. 2017;156:901–5.PubMedCrossRef
25.
Zurück zum Zitat Leonard RJ, Kendall KA, McKenzie S, Gonçalves MI, Walker A. Structural displacements in normal swallowing: a videofluoroscopic study. Dysphagia. 2000;15:146–52.PubMedCrossRef Leonard RJ, Kendall KA, McKenzie S, Gonçalves MI, Walker A. Structural displacements in normal swallowing: a videofluoroscopic study. Dysphagia. 2000;15:146–52.PubMedCrossRef
26.
27.
Zurück zum Zitat Patterson J, Gibson A. Deep Learning: A Practitioner’s Approach. Sebastopol. CA: O’Reilly Media; 2017 Patterson J, Gibson A. Deep Learning: A Practitioner’s Approach. Sebastopol. CA: O’Reilly Media; 2017
28.
Zurück zum Zitat McBee MP, Awan OA, Colucci AT, Ghobadi CW, Kadom N, Kansagra AP, et al. Deep learning in radiology. Acad Radiol. 2018;25(11):1472–80.PubMedCrossRef McBee MP, Awan OA, Colucci AT, Ghobadi CW, Kadom N, Kansagra AP, et al. Deep learning in radiology. Acad Radiol. 2018;25(11):1472–80.PubMedCrossRef
29.
Zurück zum Zitat Xu J, Zhou C, Lang B, Liu Q. Deep learning and convolutional neural networks for medical image computing: precision medicine, high performance and large-scale datasets. In: Priya D, editor. Advances in computer vision and pattern recognition. 1st ed. Cham: Springer International Publishing; 2017. Xu J, Zhou C, Lang B, Liu Q. Deep learning and convolutional neural networks for medical image computing: precision medicine, high performance and large-scale datasets. In: Priya D, editor. Advances in computer vision and pattern recognition. 1st ed. Cham: Springer International Publishing; 2017.
30.
Zurück zum Zitat Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60–88.PubMedCrossRef Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60–88.PubMedCrossRef
31.
Zurück zum Zitat Lee SJ, Ko JY, Il KH, ChoiIl S. Automatic detection of airway invasion from videofluoroscopy via deep learning technology. Appl Sci (Switzerland). 2020;10:6179. Lee SJ, Ko JY, Il KH, ChoiIl S. Automatic detection of airway invasion from videofluoroscopy via deep learning technology. Appl Sci (Switzerland). 2020;10:6179.
33.
Zurück zum Zitat Lee JT, Park E, Hwang JM, du Jung T, Park D. Machine learning analysis to automatically measure response time of pharyngeal swallowing reflex in videofluoroscopic swallowing study. Sci Rep. 2020;10:14735.PubMedPubMedCentralCrossRef Lee JT, Park E, Hwang JM, du Jung T, Park D. Machine learning analysis to automatically measure response time of pharyngeal swallowing reflex in videofluoroscopic swallowing study. Sci Rep. 2020;10:14735.PubMedPubMedCentralCrossRef
34.
Zurück zum Zitat Lee JT, Park E, Du JT. Automatic detection of the pharyngeal phase in raw videos for the videofluoroscopic swallowing study using efficient data collection and 3d convolutional networks. Sensors. 2019;19:3873.PubMedPubMedCentralCrossRef Lee JT, Park E, Du JT. Automatic detection of the pharyngeal phase in raw videos for the videofluoroscopic swallowing study using efficient data collection and 3d convolutional networks. Sensors. 2019;19:3873.PubMedPubMedCentralCrossRef
35.
Zurück zum Zitat El-Amir H, Hamdy M. Deep learning pipeline building a deep learning model with tensorflow Deep Learning Pipeline Springer. CA, Berkeley: Apress; 2020.CrossRef El-Amir H, Hamdy M. Deep learning pipeline building a deep learning model with tensorflow Deep Learning Pipeline Springer. CA, Berkeley: Apress; 2020.CrossRef
36.
Zurück zum Zitat Saleh H. The deep learning with pytorch workshop: build deep neural networks and artificial intelligence applications with pytorch. Birmingham: Packt Publishing, Limited; 2020. Saleh H. The deep learning with pytorch workshop: build deep neural networks and artificial intelligence applications with pytorch. Birmingham: Packt Publishing, Limited; 2020.
37.
Zurück zum Zitat Saleh H. Applied deep learning with pytorch. Birmingham: Packt Publishing, Limited; 2019. Saleh H. Applied deep learning with pytorch. Birmingham: Packt Publishing, Limited; 2019.
38.
Zurück zum Zitat Osinga D. Deep learning cookbook. 1st ed. Sebastopol, CA: Packt Publishing; 2018. Osinga D. Deep learning cookbook. 1st ed. Sebastopol, CA: Packt Publishing; 2018.
39.
Zurück zum Zitat Ketkar N, Moolayil J. Deep learning with Python: Learn best practices of deep learning models with PyTorch. CA: Springer; 2021.CrossRef Ketkar N, Moolayil J. Deep learning with Python: Learn best practices of deep learning models with PyTorch. CA: Springer; 2021.CrossRef
40.
Zurück zum Zitat Kolodiazhnyi K. Hands-on machine learning with C ++: build, train, and deploy end-to-end machine learning and deep learning pipelines. 1st ed. Birmingham: Packt Publishing; 2020. Kolodiazhnyi K. Hands-on machine learning with C ++: build, train, and deploy end-to-end machine learning and deep learning pipelines. 1st ed. Birmingham: Packt Publishing; 2020.
41.
Zurück zum Zitat Kohinata K, Kitano T, Nishiyama W, Mori M, Iida Y, Fujita H, et al. Deep learning for preliminary profiling of panoramic images. Oral Radiol. 2022;27:1–7. Kohinata K, Kitano T, Nishiyama W, Mori M, Iida Y, Fujita H, et al. Deep learning for preliminary profiling of panoramic images. Oral Radiol. 2022;27:1–7.
42.
Zurück zum Zitat Mori M, Ariji Y, Fukuda M, Kitano T, Funakoshi T, Nishiyama W, et al. Performance of deep learning technology for evaluation of positioning quality in periapical radiography of the maxillary canine. Oral Radiol. 2022;38:147–54.PubMedCrossRef Mori M, Ariji Y, Fukuda M, Kitano T, Funakoshi T, Nishiyama W, et al. Performance of deep learning technology for evaluation of positioning quality in periapical radiography of the maxillary canine. Oral Radiol. 2022;38:147–54.PubMedCrossRef
43.
Zurück zum Zitat Kamimura H, Nonaka H, Mori M, Kobayashi T, Setsu T, Kamimura K, et al. Use of a deep learning approach for the sensitive prediction of hepatitis b surface antigen levels in inactive carrier patients. J Clin Med. 2022;11(2):387.PubMedPubMedCentralCrossRef Kamimura H, Nonaka H, Mori M, Kobayashi T, Setsu T, Kamimura K, et al. Use of a deep learning approach for the sensitive prediction of hepatitis b surface antigen levels in inactive carrier patients. J Clin Med. 2022;11(2):387.PubMedPubMedCentralCrossRef
44.
Zurück zum Zitat McAlister WH, Askin FB. The effect of some contrast agents in the lung: an experimental study in the rat and dog. Am J Roentgenol. 1983;140:245–51.CrossRef McAlister WH, Askin FB. The effect of some contrast agents in the lung: an experimental study in the rat and dog. Am J Roentgenol. 1983;140:245–51.CrossRef
45.
Zurück zum Zitat LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE. 1998;86:2278–323.CrossRef LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proc IEEE. 1998;86:2278–323.CrossRef
46.
Zurück zum Zitat Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. Proceedings of the 32nd International Conference on Machine Learning (ICML). 2015;1–11. Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. Proceedings of the 32nd International Conference on Machine Learning (ICML). 2015;1–11.
47.
Zurück zum Zitat DeVries Z, Locke E, Hoda M, Moravek D, Phan K, Stratton A, et al. Using a national surgical database to predict complications following posterior lumbar surgery and comparing the area under the curve and F1-score for the assessment of prognostic capability. The spine journal. 2021;21:1135–42.PubMedCrossRef DeVries Z, Locke E, Hoda M, Moravek D, Phan K, Stratton A, et al. Using a national surgical database to predict complications following posterior lumbar surgery and comparing the area under the curve and F1-score for the assessment of prognostic capability. The spine journal. 2021;21:1135–42.PubMedCrossRef
48.
Zurück zum Zitat Alzubaidi L, Zhang J, Humaidi AJ, Al-Dujaili A, Duan Y, Al-Shamma O, et al. Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. J Big Data. 2021;8:53.PubMedPubMedCentralCrossRef Alzubaidi L, Zhang J, Humaidi AJ, Al-Dujaili A, Duan Y, Al-Shamma O, et al. Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. J Big Data. 2021;8:53.PubMedPubMedCentralCrossRef
49.
Zurück zum Zitat Zarkada A, Regan J. Inter-rater reliability of the dysphagia outcome and severity scale (DOSS): effects of clinical experience. Audio-Rec Train Dysphagia. 2017;33:329–36.PubMedCrossRef Zarkada A, Regan J. Inter-rater reliability of the dysphagia outcome and severity scale (DOSS): effects of clinical experience. Audio-Rec Train Dysphagia. 2017;33:329–36.PubMedCrossRef
Metadaten
Titel
Detection of aspiration from images of a videofluoroscopic swallowing study adopting deep learning
verfasst von
Yukihiro Iida
Janne Näppi
Tomoya Kitano
Toru Hironaka
Akitoshi Katsumata
Hiroyuki Yoshida
Publikationsdatum
08.02.2023
Verlag
Springer Nature Singapore
Erschienen in
Oral Radiology / Ausgabe 3/2023
Print ISSN: 0911-6028
Elektronische ISSN: 1613-9674
DOI
https://doi.org/10.1007/s11282-023-00669-8

Weitere Artikel der Ausgabe 3/2023

Oral Radiology 3/2023 Zur Ausgabe

„Übersichtlicher Wegweiser“: Lauterbachs umstrittener Klinik-Atlas ist online

17.05.2024 Klinik aktuell Nachrichten

Sie sei „ethisch geboten“, meint Gesundheitsminister Karl Lauterbach: mehr Transparenz über die Qualität von Klinikbehandlungen. Um sie abzubilden, lässt er gegen den Widerstand vieler Länder einen virtuellen Klinik-Atlas freischalten.

Klinikreform soll zehntausende Menschenleben retten

15.05.2024 Klinik aktuell Nachrichten

Gesundheitsminister Lauterbach hat die vom Bundeskabinett beschlossene Klinikreform verteidigt. Kritik an den Plänen kommt vom Marburger Bund. Und in den Ländern wird über den Gang zum Vermittlungsausschuss spekuliert.

Darf man die Behandlung eines Neonazis ablehnen?

08.05.2024 Gesellschaft Nachrichten

In einer Leseranfrage in der Zeitschrift Journal of the American Academy of Dermatology möchte ein anonymer Dermatologe bzw. eine anonyme Dermatologin wissen, ob er oder sie einen Patienten behandeln muss, der eine rassistische Tätowierung trägt.

Ein Drittel der jungen Ärztinnen und Ärzte erwägt abzuwandern

07.05.2024 Klinik aktuell Nachrichten

Extreme Arbeitsverdichtung und kaum Supervision: Dr. Andrea Martini, Sprecherin des Bündnisses Junge Ärztinnen und Ärzte (BJÄ) über den Frust des ärztlichen Nachwuchses und die Vorteile des Rucksack-Modells.

Update Zahnmedizin

Bestellen Sie unseren kostenlosen Newsletter und bleiben Sie gut informiert – ganz bequem per eMail.