Skip to main content

16.05.2024 | Original Article

An endoscopic chisel: intraoperative imaging carves 3D anatomical models

verfasst von: Jan Emily Mangulabnan, Roger D. Soberanis-Mukul, Timo Teufel, Manish Sahu, Jose L. Porras, S. Swaroop Vedula, Masaru Ishii, Gregory Hager, Russell H. Taylor, Mathias Unberath

Erschienen in: International Journal of Computer Assisted Radiology and Surgery

Einloggen, um Zugang zu erhalten

Abstract

Purpose

Preoperative imaging plays a pivotal role in sinus surgery where CTs offer patient-specific insights of complex anatomy, enabling real-time intraoperative navigation to complement endoscopy imaging. However, surgery elicits anatomical changes not represented in the preoperative model, generating an inaccurate basis for navigation during surgery progression.

Methods:

We propose a first vision-based approach to update the preoperative 3D anatomical model leveraging intraoperative endoscopic video for navigated sinus surgery where relative camera poses are known. We rely on comparisons of intraoperative monocular depth estimates and preoperative depth renders to identify modified regions. The new depths are integrated in these regions through volumetric fusion in a truncated signed distance function representation to generate an intraoperative 3D model that reflects tissue manipulation

Results:

We quantitatively evaluate our approach by sequentially updating models for a five-step surgical progression in an ex vivo specimen. We compute the error between correspondences from the updated model and ground-truth intraoperative CT in the region of anatomical modification. The resulting models show a decrease in error during surgical progression as opposed to increasing when no update is employed.

Conclusion:

Our findings suggest that preoperative 3D anatomical models can be updated using intraoperative endoscopy video in navigated sinus surgery. Future work will investigate improvements to monocular depth estimation as well as removing the need for external navigation systems. The resulting ability to continuously update the patient model may provide surgeons with a more precise understanding of the current anatomical state and paves the way toward a digital twin paradigm for sinus surgery
Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Clark DW, Del Signore AG, Raithatha R, Senior BA (2018) Nasal airway obstruction: prevalence and anatomic contributors. Ear, Nose & Throat Journal 97(6):173–176CrossRef Clark DW, Del Signore AG, Raithatha R, Senior BA (2018) Nasal airway obstruction: prevalence and anatomic contributors. Ear, Nose & Throat Journal 97(6):173–176CrossRef
2.
Zurück zum Zitat Moubayed SP, Most SP (2022) Evaluation and management of the nasal airway. Clin Plast Surg 49(1):23–31CrossRefPubMed Moubayed SP, Most SP (2022) Evaluation and management of the nasal airway. Clin Plast Surg 49(1):23–31CrossRefPubMed
3.
Zurück zum Zitat Lee K-I, In SM, Kim J-Y, Hong J-Y, Han K-D, Kim J-S, Jung YG (2021) Association of nasal septal deviation with the incidence of anxiety, depression, and migraine: A national population-based study. PLoS ONE 16(11):0259468CrossRef Lee K-I, In SM, Kim J-Y, Hong J-Y, Han K-D, Kim J-S, Jung YG (2021) Association of nasal septal deviation with the incidence of anxiety, depression, and migraine: A national population-based study. PLoS ONE 16(11):0259468CrossRef
5.
Zurück zum Zitat Lethbridge-Cejku M, Schiller J, Bernadel L (2004) Summary health statistics for u.s. adults: national health interview survey, 2002. Vital and health statistics. Series 10. Data from the National Health Survey 10:1–151 Lethbridge-Cejku M, Schiller J, Bernadel L (2004) Summary health statistics for u.s. adults: national health interview survey, 2002. Vital and health statistics. Series 10. Data from the National Health Survey 10:1–151
6.
Zurück zum Zitat Liu X, Sinha A, Ishii M, Hager GD, Reiter A, Taylor RH, Unberath M (2019) Dense depth estimation in monocular endoscopy with self-supervised learning methods. IEEE Trans Med Imaging 39(5):1438–1447CrossRefPubMedPubMedCentral Liu X, Sinha A, Ishii M, Hager GD, Reiter A, Taylor RH, Unberath M (2019) Dense depth estimation in monocular endoscopy with self-supervised learning methods. IEEE Trans Med Imaging 39(5):1438–1447CrossRefPubMedPubMedCentral
7.
Zurück zum Zitat Liu, X, Stiber M, Huang J, Ishii M, Hager GD, Taylor RH, Unberath M (2020) Reconstructing sinus anatomy from endoscopic video–towards a radiation-free approach for quantitative longitudinal assessment. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020, pp. 3–13 . Springer Liu, X, Stiber M, Huang J, Ishii M, Hager GD, Taylor RH, Unberath M (2020) Reconstructing sinus anatomy from endoscopic video–towards a radiation-free approach for quantitative longitudinal assessment. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2020, pp. 3–13 . Springer
8.
Zurück zum Zitat Liu X, Li Z, Ishii M, Hager GD, Taylor RH, Unberath M (2022) Sage: slam with appearance and geometry prior for endoscopy. In: 2022 International Conference on Robotics and Automation (ICRA), pp. 5587–5593 . IEEE Liu X, Li Z, Ishii M, Hager GD, Taylor RH, Unberath M (2022) Sage: slam with appearance and geometry prior for endoscopy. In: 2022 International Conference on Robotics and Automation (ICRA), pp. 5587–5593 . IEEE
10.
Zurück zum Zitat Chen F, Cui X, Han B, Liu J, Zhang X, Liao H (2021) Augmented reality navigation for minimally invasive knee surgery using enhanced arthroscopy. Comput Methods Programs Biomed 201:105952CrossRefPubMed Chen F, Cui X, Han B, Liu J, Zhang X, Liao H (2021) Augmented reality navigation for minimally invasive knee surgery using enhanced arthroscopy. Comput Methods Programs Biomed 201:105952CrossRefPubMed
11.
Zurück zum Zitat Hu J, Jin X, Lee JB, Zhang L, Chaudhary V, Guthikonda M, Yang KH, King AI (2007) Intraoperative brain shift prediction using a 3d inhomogeneous patient-specific finite element model. J Neurosurg 106(1):164–169CrossRefPubMed Hu J, Jin X, Lee JB, Zhang L, Chaudhary V, Guthikonda M, Yang KH, King AI (2007) Intraoperative brain shift prediction using a 3d inhomogeneous patient-specific finite element model. J Neurosurg 106(1):164–169CrossRefPubMed
12.
Zurück zum Zitat Miga MI, Paulsen KD, Hoopes PJ, Kennedy FE, Hartov A, Roberts DW (2000) In vivo quantification of a homogeneous brain deformation model for updating preoperative images during surgery. IEEE Trans Biomed Eng 47(2):266–273CrossRefPubMed Miga MI, Paulsen KD, Hoopes PJ, Kennedy FE, Hartov A, Roberts DW (2000) In vivo quantification of a homogeneous brain deformation model for updating preoperative images during surgery. IEEE Trans Biomed Eng 47(2):266–273CrossRefPubMed
13.
Zurück zum Zitat François T, Calvet L, Sève-d’ Erceville C, Bourdel N, Bartoli A (2021) Image-based incision detection for topological intraoperative 3d model update in augmented reality assisted laparoscopic surgery. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part IV 24, pp. 647–656 . Springer François T, Calvet L, Sève-d’ Erceville C, Bourdel N, Bartoli A (2021) Image-based incision detection for topological intraoperative 3d model update in augmented reality assisted laparoscopic surgery. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part IV 24, pp. 647–656 . Springer
14.
Zurück zum Zitat Chalasani P, Wang L, Roy R, Simaan N, Taylor RH, Kobilarov M (2016) Concurrent nonparametric estimation of organ geometry and tissue stiffness using continuous adaptive palpation. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 4164–4171 . IEEE Chalasani P, Wang L, Roy R, Simaan N, Taylor RH, Kobilarov M (2016) Concurrent nonparametric estimation of organ geometry and tissue stiffness using continuous adaptive palpation. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 4164–4171 . IEEE
15.
Zurück zum Zitat Wang L, Chen Z, Chalasani P, Yasin RM, Kazanzides P, Taylor RH, Simaan N (2017) Force-controlled exploration for updating virtual fixture geometry in model-mediated telemanipulation. J Mech Robot 9(2):021010 Wang L, Chen Z, Chalasani P, Yasin RM, Kazanzides P, Taylor RH, Simaan N (2017) Force-controlled exploration for updating virtual fixture geometry in model-mediated telemanipulation. J Mech Robot 9(2):021010
16.
Zurück zum Zitat Yasin R, Chalasani P, Zevallos N, Shahbazi M, Li Z, Deguet A, Kazanzides P, Choset H, Taylor RH, Simaan N (2020) Evaluation of hybrid control and palpation assistance for situational awareness in telemanipulated task execution. IEEE Transactions on Medical Robotics and Bionics 3(1):31–43CrossRef Yasin R, Chalasani P, Zevallos N, Shahbazi M, Li Z, Deguet A, Kazanzides P, Choset H, Taylor RH, Simaan N (2020) Evaluation of hybrid control and palpation assistance for situational awareness in telemanipulated task execution. IEEE Transactions on Medical Robotics and Bionics 3(1):31–43CrossRef
17.
Zurück zum Zitat Shi Y, Deng X, Tong Y, Li R, Zhang Y, Ren L, Si W (2022) Synergistic digital twin and holographic augmented-reality-guided percutaneous puncture of respiratory liver tumor. IEEE Trans on Human-Mach Syst 52(6):1364–1374CrossRef Shi Y, Deng X, Tong Y, Li R, Zhang Y, Ren L, Si W (2022) Synergistic digital twin and holographic augmented-reality-guided percutaneous puncture of respiratory liver tumor. IEEE Trans on Human-Mach Syst 52(6):1364–1374CrossRef
21.
Zurück zum Zitat Curless B, Levoy M (1996) A volumetric method for building complex models from range images. Proceedings of the 23rd annual conference on Computer graphics and interactive techniques Curless B, Levoy M (1996) A volumetric method for building complex models from range images. Proceedings of the 23rd annual conference on Computer graphics and interactive techniques
22.
Zurück zum Zitat Lorensen W, Cline H (1987) Marching cubes: a high resolution 3d surface construction algorithm. ACM SIGGRAPH Computer Graphics 21:163CrossRef Lorensen W, Cline H (1987) Marching cubes: a high resolution 3d surface construction algorithm. ACM SIGGRAPH Computer Graphics 21:163CrossRef
23.
Zurück zum Zitat Zeng A, Song S, Nießner M, Fisher M, Xiao J, Funkhouser T (2017) 3dmatch: Learning local geometric descriptors from rgb-d reconstructions. In: CVPR Zeng A, Song S, Nießner M, Fisher M, Xiao J, Funkhouser T (2017) 3dmatch: Learning local geometric descriptors from rgb-d reconstructions. In: CVPR
24.
Zurück zum Zitat Mangulabnan JE, Soberanis-Mukul RD, Teufel T, Hernáindez I, Winter J, Sahu M, Porras JL, Vedula SS, Ishii M, Hager G, Taylor RH, Unberath M (2023) A Quantitative Evaluation of Dense 3D Reconstruction of Sinus Anatomy from Monocular Endoscopic Video Mangulabnan JE, Soberanis-Mukul RD, Teufel T, Hernáindez I, Winter J, Sahu M, Porras JL, Vedula SS, Ishii M, Hager G, Taylor RH, Unberath M (2023) A Quantitative Evaluation of Dense 3D Reconstruction of Sinus Anatomy from Monocular Endoscopic Video
Metadaten
Titel
An endoscopic chisel: intraoperative imaging carves 3D anatomical models
verfasst von
Jan Emily Mangulabnan
Roger D. Soberanis-Mukul
Timo Teufel
Manish Sahu
Jose L. Porras
S. Swaroop Vedula
Masaru Ishii
Gregory Hager
Russell H. Taylor
Mathias Unberath
Publikationsdatum
16.05.2024
Verlag
Springer International Publishing
Erschienen in
International Journal of Computer Assisted Radiology and Surgery
Print ISSN: 1861-6410
Elektronische ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-024-03151-w

Mammakarzinom: Brustdichte beeinflusst rezidivfreies Überleben

26.05.2024 Mammakarzinom Nachrichten

Frauen, die zum Zeitpunkt der Brustkrebsdiagnose eine hohe mammografische Brustdichte aufweisen, haben ein erhöhtes Risiko für ein baldiges Rezidiv, legen neue Daten nahe.

„Übersichtlicher Wegweiser“: Lauterbachs umstrittener Klinik-Atlas ist online

17.05.2024 Klinik aktuell Nachrichten

Sie sei „ethisch geboten“, meint Gesundheitsminister Karl Lauterbach: mehr Transparenz über die Qualität von Klinikbehandlungen. Um sie abzubilden, lässt er gegen den Widerstand vieler Länder einen virtuellen Klinik-Atlas freischalten.

Klinikreform soll zehntausende Menschenleben retten

15.05.2024 Klinik aktuell Nachrichten

Gesundheitsminister Lauterbach hat die vom Bundeskabinett beschlossene Klinikreform verteidigt. Kritik an den Plänen kommt vom Marburger Bund. Und in den Ländern wird über den Gang zum Vermittlungsausschuss spekuliert.

Darf man die Behandlung eines Neonazis ablehnen?

08.05.2024 Gesellschaft Nachrichten

In einer Leseranfrage in der Zeitschrift Journal of the American Academy of Dermatology möchte ein anonymer Dermatologe bzw. eine anonyme Dermatologin wissen, ob er oder sie einen Patienten behandeln muss, der eine rassistische Tätowierung trägt.

Update Radiologie

Bestellen Sie unseren Fach-Newsletter und bleiben Sie gut informiert.