OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understanding

Ming Hu1,2,3 *, Peng Xia1,3 *, Lin Wang5 *, Siyuan Yan1,2, Feilong Tang1,3, Zhongxing Xu7, Yimin Luo6, Kaimin Song3, Jurgen Leitner2, Xuelian Cheng1, Jun Cheng8, Chi Liu9, Kaijing Zhou4 †, Zongyuan Ge1,2,3 †
1AIM Lab, Faculty of IT, Monash University, 2Faculty of Engineering, Monash University 3Airdoc-Monash Research, Airdoc, 4Eye Hospital, Wenzhou Medical University, 5Bosch Corporate Research 6King's College London, 7Cornell University, 8Institute for Infocomm Research, A*STAR 9Faculty of Data Science, City University of Macau
ECCV 2024
arXiv Code Data

OphNet:the largest video dataset for ophthalmic surgical workflow analysis

Abstract

Surgical scene perception via videos is critical for advancing robotic surgery, telesurgery, and AI-assisted surgery, particularly in ophthalmology. However, the scarcity of diverse and richly annotated video datasets has hindered the development of intelligent systems for surgical workflow analysis. Existing datasets face challenges such as small scale, lack of diversity in surgery and phase categories, and absence of time-localized annotations. These limitations impede action understanding and model generalization validation in complex and diverse real-world surgical scenarios. To address this gap, we introduce OphNet, a large-scale, expert-annotated video benchmark for ophthalmic surgical workflow understanding. OphNet features:

1) A diverse collection of 2,278 surgical videos spanning 66 types of cataract, glaucoma, and corneal surgeries, with detailed annotations for 102 unique surgical phases and 150 fine-grained operations.

2) Sequential and hierarchical annotations for each surgery, phase, and operation, enabling comprehensive understanding and improved interpretability.

3) Time-localized annotations, facilitating temporal localization and prediction tasks within surgical workflows. With approximately 205 hours of surgical videos, OphNet is about 20 times larger than the largest existing surgical workflow analysis benchmark.

Re-rendering the input video

Re-rendering the input video

Challenge/Workshop

Coming soon...

BibTeX

@article{hu2024ophnet,
  title={OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understanding},
  author={Hu, Ming and Xia, Peng and Wang, Lin and Yan, Siyuan and Tang, Feilong and Xu, Zhongxing and Luo, Yimin and Song, Kaimin and Leitner, Jurgen and Cheng, Xuelian and others},
  journal={arXiv preprint arXiv:2406.07471},
  year={2024}
}