Maritime vision datasets for autonomous navigation: A comparative analysis
DOI:
https://doi.org/10.33175/mtr.2025.277976Keywords:
Maritime Domain, Datasets, Surface Vessels, Computer Vision, Deep Learning, Object Detection, Maritime domain; Datasets; Surface vessels; Computer vision; Deep learning; Object detectionAbstract
Artificial intelligence is becoming an increasingly essential component in many areas, with notable advancements being made in the field of maritime computer vision. The employed deep learning models require substantial quantities of high-quality training data that are specifically tailored to the tasks for which they are being applied in the maritime domain. Training autonomous navigation systems for unmanned surface vehicles has been significantly enhanced by using extensive visual datasets captured through high-quality cameras, enabling these systems to learn from diverse environmental scenarios and improve the decision-making accuracy. However, the identification of suitable publicly accessible maritime vision datasets is challenging, and there is currently no broad overview of datasets that have been specifically designed for computer vision tasks related to unmanned surface vehicles in the maritime domain. This survey addresses the identified research gap by providing a comprehensive and systematic overview of open-source vision datasets containing ships, taking into account the specific task, the surrounding environment, and additional available data, such as infrared images or time series information. It is our aim to assist new researchers in the field of maritime computer vision to gain a rapid overview and facilitate initial access to this domain, enabling them to identify the most suitable dataset for their particular task.
------------------------------------------------------------------------------
Cite this article:
Jungbauer, N., Huang, H., Mayer, H. (2025). Maritime vision datasets for autonomous navigation: A comparative analysis. Maritime Technology and Research, 7(4), 277976. https://doi.org/10.33175/mtr.2025.277976
------------------------------------------------------------------------------
Highlights
- The review provides the most comprehensive and up-to-date survey of 25 open-source maritime vision datasets published between 2015 and October 2024, with a focus on autonomous navigation.
- It introduces a novel set of systematic criteria for dataset analysis tailored to autonomous navigation, evaluating task suitability, environmental diversity, and annotation quality.
- It reveals a positive trend in the annual publication of maritime datasets and identifies the most influential datasets through a citation analysis.
- It offers specific, novel recommendations for autonomous navigation tasks, highlighting the LaRS and MVDD13 datasets for their suitability in advanced applications and robustness in diverse conditions.
References
Abdelsalam, H. E. B., & Elnabawi, M. N. (2024). The transformative potential of artificial intelligence in the maritime transport and its impact on port industry. Maritime Research and Technology, 3(1), 19-31. https://doi.org/10.21622/MRT.2024.03.1.752
Bird, J. J., & Lotfi, A. (2024). CIFAKE: Image classification and explainable identification of aigenerated synthetic images. IEEE Access, 12, 15642-15650. https://doi.org/10.1109/ACCESS.2024.3356122
Bloisi, D.D., Iocchi, L., Pennisi, A., & Tombolini, L. (2015). Argos-venice boat classification (pp. 1-6). In Proceedings of the IEEE International Conference on Advanced Video and Signal Based Surveillance. https://doi.org/10.1109/AVSS.2015.7301727
Bovcon, B., Muhovicˇ, J., Perš, J., & Kristan, M. (2019). The mastr1325 dataset for training deep USV obstacle detection models (pp. 3431-3438). In Proceedings of the International Conference on Intelligent Robots and Systems. https://doi.org/10.1109/IROS40897.2019.8967909
Bovcon, B., Muhovicˇ, J., Vranac, D., Mozeticˇ, D., Perš, J., & Kristan, M. (2021). MODS: A USV-oriented object detection and obstacle segmentation benchmark. IEEE Transactions on Intelligent Transportation Systems, 23(8), 13403-13418. https://doi.org/10.1109/TITS.2021.3124192
Bovcon, B., Mandeljc, R., Perš, J., & Kristan, M. (2018). Stereo obstacle detection for unmanned surface vehicles by IMU-assisted semantic segmentation. Robotics and Autonomous Systems, 104, 1-13. https://doi.org/10.1016/j.robot.2018.02.017
Bradski, G. (2000). The OpenCV Library. Dr. Dobb’s Journal of Software Tools.
Cheng, Y., Zhu, J., Jiang, M., Fu, J., Pang, C., Wang, P., Sankaran, K., Onabola, O., Liu, Y., Liu, D., & Bengio, Y. (2021). FloW: A dataset and benchmark for floating waste detection in Inland Waters (pp. 10953-10962). In Proceedings of the International Conference on Computer Vision. https://doi.org/10.1109/ICCV48922.2021.01077
Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., & Fei-Fei, L. (2009). ImageNet: A large-scale hierarchical image database (pp. 248-255). In Proceedings of the Computer Vision and Pattern Recognition. https://doi.org/10.1109/CVPR.2009.5206848
Dosovitskiy, A., Dosovitskiy, A., Beyer, L., Beyer, L., Kolesnikov, A., Kolesnikov, A., Weissenborn, D., Weissenborn, D., Zhai, X., Zhai, X., Unterthiner, T., Unterthiner, T., Dehghani, M., Dehghani, M., Minderer, M., Minderer, M., Heigold, G., Heigold, G., Gelly, S., Gelly, S., Uszkoreit, J., Uszkoreit, J., Houlsby, N., & Houlsby, N. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. Computer Vision and Pattern Recognition. https://doi.org/10.48550/arXiv.2010.11929
Everingham, M., Eslami, S. A., Van Gool, L., Williams, C. K., Winn, J., & Zisserman, A. (2015). The PASCAL visual object classes challenge: A retrospective. International Journal of Computer Vision, 111, 98-136 https://doi.org/10.1007/s11263-014-0733-5
Gribbestad, M., Hassan, M. U., & Hameed, I. A. (2021). Transfer learning for prognostics and health management (PHM) of marine air compressors. Journal of Marine Science and Engineering, 9(1), 47. https://doi.org/10.3390/jmse9010047
Gundogdu, E., Solmaz, B., Yücesoy, V., & Koc, A. (2017). MARVEL: A large-scale image dataset for maritime vessels (pp. 165-180). In Proceedings of the Asian Conference on Computer Vision. https://doi.org/10.1007/978-3-319-54193-8_11 . Springer
Guo, Y., Liu, R. W., Qu, J., Lu, Y., Zhu, F., & Lv, Y. (2023). Asynchronous trajectory matching-based multimodal maritime data fusion for vessel traffic surveillance in Inland Waterways. IEEE Transactions on Intelligent Transportation Systems, 24(11), 12779-12792. https://doi.org/10.1109/TITS.2023.3285415
Hashmani, M. A., & Umair, M. (2022). A novel visual-range sea image dataset for sea horizon line detection in changing maritime scenes. Journal of Marine Science and Engineering, 10(2), 193. https://doi.org/10.3390/jmse10020193
Iancu, B., Soloviev, V., Zelioli, L., & Lilius, J. (2021). ABOships: An inshore and offshore maritime vessel detection dataset with precise annotations. Remote Sensing, 13(5), 988. https://doi.org/10.3390/rs13050988
Iancu, B., Winsten, J., Soloviev, V., & Lilius, J. (2023). A benchmark for maritime object detection with centernet on an improved dataset, ABOships-PLUS. Journal of Marine Science and Engineering, 11(9), 1638. https://doi.org/10.3390/jmse11091638
International Maritime Organisation: Autonomous Shipping. (2024). Autonomous shipping. Retrieved from https://www.imo.org/en/MediaCentre/HotTopics/Pages/Autonomous-shipping.aspx
Johnson, J. M., & Khoshgoftaar, T. M. (2019). Survey on deep learning with class imbalance. Journal of Big Data, 6(1), 1-54. https://doi.org/10.1186/s40537-019-0192-5
Khan, M. M., Schneidereit, T., Mansouri Yarahmadi, A., & Breuß, M. (2024). Investigating training datasets of real and synthetic images for outdoor swimmer localisation with YOLO. AI, 5(2), 576-593. https://doi.org/10.3390/ai5020030
Kim, J. H., Kim, N., Park, Y. W., & Won, C. S. (2022). Object detection and classification based on YOLO-v5 with improved maritime dataset. Journal of Marine Science and Engineering, 10(3), 377. https://doi.org/10.3390/jmse10030377
Kristan, M., Kenk, V. S., Kovacˇicˇ, & S., Perš, J. (2015). Fast image-based obstacle detection from unmanned surface vehicles. IEEE Transactions on Cybernetics, 46(3), 641-654. https://doi.org/10.1109/TCYB.2015.2412251
Lin, T. Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014). Microsoft COCO: Common objects in context (pp. 740-755). In Proceedings of the 13th European Conference, Zurich, Switzerland. https://doi.org/10.1007/ 978-3-319-10602-1_48
Liu, J., Li, H., Luo, J., Xie, S., & Sun, Y. (2021). Efficient obstacle detection based on prior estimation network and spatially constrained mixture model for unmanned surface vehicles. Journal of Field Robotics, 38(2), 212-228. https://doi.org/10.1002/ rob.21983
Liu, T., Pang, B., Ai, S., & Sun, X. (2020). Study on visual detection algorithm of sea surface targets based on improved YOLOv3. Sensors, 20(24), 7263. https://doi.org/10.3390/ s20247263
Nirgudkar, S., DeFilippo, M., Sacarny, M., Benjamin, M., & Robinette, P. (2023). Massmind: massachusetts maritime infrared dataset. The International Journal of Robotics Research, 42(1-2), 21-32. https://doi.org/10.1177/02783649231153020
Petkovic´, M., Vujovic´, I., Lušic´, Z., & Šoda, J. (2023). Image dataset for neural network performance estimation with application to maritime ports. Journal of Marine Science and Engineering, 11(3), 578. https://doi.org/10.3390/jmse11030578
Prasad, D. K., Rajan, D., Rachmawati, L., Rajabally, E., & Quek, C. (2017). Video processing from electro-optical sensors for object detection and tracking in a maritime environment: A survey. IEEE Transactions on Intelligent Transportation Systems, 18(8), 1993-2016. https://doi.org/10.1109/TITS.2016.2634580
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021). Everyone wants to do the model work, not the data work: Data cascades in high-stakes AI (pp. 1-15). In Proceedings of the Chi Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3411764.3445518
Shao, Z., Wang, Y., Wang, J., Deng, L., Huang, X., Lu, T., Luo, F., & Zhang, R. (2024). GLSD: A global large-scale ship database with baseline evaluations. Geo-Spatial Information Science, 2024; 1-15. https://doi.org/10.1080/10095020.2024.2416896
Shao, Z., Wu, W., Wang, Z., Du, W., & Li, C. (2018). SeaShips: A large-scale precisely annotated dataset for ship detection. IEEE Transactions on Multimedia, 20(10), 2593-2604. https://doi.org/10.1109/TMM.2018.2865686
Su, L., Chen, Y., Song, H., & Li, W. (2023). A survey of maritime vision datasets. Multimedia Tools and Applications, 82(19), 28873-28893. https://doi.org/10.1007/s11042-023-14756-9
Sun, Y., Su, L., Luo, Y., Meng, H., Li, W., Zhang, Z., Wang, P., & Zhang, W. (2022). Global mask R-CNN for marine ship instance segmentation. Neurocomputing, 480, 257-270. https://doi.org/10.1016/j.neucom.2022.01.017
Sun, Y., Su, L., Luo, Y., Meng, H., Zhang, Z., Zhang, W., & Yuan, S. (2022). IRDCLNet: Instance segmentation of ship images based on interference reduction and dynamic contour learning in foggy scenes. IEEE Transactions on Circuits and Systems for Video Technology, 32(9), 6029-6043. https://doi.org/10.1109/tcsvt.2022.3155182
Sun, Z., Hu, X., Qi, Y., Huang, Y., & Li, S. (2023). MCMOD: The multi-category large-scale dataset for maritime object detection. Computers, Materials and Continua, 75(1), 1657-1669. https://doi.org/10.32604/cmc.2023.036558
Teršek, M., Žust, L., & Kristan, M. (2023). eWaSR: An embedded-compute-ready maritime obstacle detection network. Sensors, 23(12), 5386. https://doi.org/10.3390/ s23125386
Wang, N., Wang, Y., Wei, Y., Han, B., & Feng, Y. (2024). Marine vessel detection dataset and benchmark for unmanned surface vehicles. Applied Ocean Research, 142, 103835. https://doi.org/10.1016/j.apor.2023.103835
Wang, P. (2021). Research on comparison of LiDAR and camera in autonomous driving. Journal of Physics: Conference Series, 2093(1), 012032. https://doi.org/10. 1088/1742-6596/2093/1/012032
Zhang, M. M., Choi, J., Daniilidis, K., Wolf, M. T., & Kanan, C. (2015). VAIS: A dataset for recognizing maritime imagery in the visible and infrared spectrums (pp. 10-16). In Proceedings of the Conference on Computer Vision and Pattern Recognition Workshops. https://doi.org/10.1109/CVPRW.2015.7301291
Zhang, W., He, X., Li, W., Zhang, Z., Luo, Y., Su, L., & Wang, P. (2020). An integrated ship segmentation method based on discriminator and extractor. Image and Vision Computing, 93, 103824. https://doi.org/10.1016/j.imavis.2019.11.002
Zheng, Y., & Zhang, S. (2020). McShips: A large-scale ship dataset for detection and fine-grained categorization in the wild (pp. 1-6). In Proceedings of the 2020 IEEE International Conference on Multimedia and Expo. https://doi.org/10.1109/ICME46284. 2020.9102907
Žust, L., & Kristan, M. (2022). Temporal context for robust maritime obstacle detection (pp. 6340-6346). In Proceedings of the International Conference on Intelligent Robots and Systems. https://doi.org/10.1109/IROS47612.2022.9982043 . IEEE
Žust, L., Perš, J., & Kristan, M. (2023). LaRS: A diverse panoptic maritime obstacle detection dataset and benchmark (pp. 20304-20314). In Proceedings of the International Conference on Computer Vision. https://doi.org/10.1109/ICCV51070.2023.01857
Zwemer, M. H., Wijnhoven, R. G., & With, P. H. (2018). Ship detection in harbour surveillance based on large-scale data and CNNs. Visigrapp, 5, 153-160. https://doi.org/10.5220/0006541501530160
Downloads
Published
Issue
Section
Categories
License
Copyright (c) 2025 Maritime Technology and Research
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Copyright: CC BY-NC-ND 4.0
