Autonomous robotics is a fast-growing research discipline opening new scientific as well as technical challenges. Most of the research is invested in self-driving ground vehicles and space exploration, while marine robotics received much less attention. But with 90% of goods moved across the world in vessels, the interest in developing autonomy capabilities for unmanned surface vehicles (USV) has been increasing. A crucial element for autonomous operation is environment perception, which lags far behind the control and hardware research in USVs.
In a closely related field of autonomous vehicles (AV), perception advancements have been primarily driven by deep models, which allow end-to-end learning of complex functions required for a reliable operation. But most of the perception methods developed for USV are hand-crafted or re-use parts of deep models pre-trained on general-purpose RGB-only datasets and fine-tuned on much smaller corpora of maritime images. These datasets are insufficient for developing and training complex models required for highly dynamic, illumination-varying marine environment, in which specular reflections, haze and mirroring are frequently observed. Thus, an opportunity is missed for end-to-end training for complex maritime perception tasks due to the lack of sufficiently large and diverse multi-modal datasets that would reflect the behavior of typical USV sensor modalities in a maritime environment. Another issue with pre-trained models is a limited generalization ability. A sensor replacement or deployment of deep pre-trained methods at a new location (e.g., moving AV from city to a rural environment), typically requires re-capturing and re-annotation of a dataset for re-training the deep perception models, which is time consuming and costly. This is even more pronounced problem in a highly variable maritime environment.
Our goal is to develop the next-generation marine environment perception methods, which will harvest the power of end-to-end trainable deep models. Research challenges essential for safe USV operation will be addressed: (i) general obstacle detection, (ii) long-term tracking with re-identification, (iii) implicit detection of hazardous areas and (iv) multi-modal sensor fusion. Particular focus will be placed on the adaptivity of the models and self-supervised tuning to new environments. New multi-modal datasets will be captured to facilitate the development of these next-generation models.
Research activity
Engineering sciences and technologies
Range on year
1,8 FTE (1,12 at UL FRI)
Research organisations
Faculty of computer and information science, UL
Faculty of electrical engineering, UL
Sirio, d.o.o.
Researchers
izr. prof. dr. Matej Kristan (PI)
doc. dr. Janez Perš (PI at the Faculty of electrical engineering)
izr. prof. dr. Danijel Skočaj
mag. Borja Bovcon
mag. Alan Lukežič
mag. Jon Natanael Muhovič
mag. Mozetič Dean
Aljoša Žerjal
Project phases and their realization
The project will be composed of six work packages. Deep models for robust obstacle detection with scene adaptation capabilities (WP1); Segmentation-based tracking algorithms compatible with the deep obstacle detection architectures (WP2); New trainable deep sensor fusion methods for environment perception (WP3); We will construct large annotated multimodal USV datasets for training and objective evaluation of deep networks in realistic
scenarios (WP4); Two work packages (WP5 and WP6) will contain support activities such as results dissemination and project management. The realization will be carried out in three phases, each one year long.
Project bibliographic references
Financed by
Slovenian Research Agency