Computer vision tries to replicate, at least partially, the functionality of human visual perception. Some of the many goals of visual perception is to enable our interaction with the physical world which is surrounding us, such as moving around without bumping into obstacles, grasping and touching of objects, and recognizing objects on several levels of abstraction. It has been acknowledged quite early in the progress of computer vision that to achieve these goals, the visual information must be at some point represented in terms of spatial or volumetric models since they can be directly related to the actual 3D physical space that surrounds us.
One of the still popular volumetric part-level models where the actual 3D shape needs to be represented are superquadrics. Superquadrics are defined by a closed surface that can take up the shape of ellipsoids, cylinders, parallelopipeds, and all shapes in-between. They are popular in robotics, for example for grasp planning of previously unknown objects.
We developed in the 1990s the state of the art method for segmentation and reconstruction of superquadrics from range images. The method is still popular and quite widely used which is testified by many citations in Google Scholar (1500 citations anytime, 100 citations since 2014).
There were two reasons that prevented a wider use of this modeling approach in the past:
§lack or a high cost of acquiring 3D data
§iterative method of model recovery that made the method not suitable for real-time applications.
Due to the hardware and algorithmic advances in the past decade there is now a multitude of new methods and devices to acquire 3D image data. However, the iterative nature of the original superquadric recovery method still prevents its use when real-time operation is required.
The path to a faster method is actually quite evident-use deep neural networks which have revolutionized computer vision research in the past few years. During the last few years, Convolutional Neural Networks (CNN) are slowly but surely becoming the default method solve many computer vision related problems. CNN based computational approach in computer vision in general is very fast, can cope with large data input, and has also similarities with the way how our brains are coping with processing of visual data.
We propose therefore in this project proposal to implement segmentation and superquadric model recovery using CNNs. As input to CNNs not only range images should be considered, but 3D point clouds in general. There are two types of applications that would benefit greatly from the results of this project:
§applications where real-time operation is required, such as in autonomous driving,
§applications where huge amounts of 3D data is generated (LiDAR, multi-image photogrammetry) and some intelligent automated processing of such data is needed.
In the proposed project group we have ample experience with superquadric modeling since we are the authors of the state of the art method. On the other hand, we have also ample experience in developing CNN solutions for computer vision tasks. This makes us exceptionally qualified for the proposed project.
Research activity
Engineering sciences and technologies
Range on year
1,72 FTE
Research organisations <https://www.sicris.si/public/jqm/prj.aspx?lang=eng&opdescr=search&opt=2&subopt=403&code1=cmn&code2=auto&psize=10&hits=1&page=1&count=&search_term=J2-9228&id=17293&slng=&order_by=>
Researchers <https://www.sicris.si/public/jqm/prj.aspx?lang=eng&opdescr=search&opt=2&subopt=402&code1=cmn&code2=auto&psize=10&hits=1&page=1&count=&id=17293&slng=&search_term=J2-9228&order_by=>
Project phases and their realization
§WP1: develop a CNN for recovery of single SQs
§WP2: decide in what format to input 3D data points into CNN
§WP3: generate a sufficient set of labeled learning data to train CNNs
All three WorkPackages in the 1st year of the project are under way.
Project bibliographic references
§SLABANJA, Jurij, MEDEN, Blaž, PEER, Peter, JAKLIČ, Aleš, SOLINA, Franc. Segmentation and reconstruction of 3D models from a point cloud with deep neural networks. V: ICT convergence powered by smart intelligence : ICTC 2018. [S. l.: s. n.]. 2018, str. 118-123, ilustr. [COBISS.SI-ID 1537994435], [Scopus up to 10. 2. 2019: no. of citations (TC): 0, pure citations (CI): 0]
Financed by
Slovenian Research Agency