Point Cloud Library (PCL)

Web Page: http://www.pointclouds.org/gsoc2012/ideas.html

Mailing List: mailto:gsoc2012@pointclouds.org

The Point Cloud Library (PCL) is a standalone, large scale, open project for point cloud processing. A point cloud is a data structure used to represent a collection of multi-dimensional points and is commonly used to represent three-dimensional data such as the output of a stereo camera, 3D scanner, or time-of-flight camera. With the advent of new, low-cost hardware such as the Microsoft Kinect or Asus XTionPro 3D cameras and continued efforts in advanced open source 3D point cloud processing, 3D perception gains more and more importance in many fields.

The PCL library contains numerous state-of-the art algorithms including filtering, feature estimation, surface reconstruction, registration, model fitting, segmentation, tracking, recognition, and many more. These algorithms can be used, for example, to filter outliers from noisy data, stitch 3D point clouds together, segment relevant parts of a scene, extract keypoints and compute descriptors to recognize objects in the world, and create surfaces from point clouds and visualize them -- to name a few.

PCL is released under the terms of the BSD license and is open source software. It is free for commercial and research use. Development of PCL is a large collaborative effort driven by researchers and engineers from many different institutions and companies around the world. PCL is cross-platform, and has been successfully compiled and deployed on Linux, MacOS, Windows, and Android. To simplify development, PCL is split into a series of smaller code libraries, that can be compiled separately. This modularity is important for distributing PCL on platforms with reduced computational or size constraints. PCL aims to be for 3D processing, what the Boost set of libraries are for C++: a collection of fast, modular, and community peer reviewed code libraries that can be used to create powerful and complex applications. PCL is written entirely in C++ and makes use of modern C++ libraries such as Boost and Eigen for all its internal data structures. In order to support applications that require real time point cloud processing, PCL has been designed to take advantage of SSE instructions when available, and a GPU interface is currently under development in association with NVIDIA.

The Point Cloud Library aims to unite the field of point cloud processing. By providing an extensible framework for all the geometric algorithms necessary for 3D perception, PCL enables developers to create applications limited by their imaginations, rather than their 3D geometric knowledge.

Projects

• 2D Image Drawing Operators from VTK in PCL The purpose of the project is to Implement and document various drawing functions using VTK in PCL. Drawing functions include primitive drawings such as lines, circles, polygons etc. as well as alpha blended polygons and graphs, both 2D and 3D, for visualizing histograms and other functions.
• 3D edge extraction from an organized point cloud The goal of our GSOC is to implement a 3D edge extraction algorithm from an organized point cloud. We are interested in the edges come from boundaries of either geometric structures or photometric texture. To find these edges, we will combine multiple cues from RGB-D channels. We plan to extract 2D edges from the given RGB channels and back-project these edges to the given depth channel so as to determine reliable 3D edges. We also plan to exploit edges from depth discontinuities or high curvature regions that may not be captured from the RGB channels. We want the code to provide several options to choose the types of resulting edges such as, depth discontinuity, RGB edge, high curvature, and etc.
• Additional functionalities and improvement for PCL modules My aim is to bring the knowledge acquired in my experience to help the enhancement and continuous growing of PCL library. I expect to give my contribution through the improvement of pcl modules and the development of new functionalities. My focus is on tracking, filters, keypoints, registration, and tree-based structures like kdtree and octree. However, I'm available to work on any module as well.
• Browser Implementation As it appears here, http://www.pointclouds.org/gsoc2012/ideas.html under "WebGL Development"
• Modular Interactive Application for Static and Streaming Point Cloud Data This project aims to develop a full featured GUI for visualization and application of the Point Cloud Library's algorithms. Interfaces will be provided for filtering, registration, surface reconstruction, model fitting, and segmentation of point cloud data. The project will leverage the modular nature of the library to create an application which is easy to maintain and extend as the library develops. Optional additional work will look towards developing a "streaming" mode for working with live camera sources, such as those provided using OpenNI.
• Organized Point Cloud Data Operations The project involves implementation of several algorithms and techniques which are very popular in computer vision. I have already implemented all the algorithms mentioned in the project idea for my courses. I have also been involved in many projects where I needed to implement or use these algorithms. In addition, I would also like to include implementation of some popular computer vision algorithms based on graph cuts for this project.
• PCLModeler: a PCL based Reconstruction Platform There are quite a few related libraries and tools for point cloud based reconstruction, like CGAL, Scanalyze, Meshlab and so on. The key reconstruction steps including registration, cleaning up, surface reconstruction, and visualizing are scattered in different places, making it not very easy to get an easy to use reconstruction platform. It will be great if a ready to use reconstruction platform is provided, which will save users’ efforts a lot from building up a platform from scratch themselves. We believe such a platform will stimulate more researchers and developers to try their new ideas and develop new applications based on PCL.
• Point Cloud Segmentation using Graphical Models In this project I propose to implement a state of the art segmentation approach using graphical models and a highly efficient approximate inference algorithm, which results in faster and better segmentations. Furthermore I propose to create a new pcl package ’pcl_ml’ containing the necessary machine learning techniques such as Markov network structures, inference algorithms and low level functions in a general way such they can be reused for future applications.
• Real-time 3D Applications based on PCL, Microsoft Kinect, and Tegra 3. Today, mobile platforms (such as NVIDIA Tegra 3) are equipped with multi-core CPUs and GPUs that have shown promising capabilities in creating real-time applications that involves image processing and 3D rendering. The increasing capabilities of these devices have opened possibilities of creating interactive applications that were only available to workstation desktops in the past. The goal of this project is to port over and further optimize some of the key features from the PCL library to run on Android devices. Ultimately, we would create a platform that allows developers to create interactive applications using the Microsoft Kinect, Tegra 3, and PCL all out of the box with minimum efforts. Together, we hope this would encourage developers to explore applications that utilize these hardware and provide richer and unique user experiences. Additionally, computational intense tasks such as feature extraction/recognition can be offloaded to a remote server. Then, the computed results are transferred back to the mobile devices in real-time. By decoupling the computation from the devices, we could support a large variety of platforms despite the hardware limitation on some of the mobile devices.