Flexible system of multiple RGB-D sensors for measuring and classifying fruits in agri-food Industry
MetadataShow full item record
The productivity of the agri-food sector experiences continuous and growing challenges that make the use of innovative technologies to maintain and even improve their competitiveness a priority. In this context, this paper presents the foundations and validation of a flexible and portable system capable of obtaining 3D measurements and classifying objects based on color and depth images taken from multiple Kinect v1 sensors. The developed system is applied to the selection and classification of fruits, a common activity in the agri-food industry. Being able to obtain complete and accurate information of the environment, as it integrates the depth information obtained from multiple sensors, this system is capable of self-location and self-calibration of the sensors to then start detecting, classifying and measuring fruits in real time. Unlike other systems that use specific set-up or need a previous calibration, it does not require a predetermined positioning of the sensors, so that it can be adapted to different scenarios. The characterization process considers: classification of fruits, estimation of its volume and the number of assets per each kind of fruit. A requirement for the system is that each sensor must partially share its field of view with at least another sensor. The sensors localize themselves by estimating the rotation and translation matrices that allow to transform the coordinate system of one sensor to the other. To achieve this, Iterative Closest Point (ICP) algorithm is used and subsequently validated with a 6 degree of freedom KUKA robotic arm. Also, a method is implemented to estimate the movement of objects based on the Kalman Filter. A relevant contribution of this work is the detailed analysis and propagation of the errors that affect both the proposed methods and hardware. To determine the performance of the proposed system the passage of different types of fruits on a conveyor belt is emulated by a mobile robot carrying a surface where the fruits were placed. Both the perimeter and volume are measured and classified according to the type of fruit. The system was able to distinguish and classify the 95% of fruits and to estimate their volume with a 85% of accuracy in worst cases (fruits whose shape is not symmetrical) and 94% of accuracy in best cases (fruits whose shape is more symmetrical), showing that the proposed approach can become a useful tool in the agri-food industry.
Is part ofComputers and Electronics in Agriculture, 2017, vol. 139, p. 231-342
European research projects
The following license files are associated with this item:
Showing items related by title, author, creator and subject.
Yandun, Francisco J.; Gregorio López, Eduard; Zúñiga, Marcos; Escolà i Agustí, Alexandre; Rosell Polo, Joan Ramon; Auat Cheein, Fernando A. (ElsevierIFAC (International Federation of Automatic Control), 2016)The detection of the type of soil surface where a robotic vehicle is navigating on is an important issue for performing several agricultural tasks. Satisfactory results in activities such as seeding, plowing, fertilizing, ...
Terrain classification using ToF sensors for the enhancement of agricultural machinery traversability Yandun, Francisco J.; Gregorio López, Eduard; Escolà i Agustí, Alexandre; Rosell Polo, Joan Ramon; Torres-Torriti, Miguel; Auat Cheein, Fernando A. (2017-11-14)Ground properties influence various aspects of mobile machinery navigation including localization, mobility status or task execution. Excessive slipping, skidding or trapping situations can compromise the vehicle itself ...
Gené Mola, Jordi; Gregorio López, Eduard; Guevara, Javier; Auat Cheein, Fernando A.; Sanz Cortiella, Ricardo; Escolà i Agustí, Alexandre; Llorens Calveras, Jordi; Morros Rubió, Josep Ramon; Ruiz Hidalgo, Javier; Vilaplana Besler, Verónica; Rosell Polo, Joan Ramon (Academic Press. Published by Elsevier, 2019-09-21)The development of reliable fruit detection and localization systems provides an opportunity to improve the crop value and management by limiting fruit spoilage and optimised harvesting practices. Most proposed systems for ...