Browsing by Subject "Instance segmentation"

Now showing 1 - 1 of 1

AI-based planting and monitoring of cabbage with a robotic platform
(2024) Lüling, Nils; Griepentrog, Hans W.
Labour shortages, price pressure and changes in legislation are just a few of the drivers of automation and digitalization in field vegetable cultivation. Due to its high-value crops and its high demands on crop maintenance, field vegetable cultivation is the ideal working area for agricultural robotics. However, the versatile and rapid establishment of agricultural robotics systems has so far failed due to the limited adaptivity to the complex working environment under outdoor conditions, the process chain and the applications that an agricultural robot has to carry out in a field. Only through the developing possibilities of using cameras and artificial intelligence can complex automated applications be implemented. The overall aim of this cumulative dissertation was the development and analysis of systems for AI-based crop establishment and crop maintenance of white cabbage with a robotic platform. Three aspects were analysed: (1) Design, prototyping and evaluation of a planting unit for an autonomous planting process of cabbage with a robotic platform. By using AI-based image classification, a camera at the end of the planting unit was used to evaluate the planting quality and dynamically adjust individual planting parameters. (2) Development of a camera-based vegetation monitoring system for determining the fruit volume and leaf area of white cabbage across several growth stages. (3) Analysis of a method for unsupervised image translation for automated exposure adjustment. By reducing the exposure variation, a lower implementation effort and a higher robustness of the detection and segmentation of white cabbage are aimed for. As part of the autonomous crop establishment, a planting unit was developed and constructed that can carry out an automated crop stand establishment process using a robot platform. The analysis of the quality of the planting process showed a comparable planting performance and planting accuracy to conventional systems of automated field vegetable planting. During the development of the planting unit, the focus was placed on an adaptive design of the unit so that machine parameters can be dynamically adjusted during the planting process. It was possible to reduce the energy requirement of the overall system by dynamically opening and closing the planting furrow during the planting process in order to minimize the draft force. It also creates the basis for an autonomous planting process. Using an attached camera and an AI for image classification, the planting quality can also be recorded and planting parameters such as the planting depth and furrow width can then be adjusted in order to influence the plant placement. At the same time, the AI-based image classification can also be used to control the planting process itself. If the planting tape tears or the separation is blocked, no seedlings are planted. The AI recognizes this and can instruct the robot to suspend the planting process. For automated crop monitoring, the camera, in cooperation with a neural network for instance segmentation, offers the possibility of a contact-free and high-resolution recording of plant parameters. Using instance segmentation of the cabbage head, the cabbage plant and the individual cabbage leaves, as well as a depth image generation using structure-from-motion, it was possible to determine plant parameters such as the absolute leaf area, the number of leaves or the fruit volume of the cabbage head across several growth stages. This offers farmers new opportunities in crop management, which can be tailored even more specifically to individual plants using the information collected. As many possibilities as the use of cameras in combination with neural network-based image analysis offers, there are still some challenges. One of the fundamental challenges lies in the provision and annotation of image data to ensure robust image analysis. The more complex the use case, the more varying images the data set must contain in order to provide the neural network with a basis of information with which it can learn the necessary features. To reduce the complexity of the use case of detecting and segmenting cabbage plants, an AI-based image translation was used to standardize the exposure variations. No annotation is required to train the AI-based image translation, which is trained unsupervised. By standardizing the exposure, the complexity of the images can be reduced, which means that fewer images need to be annotated for a robust use of instance segmentation. This method was also tested for varying growth stages and varieties.