In order to do their job, many robots need good visual information about the crop or fruit. These images are now mainly available in 2D. 3D images could give the robot even more information, such as the size and shape of a fruit. There are cameras that can take such images, but they are traditionally stationary. The Business Unit Greenhouse Horticulture and Flower Bulbs of Wageningen University & Research (WUR) is therefore developing an app with which good 3D images can be made with a smartphone.
Photogrammetry is a classic technique to create high-resolution 3D images. In that case, an object is photographed from multiple sides (by as many as 16 cameras), and those images are merged. However, this setup of cameras is not easy to move, the position and angle of each camera must be precisely known, and so the crop must therefore be taken to the photo studio.
Lidar
That is why WUR was looking for mobile alternatives to these setups. Two advances in the past decade looked promising, Lidar and Stereovision. A Lidar camera works with a laser that scans the area in front of it. Software can map the scene by analyzing the reflection of the rays back to the Lidar. However, such cameras work with infrared and get interference from bright infrared lights like the
sun, they are therefore not suitable for taking photos in a greenhouse (where there is a lot of sunlight).
Stereovision
A stereo camera (with two cameras) works by using the parallax between the cameras to calculate distance and can be used to image large objects like tree lines and buildings. But this camera is not accurate enough for small objects at a distance.
SLAM algorithm
That is why WUR is developing several apps using the SLAM (Simultaneous Localization and Mapping) algorithm. This algorithm merges photos based on the exact location and direction of the camera. This makes it possible to make 3D images with a smartphone: the user (for example, the grower) then has to take a number of photos in a circle around the object, in a sense bringing photogrammetry to the field. The apps that WUR is developing also contains an algorithm for image recognition: this allows, for example, the tomato to be recognized, and the size and shape can be mapped out.
These apps are currently being tested by WUR researchers, and one is expected to be publicly available in 2023.
For more information:
Wageningen University & Research
www.wur.nl