How cameras enable vision in new-age agricultural robots

Auto farming or agriculture automation is no longer a distant dream for an industry that has historically dealt with high labor costs, poor manual productivity, and siloed data. There are also constantly major problems concerning policy changes, weather volatility, and other environmental factors that could cause untold losses. It’s why AI-powered autonomous mobile robots have been such a boon to the modern agriculture industry.

And embedded vision has been at the forefront of this revolution by leveraging camera technology to help robots ‘see’. Embedded vision and artificial intelligence have, after all, enabled agriculture businesses of all sizes to significantly cut down manual labor costs while accelerating the speed of operations. In addition, it has led to an unprecedented improvement in crop yields, which has increased profitability.

Today, the scope of embedded vision applications stretches as long as the horizon. Agricultural robots, drones, and tractors used for auto farming make use of camera-based systems for the purposes of navigation, crop health assessment, pest detection, harvesting, planting, weed detection, watering, spreading fertilizer, etc.

Equipped with the right sensor, interface, processor, and ISP (Image Signal Processor), the camera acts as the eye – capturing high-quality images of crops and vegetation. Images obtained from these camera modules are analyzed using computer vision and AI algorithms to identify patterns and check for anomalies or other requirements. The farmer gets access to real-time data about crops, terrains, and weather patterns. It provides actionable insights that tremendously improve their decision-making capabilities.

In this article, we look at some of the modern embedded vision applications in agriculture where robots and drones are used, and how cameras play a part in each of those.

Applications of embedded vision in agricultural robotics

When we say AMRs (Autonomous Mobile Robots) or simply robots in agriculture, we refer to any autonomous equipment that can automate agricultural activities by seamlessly navigating through fields and performing tasks that were traditionally carried out by humans. Hence, we will look at the role of cameras and embedded vision in agricultural robots along the following types of machines:

• Agricultural robots
• Agricultural drones
• Autonomous tractors

Agricultural robots

Agricultural robots come in a very wide range of sizes starting from a little more than a couple of feet in length to many meters long. They are used predominantly for the following tasks:

• Picking, harvesting, and planting
• Weed and bug detection
• Crop detection and assessment
• Watering and spreading fertilizer
• Land preparation and soil analysis

Picking, harvesting, and planting

As the term suggests, picking and harvesting involve plucking fruits, vegetables, flowers, etc. The items or produce picked are then placed in a basket or container. This entire process can be automated using pick and place robots. Here, cameras help to identify the exact coordinates of fruits and vegetables, and capture images for identifying ripeness, size, and type of the produce. AMRs can also be used for planting purposes.

Weed and bug detection

This involves detecting harmful weeds and bugs in order to precisely spread fertilizers. Embedded cameras used in such robots need to deliver clear images even in challenging conditions for the AI algorithm to accurately detect weeds and bugs. Robots that perform these tasks are typically called automated weeders.

Crop detection and assessment

This involves activities like measuring NDVI (Normalized Difference Vegetation Index). NDVI measurement helps in assessing crop growth, density, presence of water & soil, etc. This requires a camera that comes with high NIR (near infrared) sensitivity. This is because, NDVI is calculated by measuring the difference between near-infrared (which is reflected by vegetation) and red light (which is absorbed by vegetation).

Watering and spreading fertilizer

Based on the data collected on weeds, bugs, plants, etc., agricultural robots are used to precisely water crops and spread fertilizers on them. Here, the quality of images plays a significant role since the accuracy of detection by the AI algorithm will heavily depend on the details captured by the camera.

Land preparation and soil analysis

Agricultural robots equipped with cameras also help in land preparation and analyzing soil texture, water content, soil type, etc. These can be done using the same robot or two different robots. Cameras are used to capture visible or NIR images of the soil to enable this.

It can be noted that most agricultural robots are multi-purpose in nature. This means that many of them perform two or more tasks mentioned above.

Camera-enabled autonomous navigation in agricultural robots

In addition to the specific agricultural tasks mentioned above, one of the most important (if not the most important) tasks performed by a camera is depth sensing and obstacle detection. For this, robots use 3D depth cameras to measure depth for easy navigation and avoiding accidents.

Agricultural drones

Drones used in agriculture perform similar tasks as robots with the key difference being their ability to cover a wide area leveraging aerial view. For instance, drones with high-quality cameras that can capture images while they are in motion can cover tens of thousands of acres in a single day! They can be used to monitor granular details of crops, such as exact plant growth, vegetable color, field row spacing, flower count, etc.

Autonomous tractors

Autonomous tractors are also a type of auto farming equipment. Like robots, they operate completely autonomously. They can automate functions like plowing, harvesting, and capturing crop details. Cameras play a part in helping these tractors in depth sensing, obstacle detection, and image capture.

To learn more about how embedded vision is transforming agriculture using autonomous tractors, please read the article How embedded vision is contributing to the AI revolution in autonomous tractors.

Why camera selection is a crucial process in agricultural robots

Now that we have understood various agricultural applications where cameras are used, let us look at why camera selection and evaluation are critical to ensuring the reliable performance of agricultural robots.

Selecting the right camera solution can be a grueling process. Remember that these applications operate in unpredictable outdoor environments where the lighting can vary without warning. Therefore, capturing high-quality images is no easy task. These robots and equipment also come in large sizes which require the cameras to be installed at a large distance from the host processor. Furthermore, these applications need cutting-edge AI processors that are capable of running machine and deep learning algorithms for accurate data analysis.

Some of the critical features required for a camera used in agricultural robots are:

• Flexible cabling
• Shutter type and frame rate
• High dynamic range
• Multi-camera synchronization and external trigger support
• NIR sensitivity
• Enclosures for use in harsh environments

Why e-con Systems is the camera partner of choice for your agricultural robots

e-con Systems offers state-of-the-art embedded cameras to empower modern agriculture applications, whether to detect bugs/weeds, evaluate crop health, monitor seasonal growth, or simply automate your entire harvesting process. e-con’s wide portfolio includes cameras with features such as high dynamic range, global shutter, GMSL2 and FPD Link interfaces for long cable support, multi-camera synchronization, external trigger support, NIR sensitivity, and IP-rated enclosures. We also offer a 3D time of flight camera that offers an accuracy of less than 1% while delivering both the depth and RGB data in a single frame. In addition, we also recently launched NVIDIA Jetson AGX Orin cameras that are suitable for agricultural robots.

Other pertinent features of e-con’s cameras include

• Lightweight, versatile, and optimized for power consumption
• Inbuilt image signal processor for auto exposure and auto white balance functions
• High signal to noise ratio for image output with less noise

Below is a list of e-con cameras suitable for various agricultural applications:

• DepthVista: 3D time of flight camera
• STURDeCAM31_CUOAGX – 3MP 120dB HDR NVIDIA® Jetson Orin™ Camera for Autonomous Mobility
• e-CAM31_CUOAGX – SONY ISX031 120dB HDR camera for NVIDIA® Jetson AGX Orin™
• NileCAM81_CUOAGX: 4K HDR GMSL2 camera for NVIDIA Jetson AGX Orin
• e-CAM82_CUOAGX: 4K SONY STARVIS™ IMX485 ultra low light camera for NVIDIA® Jetson AGX Orin
• STURDeCAM31 – IP69K Automotive Grade GMSL2 HDR camera with LFM
• STURDeCAM25: IP67 Full HD GMSL2 global shutter color camera module
• See3CAM_CU27: Sony STARVIS IMX462 based ultra-low light USB camera with superior NIR performance
• See3CAM_CU81: 4K HDR USB camera based on the 1/2″ AR0821 sensor
• NileCAM21: Full HD GMSL2 HDR camera with LED flicker mitigation and 15 meter cable support
• See3CAM_CU22: Full HD HDR USB camera with LED flicker mitigation
• eCAM22_CUXVR: Sony STARVIS IMX327 based ultra-low light MIPI camera

e-con Systems also offers extensive customization services to ensure its cameras meet the specific requirements of your agricultural robots, vehicles, or drones. We can help you choose the right sensor, interface, optics, and processing platform to make sure your agricultural equipment meet the desired performance standards.

We have worked with numerous agritech customers where we helped them smoothly integrate our cameras into their device.

See how e-con Systems helped a large US-based auto farming solution provider to automate crop yield assessment.

Looking for help in choosing the right camera for your auto farming device? Please write to camerasolutions@e-consystems.com. Our camera experts will be happy to help you. You could also visit the Camera Selector to have a look at our complete portfolio of cameras.