How to Our Product Insights

Enabling auto exposure control for monochrome sensors using NVIDIA ISP

Monochrome sensors lack an inbuilt auto exposure engine - depending on costly external ISPs to enable Auto Exposure Control (AEC). Find out how to set auto exposure in monochrome cameras using the NVIDIA ISP.

Enabling auto exposure control for monochrome sensors using NVIDIA ISP-05-05

Many embedded vision applications today require monochrome cameras to be placed in varying lighting conditions. Some of these applications include smart traffic management, auto farming, outdoor AMRs (Autonomous Mobile Robots), etc. Unfortunately, many monochrome sensors do not come with an inbuilt auto exposure engine, and depend on an external ISP to enable AEC (Auto Exposure Control). However, adding additional hardware or ASCII ISP is a costly affair. The alternate method is to go with an inbuilt ISP that works seamlessly with powerful host processors like the NVIDIA Jetson series which are capable of running heavy workloads and algorithms on them.

At the same time, it is to be noted that using an external ISP does come with certain advantages. Since it is beyond the scope of this article, read about it in the article Camera ISP and significance of using an external ISP in imaging solutions.

Now coming back to using an inbuilt ISP. For product engineers using NVIDIA as the host platform, e-con Systems offers AEC-enabled camera solutions. These include camera systems – that leverage the NVIDIA ISP – such as e-CAM222_CUMI2311_MOD (2MP OV2311 monochrome global shutter camera module) and cameras with monochrome sensors including AR0521. If you wish to look at the entire portfolio of e-con’s cameras for NVIDIA Jetson, please visit the NVIDIA Jetson cameras page.

In this article, we look at the pipeline used to enable AEC using the NVIDIA ISP for such camera solutions.

Auto exposure vs manual exposure

Before we even look at the AEC enablement methodology using NVIDIA ISP, let us first try to understand the terms exposure, auto exposure, and manual exposure.

Camera exposure is nothing but the overall brightness or darkness of an image. It is the amount of light that reaches the film or camera sensor when an image is captured. The more you expose the film or camera sensor to light, higher would be the brightness of the image. On the other hand, a low exposure time implies lesser light falling on the sensor, which in turn results in a darker image.

There are two methods to set the exposure of a camera – manual exposure and auto exposure. In the manual exposure mode, exposure time is fixed by means of manually adjusting the shutter speed of the camera. Auto exposure relies on the ISP block to set and manage shutter speed.

Setting auto exposure using the NVIDIA ISP

Setting auto exposure in a camera requires you to understand three things or activities:

  1. NVIDIA camera architecture
  2. Device tree settings for exposure control
  3. Exposure handling in NVIDIA platform driver

Let us now look at each of them in detail.

NVIDIA camera architecture

To understand how auto exposure is enabled in NVIDIA ISP, it is important to know how the architecture of an NVIDIA based camera system looks like. Have a look at the below diagram which represents the ISP pipeline of an NVIDIA based camera.

Figure 1: NVIDIA camera ISP pipeline

A camera sensor can be accessed using two pipelines – camera core library interface and direct V4L2 interface.

  1. Camera core library interface – It provides all the controls and performs all the processing between the application and sensor drivers. This library interface uses NVIDIA’s ISP for image processing where AEC can be enabled for monochrome sensors.
  2. Direct V4L2 interface – This method bypasses the core libraries provided by NVIDIA ISP and captures the raw data directly from the sensor using V4L2 APIs. In this mode, only inbuilt AEC from the sensor can be used. Otherwise, you can develop your own algorithm or leverage a ready to use third-party algorithm.
Device tree settings for exposure control

In this section, let us look at the device tree settings required for enabling AEC.

  1. Setting use_sensor_mode_id property value to “true” makes the NVIDIA platform drivers to use the camera exposure, gain, and frame rate related settings provided in the device tree. The below image depicts this configuration:

    Figure 2: Sensor mode setting
  2. Following entries determine the minimum exposure time, maximum exposure time, step exposure time, and exposure factor for the camera sensor in that particular resolution.

    Figure 3: Exposure time settings
  3. The exposure_factor property determines the unit in which minimum, maximum and step exposure time are mentioned in the device tree. For example, in the code snippet above, the parameter ‘exposure factor =1000000’ defines that all the exposure time values are in the unit microseconds (us).
Exposure handling in NVIDIA platform driver

Now that we understand how to configure device tree settings, let us learn how to handle exposure at the NVIDIA platform end. Following are the steps required for the same:

  1. Based on the scene, the NVIDIA ISP algorithm calculates the exposure and gain needed for the camera and communicates the values to the sensor driver through cam_ctrl_ops set_exposure ioctl as given below:

    Figure 4: Input/output control for exposure setting
  2. The cam_set_exposure function implemented in the sensor driver then sets the exposure for the camera sensor.

So that wraps up the entire process to enable auto exposure control in monochrome sensors using NVIDIA ISP. If you have any further queries on the topic, please feel free to leave a comment, or drop a note to camerasolutions@e-consystems.com.

Monochrome cameras from e-con Systems

Following is the list of monochrome cameras from e-con Systems:

To enable AEC in e-con Systems’ monochrome cameras on NVIDIA platforms, an additional driver change is required. For any support on this, please write to camerasolutions@e-consystems.com.

Related posts

2 comments

abhishek shastry March 9, 2022 at 10:21 pm

Hi,
I am working with your SEECAM_24UG camera and wanted to know a little more about your ISP algorithm. While I understand that it might be proprietary and you cannot share most of it, just a little overview will help my research. So my main question, is do you control gain settings in your auto exposure algorithm or is it just shutter speed?

Reply
Anitha Jothiprakash March 11, 2022 at 1:35 pm

Please be informed that, the gain settings is controlled in the exposure algorithm. If you require further support, please do not hesitate to contact us.

Reply

Leave a Comment