A Detailed Guide to Confidence Filtering in Indirect Time of Flight Cameras

Indirect Time-of-Flight sensors produce a confidence frame, in addition to depth information and infrared images. However, this depends on the target distance, reflectivity, and ambient illumination. The purpose of the confidence information is to express the reliability of the depth pixels. Yet, the current methods for filtering confidence data are not accurate, especially when imaging objects at varying distances.

An incorrect interpretation of the confidence frame may result in either removing valid depth pixels or including noisy values, adversely affecting depth quality. To achieve effective depth estimation even under long-range and low-signal conditions, appropriate filtering methods must be used.

Let’s start by understanding what the confidence data actually represents.

What Is Confidence Filter?

Confidence Filter is a processing technique used in indirect Time-of-Flight (iTOF) cameras to improve depth data quality by removing unreliable measurements.

An indirect Time-of-Flight camera returns a confidence frame in addition to the Depth and IR frames. The confidence frame represents the quality and reliability of the depth measurement at each pixel. The confidence filter uses this confidence frame to selectively filter the less reliable depth pixels.

The confidence frame is derived from several metrics: SNR, phase ambiguity, and ambient light interference. The higher the confidence value, the more likely a correct depth measurement is taken, while a lower value indicates noise or uncertainty. Most iTOF sensors output the confidence frame in 12bpp format, where the pixel values will be in the normalized range, typically 0-4095.

Why Do We Need a Confidence Filter?

The quality of depth is very much dependent on several conditions, such as

• Distance to the camera: Generally, objects that are closer to the camera yield stronger reflected intensities, leading to higher SNR. As distance increases, SNR decreases.
• Object Reflectivity: Highly reflective objects generally produce better SNR as they reflect more IR radiation, while dark objects also tend to reflect less IR, resulting in the lower SNR.
• Multipath interference (MPI): Reflections coming from multiple surfaces may influence the depth signal, causing an inappropriate depth estimation.

Due to these considerations, not all pixels in the depth image can be relied upon equally. Using the confidence filter, imprecise pixels in the depth image can be removed, resulting in a more accurate image.

Confidence-Based Depth Filtering in iTOF Cameras: Static vs. Dynamic Approaches

Now, let’s explore how it’s applied in practice to filter out noise and identify accurate depth data using standard and advanced methods.

Standard Confidence-Based Depth Filtering

The confidence value is directly proportional to depth quality:

• High confidence → Depth most likely to be accurate
• Low confidence → Noisy or unreliable depth

A common filtering technique is threshold-based, in which pixels in the depth image with confidence below a set threshold are removed. Confidence-based filtering is available in some iTOF cameras as an integrated processing capability, enabling users to set a confidence threshold at the camera level.

But the use of the fixed confidence threshold has its disadvantages. For greater distances, the value of SNR decreases, leading to a reduction in confidence level. As such, valid pixels that are far may be filtered out.

In the figure below, the image obtained at Threshold 30 (Th_30) appears significantly clearer than that at Threshold 0 (Th_0), where noise is high.
At Threshold 50 (Th_50), noise is further reduced compared to Th_30, at the expense of pixel loss.

Static Threshold_0

Static Threshold_30

Static Threshold_50

The 3D visualization of the above image is shown below.

Enhanced Confidence-Based Depth Filtering

To overcome the drawbacks associated with a uniform confidence threshold value, a dynamic/depth-dependent confidence threshold can be utilized. This method considers the reduction in intensity value with respect to the depth or distance.

The greater the distance involved, the lower the reflective strength of the IR signal. Therefore, SNR and confidence values will decrease as the distance increases.

A better approach to this problem would involve the use of depth-dependent confidence thresholds stated as follows:

• Apply higher confidence scores for nearby objects
• Lower the threshold as the distance increases.

This depth-based filtering, using confidence levels for all pixels, can manage SNR with distance and efficiently and accurately filter noisy pixels.

In the figure below, at Static Threshold 0, all pixels are accepted, causing higher levels of noise. By setting a higher static threshold (e.g., Th_50), we can reduce noise, thereby discarding useful depth data.

Dynamic Threshold eliminates noise whilst preserving valuable depth information due to dynamic adjustment according to the depth of individual pixels or confidence level measurements.

Static Threshold_0

Static Threshold_50

Dynamic Threshold

The 3D visualization of the above image is shown below.

This depth-dependent confidence threshold filtering technique will be available to the upcoming DepthVista_Helix_GMSL_IRD – 1MP 3D Depth CW iToF Camera.

e-con Systems’ Time of Flight (ToF) Cameras

Since 2003, e-con Systems has been designing, developing, and manufacturing OEM and ODM vision platforms. We provide ToF cameras operating in the NIR range (940nm/850nm) for both indoor and outdoor applications.
Our ToF cameras come with advanced technology, including e-con systems’ patent dust denoising filter, flying pixel and many more, to enhance the performance of vision applications.

Building on our expertise, DepthVista_Helix, our next-generation 1MP 3D Depth CW iToF Camera—is set to launch soon, engineered to deliver exceptional 3D accuracy and performance.

We also provide customization based on your needs.
Visit Time of Flight (ToF) Cameras | USB, MIPI, GMSL2 to explore 3D cameras – powered by Time-of-Flight technology.
To check the complete portfolio, click Camera Selector – find your cameras quickly and easily!
For guidance on camera integration, contact us at camerasolutions@e-consystems.com.

FAQS

1. 1. What is a confidence frame in an iToF camera?
      Confidence data indicates the reliability of each depth pixel, helping determine whether a depth measurement is accurate or noisy.
   2. How is confidence data calculated in indirect Time-of-Flight sensors?
      The confidence data are derived from factors such as signal-to-noise ratio (SNR), phase ambiguity, and ambient light interference.
   3. Why does depth quality degrade at longer distances in iToF cameras?
      As distance increases, the reflected IR signal weakens, reducing SNR and leading to lower confidence and noisier depth measurements.
   4. What are the limitations of fixed confidence threshold filtering?
      Fixed thresholds can incorrectly remove valid depth pixels at longer distances, where confidence naturally decreases despite accurate measurements.
   5. What is depth-dependent (adaptive) confidence thresholding?
      Depth-dependent confidence thresholding is a filtering approach in which confidence thresholds vary with depth, allowing lower thresholds for distant objects and higher thresholds for nearby ones.