Time-of-Flight (ToF) cameras measure depth for every pixel by transmitting modulated infrared light and calculating how long the reflections take to return to the sensor. The result is a powerful vision capability for robotics, industrial automation systems, drones, and more.
Yet, the raw signals generated by ToF sensors are rarely free of interference. Factors like ambient light, limited reflectivity of surfaces, integration time constraints, and sensor design all contribute to fluctuations in the measured depth. Without correction, these fluctuations manifest as jitter in point clouds and instability in depth maps, degrading the reliability of downstream applications.
Temporal filters play a critical role by smoothing values across frames. They stabilize static regions and reduce visual noise, preparing data for further refinement.
In this blog, you’ll find out how temporal filters work, the primary algorithms used in ToF cameras, and the trade-offs to be considered when deploying them.
What are the Fundamentals of Time-of-Flight Imaging?
CW ToF cameras operate by emitting a continuous-wave signal at a defined modulation frequency. Each pixel measures the phase difference between emitted and reflected light. Using the speed of light as a constant, the phase data is converted into depth. Alongside the phase map, an amplitude map captures signal strength, providing an additional reference for measurement confidence.
Depth maps can then be converted into 3D point clouds. Each pixel becomes a coordinate in space, colored by RGB, amplitude, or Depth gradient values. These point clouds form the basis for real-time rendering and analysis.
But, without filtering, random noise in the depth map leads to unstable reconstructions and poor usability in practical scenarios.
The comparison video of the iTOF camera with and without temporal filter is shown below.
Why is Temporal Filtering Actually Needed?
Noise in ToF camera systems persists even when exposure and illumination are tuned to their upper limits. Integration time can be extended by lowering the frame rate, which increases photon capture and reduces random shot noise. Yet even with these adjustments, residual fluctuations remain in the depth map due to sensor design constraints, ambient interference, and reflectivity differences across surfaces.
The video below shows the ToF camera output without and with the temporal filter.
The left-side point cloud video shows the unprocessed/raw depth output, where the depth values of a poorly reflective black object fluctuate over time from 892 mm to 896 mm.
The right-side point cloud video shows the filtered output, where the average depth values are considerably more stable, ranging from 893.3 mm to 893.6 mm.
Temporal filtering tackles it by referencing depth values across multiple consecutive frames. It smoothens out pixel-level inconsistencies and suppresses random variations that would otherwise appear as jitter or flicker in static areas of the scene.
This enables greater stability in reconstructed point clouds and clearer inputs for applications that depend on consistent depth data.
What Are the Types of Temporal Filters?
1) Infinite Impulse Response (IIR)
The IIR filter updates each pixel through a recursive equation:
A small α dampens noise strongly, reducing jitter but introducing lag. A large α tracks changes more quickly, at the expense of higher sensitivity to noise. IIR is favored for embedded deployments due to its low computational load and memory demand.
2) Median-Average Filter
The Median-Average filter buffers several frames, sorts pixel values, and outputs either the median or a middle-range average. The method is nonlinear, rejecting outliers that would skew results. Typically, 3–5 buffered frames strike a balance between improved stability and processing overhead.
Its main advantage is strong resilience to spurious measurements, though it requires greater memory and CPU resources than IIR.
3) Adaptive Mean Filter
Adaptive Mean Filter computes temporal variance over a sequence of frames and applies mean averaging only when the variance is below a defined threshold. When the variance is high – typically due to sudden scene changes, the smoothing process is reduced.
This approach preserves sharp transitions and dynamic changes while still reducing noise in stable regions. As a result, it provides better edge and motion preservation compared to standard IIR and median filters.
The video below compares the performance of the Median filter, the IIR Filter, and the Adaptive Mean filter.
The Median filter reduces spatial noise but can distort moving objects, and the IIR filter improves temporal stability but introduces motion ghosting for moving scenes.
Compared to the other filters, the Adaptive Mean filter provides better depth quality by reducing noise while preserving object structure. This filter is used in the latest 1MP 3D iToF GMSL2 Camera.
What Are the Trade-offs in Temporal Filtering?
The biggest challenge in temporal filtering is motion blur. Because values are averaged across frames, rapidly moving objects appear smeared. This makes conventional temporal filters less suitable for scenarios involving continuous fast motion, such as gesture control or autonomous robots operating in crowded environments.
However, the Adaptive Mean temporal filter can be more effective in dynamic scenes, as it selectively applies stronger smoothing to static regions while reducing it for moving objects. This adaptive behavior improves robustness in fast-moving scenarios by reducing motion artifacts without significantly compromising noise reduction performance.
For stationary or slowly changing scenes, temporal filters significantly enhance depth stability, improving the clarity of depth data, and point cloud consistency.
Temporal Filtering: Integration into ToF Pipelines
A ToF processing pipeline begins with tuning system settings: illumination, modulation frequency, and exposure. Once these parameters are fixed, temporal filters handle frame-to-frame instability. The choice of algorithms depends on hardware resources and application context. For instance,
- IIR is best for real-time systems with limited processing budgets
- Median-Average fits higher-resource platforms where improved noise suppression outweighs increased computational cost
- Adaptive Mean filtering is preferred in applications that require noise reduction while preserving object structure and minimizing motion artefacts.
It is also important to note that IIR filtering operates independently on each pixel over time, providing localized temporal smoothing. Median and Mean-based approaches may additionally incorporate neighbourhood information, depending on the implementation.
e-con Systems Offers Trusted Time-of-Flight Cameras
Since 2003, e-con Systems has been designing, developing, and manufacturing embedded vision solutions—from custom OEM cameras to complete ODM platforms. Our ToF cameras use NIR light at 940nm or 850nm for accurate 3D imaging in indoor and outdoor environments. Supported interfaces include USB, MIPI, and GMSL2, with SDKs available for NVIDIA Jetson AGX Orin, AGX Xavier, and x86 systems.
Explore all e-con Systems’ ToF cameras
Browse our Camera Selector to check out our complete portfolio.
As always, if you need help selecting and integrating the right ToF camera into your embedded vision system, please write to camerasolutions@e-consystems.com.
Frequently Asked Questions
- What is the main role of a temporal filter in ToF cameras?
A temporal filter smooths depth measurements over time by combining data from consecutive frames. It reduces jitter and random fluctuations, especially in static regions. The result is a cleaner depth map that improves the quality of point clouds and 3D reconstructions.
- How does an IIR filter differ from a median-average filter?
The IIR filter applies a weighted moving average between past and present values, making it lightweight and resource-friendly. The Median-Average filter, on the other hand, buffers multiple frames, sorts values, and outputs a median or middle-range average. While more robust against outliers, it requires greater memory and processing power.
- Why can temporal filters cause motion blur?
Since temporal filters average or reference multiple frames, fast-moving objects may appear blurred. The blending effect happens because the filter mixes values from different positions across time. For applications with high-speed motion, spatial filters or hybrid approaches are better suited.
- When should I use an IIR filter instead of a median-average filter?
IIR filters are ideal for embedded systems with limited computational resources and for real-time scenarios where low latency is critical. Median-average filters are better for systems with more processing capability, where stronger suppression of outliers is required and minor delays are acceptable.
- What filter should I use for Fast moving scenes?
Adaptive Mean temporal filter can be more effective for fast moving scenes as they can selectively apply stronger smoothing to static regions while reducing the level of smoothing for moving objects, thereby eliminating motion artifacts. This adaptive behavior makes it suitable for fast-moving scenarios without significantly compromising noise reduction performance.
- How do temporal filters fit into a complete ToF camera pipeline?
They are applied after system settings like illumination, exposure, and modulation frequency have been optimized. Temporal filters handle residual frame-to-frame instability before spatial filters or advanced algorithms are used. It ensures stable inputs for point cloud generation and higher-level processing.
Prabu is the Chief Technology Officer and Head of Camera Products at e-con Systems, and comes with a rich experience of more than 15 years in the embedded vision space. He brings to the table a deep knowledge in USB cameras, embedded vision cameras, vision algorithms and FPGAs. He has built 50+ camera solutions spanning various domains such as medical, industrial, agriculture, retail, biometrics, and more. He also comes with expertise in device driver development and BSP development. Currently, Prabu’s focus is to build smart camera solutions that power new age AI based applications.