Lattice FPGA–Based Holoscan Cameras on NVIDIA AGX Thor & Orin for Scalable Multi-Sensor Edge AI Systems

Jetson AGX Thor is NVIDIA’s most powerful Edge AI computing platform to date. Built on the Blackwell architecture, Thor delivers up to 2070 FP4 TFLOPS of AI compute, approximately 7.5× higher than NVIDIA AGX Orin. It features 2560 CUDA cores, 96 Tensor Cores, and up to 128 GB LPDDR5X memory, making it ideal for humanoid robotics and real-time, multi-sensor Edge AI workloads.

When paired with a Holoscan camera powered by Lattice Holoscan Sensor Bridge, Edge AI systems can achieve sub-microsecond sensor synchronization while offloading key ISP tasks, such as demosaicing, color correction, and frame alignment, from the NVIDIA Jetson Platform. By processing those operations at the edge, Jetson Thor can devote itself fully to high-compute AI inference with ultra-low latency and maximum overall throughput.

Let’s first understand the Holoscan Sensor Bridge (HSB) before getting into the comparison between the HSB on NVIDIA Jetson Thor and NVIDIA Orin.

Understanding Holoscan Sensor Bridge Board

Holoscan sensor Bridge Board is a high-performance, FPGA-powered, Ethernet-based camera interface, designed for low-latency, multi-sensor Edge AI systems. Instead of connecting cameras directly to the SoC through MIPI CSI or USB, HSB introduces a networked, FPGA-based bridge between sensors and the Host. It receives RAW data, processes it, and efficiently transports it via Ethernet to the NVIDIA host.

The HSB architecture typically runs on two Lattice FPGA CPNX/CLNX devices or on a single Lattice FPGA, based on the system requirements. The software layer contains the Holoscan Sensor Bridge IP, paired with the NVIDIA SDK, to deliver a sensor-agnostic data-to-Ethernet host platform. It is compatible with NVIDIA platforms, such as Jetson Thor series modules and AGX Orin™ series modules.

The image below provides a visual overview of the Holoscan sensor Bridge Board.

What Are the Advantages of HSB vs. Alternative Technologies?

HSB offers several advantages when compared to commonly used camera interfaces.

Quick comparison: HSB vs. other interfaces

	MIPI	GMSL	HSB
Bandwidth	Good	Good	Excellent
Cable Length	Limited	Good	Excellent
Synchronization / Sensor Fusion	Ideal for multiple cameras	Ideal for multiple cameras	Ideal for various sensors (including cameras)
PTP	Require custom hardware	Require custom hardware	Supported
Drivers	Kernel driver needs to be changed	Kernel driver needs to be changed	No change on the kernel level
Cables for multisensor	N/A	N/A	Good

Transmission method: In the Holoscan Sensor Bridge, multiple cameras share a common network architecture using high-bandwidth Ethernet transport (10G/25G/40G), which in turn enables scalable, distributed multi-sensor systems not possible with traditional point-to-point links.

The other camera interfaces-MIPI CSI, USB, and GMSL- all have stringent limits on cable lengths through their dedicated lanes or ports, making them suitable only for short-distance connections and small-scale sensor setups.

Scalability: Due to physical lane constraints and the number of CSI ports in the SoC, adding cameras is difficult. In HSB, however, it is scalable for many cameras with switches.

Single/multi-camera sync with NVIDIA host system: The HSB has built on the Precision Time Protocol (PTP) to provide reliable multi-camera time alignment. In contrast, multi-camera sync is complicated across distances in other camera interfaces.

CPU load: The CPU manages sensor control, data reception, and sync timing, while the HSB frees the CPU from AI workloads.

Flexibility: Bandwidth is fixed, and the redesign is required whenever the sensor changes. This limitation is overcome in HSB, which provides a sensor-agnostic FPGA interface and supports multiple sensors.

What is Lattice FPGA & Its Role in HSB?

Lattice Semiconductor made the first reference HSB board for the Holoscan camera in collaboration with NVIDIA.

Lattice HSB Board introduced a powerful and flexible way to enable high-speed Ethernet communication between an FPGA and an NVIDIA Holoscan/Orin platform.

This board integrates two FPGA devices, each playing a critical role in the data path.

• Lattice CrossLink-NX: Handles front-end image acquisition, receiving MIPI CSI-2 sensor data at up to 2.5 Gbps per lane.
• Lattice CertusPro-NX: Processes video streams and packetizes data into Ethernet frames using NVIDIA’s Holoscan IP.

Leveraging CertusPro-NX high-speed SerDes, the board outputs video via dual 10 Gbps SFP+ ports, with simple SFP+ to RJ45 adapters enabling seamless connections to 10G Ethernet networks and NVIDIA Orin kits.

This design makes Holoscan camera bring-up and integration extremely simple, allowing Holoscan camera to be plugged into the system with minimal effort.

Beyond video transmission, the HSB Board also supports reverse-direction Ethernet control traffic, enabling essential functions such as Ping, camera boot sequencing, IP configuration, tuning, diagnostics, and more. The architecture is also flexible enough to include a full ISP pipeline within the CertusPro-NX FPGA, providing customizable, hardware-accelerated image processing in a compact design.

Lattice FPGA-based HSB camera solutions:

1	IMX715	e-CAM85_CUHSB
2	IMX568	e-CAM56_CUHSB
3	AR0234	e-CAM25_CUHSB
4	AR2020	e-CAM200_CUHSB
5	ISX031	e-CAM31_CUHSB
6	IMX900	e-CAM37_CUHSB
7	TOF Camera	DepthVista_HSB

e-con Systems’ Compact HSB Board powered by Lattice FPGA

e-con Systems later introduced an even more optimized solution using advanced Lattice Semiconductor FPGAs: the Compact Single-FPGA HSB Board with an in-house TintE ISP. This design consolidates all the capabilities of Lattice Semiconductor’s two FPGA HSB architectures – MIPI reception, ISP processing, Holoscan IP packetization, and Ethernet data transmission, into a single Lattice CertusPro-NX FPGA.

The image below illustrates the single FPGA-based HSB architecture.

By unifying the entire pipeline in one device, the compact single-FPGA HSB board offers the following key benefits:

• Eliminates the need for two separate FPGAs
• Significantly reduces BOM costs
• Shrinks the form factor to a palm-sized board, making it highly portable for embedded vision systems
• Integrates a 10G Ethernet PHY directly on board, which removes the need for external SFP+ to RJ45 adapters
• Leverages in-house TintE ISP to provide flexible image control and consistent image quality across all supported sensors

This results in a compact, cost-effective, and production-ready 10G Holoscan camera solution.

e-con Systems’ HSB camera solutions:

1	IMX715	e-CAM85_CUHSB
2	IMX568	e-CAM56_CUHSB
3	AR0234	e-CAM25_CUHSB
4	AR2020	e-CAM200_CUHSB
5	ISX031	e-CAM31_CUHSB
6	IMX900	e-CAM37_CUHSB
7	TOF Camera	DepthVista_HSB
8	GMSL Cameras	STURDeCAM31_CUHSB
		NileCAM85_CUHSB
		NileCAM56_CUHSB
		NileCAM200_CUHSB
		NileCAM25_CUHSB
		NileCAM37_CUHSB

Latency Comparison: NVIDIA AGX Thor vs. NVIDIA AGX Orin

NVIDIA Jetson AGX Thor clearly sets a new benchmark in low-latency performance for next-generation camera powered by Lattice HSB. Lattice FPGA-powered e-con Systems’ HSB board and a userspace ISP pipeline are used on both NVIDIA Jetson platforms under the same conditions.

Lattice FPGA-powered e-con Systems’ HSB board + Userspace ISP~60 ms

Platform	Configuration	Measured Latency	Frame Rate	Resolution
NVIDIA Jetson AGX Thor	Lattice FPGA-powered e-con Systems’ HSB board + Userspace ISP	~40 ms	142	1920×1080 @10 bpp
NVIDIA AGX Orin		~60 ms	142	1920×1080 @10 bpp

Jetson Thor delivers ~20% lower latency, setting a new benchmark for real-time cameras.

Integrated Benefits of Jetson Thor, e-con Systems’ Cameras & Lattice Semiconductor FPGA

Ultra-low latency

Combining NVIDIA Jetson Thor, Lattice FPGA, and e-con Systems’ HSB architecture significantly reduces end-to-end latency by offloading ISP tasks from the host processor.

TintE ISP support for e-con Systems’ cameras

e-con Systems’ TintE ISP runs on the Lattice FPGA, delivering consistent, high-quality image processing across all supported e-con camera sensors.

Dual camera support

e-con Systems’ Holoscan Sensor Bridge supports dual-camera setups, enabling synchronized stereo and multi-view vision without additional hardware.

The image below demonstrates two cameras streaming simultaneously through a single HSB board, with separate stereo HoloViz visualizer applications running on Jetson AGX Thor:

The image below shows two cameras streaming simultaneously through a single HSB board, both visualized using the same stereo HoloViz visualizer application on Jetson AGX Thor:

Multi-camera sync using PTP

PTP-based hardware time-stamping ensures sub-microsecond synchronization across multiple cameras, enabling accurate sensor fusion for real-time Edge AI applications.

The image below shows the timestamp measurement and MIPI data reception with PTP synchronization for 1080p resolution @ 10bpp with 30 Hz VSYNC frequency from the FPGA:

The image below shows the timestamp measurement and MIPI data reception without PTP synchronization for 1080p resolution @ 10bpp with 30 Hz VSYNC frequency from the FPGA:

e-con Systems Offers Holoscan-Ready Cameras, Powered by Lattice FPGA for NVIDIA Jetson Thor

Since 2003, e-con Systems has been designing, developing, and manufacturing embedded camera solutions, continuously evolving alongside advances in imaging and AI technologies. As an NVIDIA Elite Partner, e-con Systems supports a wide range of camera products across NVIDIA platforms, including Jetson AGX Thor™, Jetson AGX Orin™, NVIDIA® Jetson Orin NX/Nano™, Jetson Xavier™ NX/Nano/TX2 NX, and Jetson AGX Xavier™.

Through close collaboration with advanced FPGA technologies such as Lattice FPGAs, e-con Systems delivers a high-performance, low-latency, and scalable Holoscan sensor Bridge camera Solution optimized for next-generation edge AI applications.

If you’re looking for the perfect Holoscan-ready camera for your embedded vision application, please email camerasolutions@e-consystems.com.

FAQs

• Why is HSB needed on NVIDIA AGX Thor?
  NVIDIA AGX Thor is designed for large-scale, multi-sensor Edge AI systems. HSB enables Thor to process data from multiple cameras efficiently by offloading sensor control, synchronization, and data transport from the CPU, a capability that other camera interfaces cannot achieve at scale.

• What role does the Lattice FPGA play in HSB?
  The Lattice FPGA acts as the core processing engine in HSB, handling MIPI CSI camera input, ISP processing, PTP-based timestamping, packetization of video data, and Ethernet transmission to the NVIDIA host with deterministic, low latency.

• What is TintE ISP, and where does it run?
  TintE ISP is e-con Systems’ in-house image signal processing pipeline that runs on the Lattice FPGA. It performs functions such as demosaicing, color correction, bandwidth optimization, and frame alignment before data reaches the NVIDIA GPU.

• How does HSB reduce CPU and GPU load on NVIDIA Thor?
  By moving sensor data processing, ISP tasks, synchronization, and data packetization to the FPGA, HSB frees the CPU and GPU to focus entirely on high-performance AI inference and real-time decision-making.