In one of the previous articles, we learned what a Robotic Operating System (ROS) is. Today, let us attempt to understand what the NVIDIA Isaac ROS is. We will also look at some of its key functions/use cases, and the steps involved in installing it.
NVIDIA Isaac GEMs ROS provides a set of GPU-accelerated packages for your ROS2 application – making it convenient for developers to create high-quality NVIDIA hardware-based solutions. NVIDIA Isaac ROS helps improve the throughput on image processing and DNN-based perception models.
These ROS2 packages are built from ROS2 Foxy, the first Long Stable Release (LTS) to come out of the robotics community. Some of the key advantages of Isaac ROS GEMs include:
- Optimized processing (image, CV & DNN) for NVIDIA GPUs and Jetson.
- Modular packages for developers to pick what they want to integrate – whether a single algorithm or the entire pipeline.
- Easy-to-integrate capabilities that ensure rapid prototyping and deployment.
Perception AI Packages for ROS Developers
It is important to note that the ROS 2 nodes should address common image, computer, and DNN processing functionalities in order to deliver high-performance perception to AI-based ROS robotics applications. The below image shows the architecture of NVIDIA Isaac ROS:
Figure 1 – architecture of NVIDIA Isaac ROS
Important use cases of Isaac ROS GEMs
Visual SLAM based Localization
Isaac ROS GEMs are hugely beneficial for autonomous systems that move within an allocated area. Leveraging Visual odometry, Isaac ROS GEMs help to assess the position of the camera with regard to its starting point. ROS developers can also make sure that the SLAM algorithms can be run – in real-time – with HD resolution (1280×720) on NVIDIA Jetson Xavier AGX.
3D Scene reconstruction
Isaac ROS GEMs empower robotic applications to maneuver and navigate through complicated environments. Using the nvblox (preview), they can ensure that the RGB-D data creates a solid 3D model of the selected environment. It can also help the robot avoid unexpected obstacles on the course.
DNN Inference Processing
ROS2 packages can equip developers to harness, fine-tune, and optimize NVIDIA’s inference models before deploying the packages by TensorRT or Triton.
Camera/image processing GEMs are built to make use of Jetson’s CV hardware. If the robotic application is integrated with camera solutions that are connected through a CSI interface, it makes sense to capitalize on NVIDIA’s hardware-accelerated Argus package.
System requirements of Isaac ROS GEMs
- Ubuntu 20.04+
- CUDA 11.4 supported discrete GPU
- VPI 1.1.11
You must first install the NVIDIA Container Toolkit to make use of the Docker container development/runtime environment.
Configure nvidia-container-runtime as the default runtime for Docker by editing /etc/docker/daemon.json to include the following:
and then restart the Docker: sudo systemctl daemon-reload && sudo systemctl restart docker
Note: For best performance on Jetson, ensure that power settings are configured appropriately (Power Management for Jetson).
ROS – Foxy
- To Install ROS 2 – foxy. Ubuntu should be upgraded to 20.04. For reference follow: https://qengineering.eu/install-ubuntu-20.04-on-jetson-nano.html
- After upgrading to 20.04, to install and test ROS 2 – fox, follow the instructions given in: https://docs.ros.org/en/foxy/Installation/Ubuntu-Install-Debians.html
All set? Now, let’s get started with the installation.
How to install and work on Isaac ROS
- Create a ROS2 workspace
mkdir -p ISSAC_ROS/src
Note: The workspace can have any name; this guide assumes you name it ISSAC_ROS.
- Clone the Isaac ROS Common repository to ISSAC_ROS/src/. Check that you have Git LFS installed before cloning to pull down all large files.
sudo apt-get install git-lfs
cd ISSAC_ROS /src
git clone https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common.git
- Source and Build the workspace:
colcon build --packages-up-to isaac_ros_common
Clone the Isaac ROS Argus camera repository to ISSAC_ROS/src/ and build the source
- Source the environment and Launch the node:
ros2 run isaac_ros_argus_camera_mono isaac_ros_argus_camera_mono --ros-args -p device:=0 -p sensor:=0 -p output_encoding:="mono8"
ros2 run isaac_ros_argus_camera_mono isaac_ros_argus_camera_mono --ros-args -p device:=0 -p sensor:=0 -p output_encoding:="yuv422"
Launch the new terminal and source the environment to launch the rqt
Note; make sure you are in the “isaac_ros_argus_camera/” directory
To view the stream
In the rqt pop window go to Plugins->Visulization->Image View
Select the /image_raw
Reference cameras from e-con Systems
e-con Systems, with 18+ years of experience as an OEM manufacturer, has been a long-standing partner of NVIDIA. We have worked with our camera partners to provide the modules – compatible with the NVIDIA Isaac ROS Argus Camera packages.
- Sony STARVIS IMX415 based camera: An 8MP ultra-low-light MIPI CSI-2 camera capable of streaming 4K @ 44 fps .
- Sony STARVIS IMX462 based camera: A Full HD ultra-low-light MIPI camera with superior NIR performance for NVIDIA® Jetson Xavier™ NX developer kit.
We hope you now better understand the NVIDIA® Isaac ROS GEMs, its major use cases, and how you can get started with it. If you are interested in integrating cameras to your robotic system using the NVIDIA Isaac ROS, please write to us at firstname.lastname@example.org. Visit the Camera Selector to see our complete portfolio.
Vinoth Rajagopalan is an embedded vision expert with 15+ years of experience in product engineering management, R&D, and technical consultations. He has been responsible for many success stories in e-con Systems – from pre-sales and product conceptualization to launch and support. Having started his career as a software engineer, he currently leads a world-class team to handle major product development initiatives