Visualizing Point Clouds in RViz2

Overview

The Determ RViz2 Viewer is a ROS 2 package that provides visualization and management capabilities for Determ 3D Scene Capture devices. This document explains how to set up, configure, and use the package.

Prerequisites

ROS 2: Humble
Ubuntu 22.04
Determ 3D Scene Capture device(s)
Determ Flowbit library 1.3x /1.4x installed (If not installed, download the built library as described in the instructions below).

Installation

1 . Install ROS 2 Humble

The official ROS 2 Humble installation guideline is here

https://docs.ros.org/en/humble/Installation/Ubuntu-Install-Debs.html

2 . Install Required Dependencies

# ROS 2 dependencies
sudo apt update
sudo apt install ros-$ROS_DISTRO-rviz2 ros-$ROS_DISTRO-cv-bridge ros-$ROS_DISTRO-pcl-conversions
sudo apt install ros-$ROS_DISTRO-geometry-msgs ros-$ROS_DISTRO-visualization-msgs
sudo apt install ros-$ROS_DISTRO-std-srvs ros-$ROS_DISTRO-rclcpp-components

# PCL and OpenCV
sudo

# ROS 2 dependencies
sudo apt update
sudo apt install ros-$ROS_DISTRO-rviz2 ros-$ROS_DISTRO-cv-bridge ros-$ROS_DISTRO-pcl-conversions
sudo apt install ros-$ROS_DISTRO-geometry-msgs ros-$ROS_DISTRO-visualization-msgs
sudo apt install ros-$ROS_DISTRO-std-srvs ros-$ROS_DISTRO-rclcpp-components

# PCL and OpenCV
sudo

# ROS 2 dependencies
sudo apt update
sudo apt install ros-$ROS_DISTRO-rviz2 ros-$ROS_DISTRO-cv-bridge ros-$ROS_DISTRO-pcl-conversions
sudo apt install ros-$ROS_DISTRO-geometry-msgs ros-$ROS_DISTRO-visualization-msgs
sudo apt install ros-$ROS_DISTRO-std-srvs ros-$ROS_DISTRO-rclcpp-components

# PCL and OpenCV
sudo

3. Create a ROS 2 Workspace

mkdir -p ~/ros2_ws/src
cd ~/ros2_ws/src

mkdir -p ~/ros2_ws/src
cd ~/ros2_ws/src

mkdir -p ~/ros2_ws/src
cd ~/ros2_ws/src

4. Clone the repository

5. Configure CMakeLists.txt

determ_flowbit_server_v1.3.zip

Download the zip file and unzip the file in some folder.

Edit the CMakeLists.txt file to point to your FlowBit SDK installation directory:

Update the path to your FlowBit SDK library as shown below:

set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib:$ENV{HOME}/Determ/Library/determ_flowbit_server/lib/release")
# For example, if you download and unzip the folder in ~/Determ/Library, it would be 
# set(CMAKE_INSTALL_RPATH"${CMAKE_INSTALL_PREFIX}/lib:$ENV{HOME}/Determ/
# Library/determ_flowbit_server/lib/release")

set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib:$ENV{HOME}/Determ/Library/determ_flowbit_server/lib/release")
# For example, if you download and unzip the folder in ~/Determ/Library, it would be 
# set(CMAKE_INSTALL_RPATH"${CMAKE_INSTALL_PREFIX}/lib:$ENV{HOME}/Determ/
# Library/determ_flowbit_server/lib/release")

set(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/lib:$ENV{HOME}/Determ/Library/determ_flowbit_server/lib/release")
# For example, if you download and unzip the folder in ~/Determ/Library, it would be 
# set(CMAKE_INSTALL_RPATH"${CMAKE_INSTALL_PREFIX}/lib:$ENV{HOME}/Determ/
# Library/determ_flowbit_server/lib/release")

6. Build the Package

source /opt/ros/humble/setup.bash  # Source the Humble setup file in the correct path
# Note: Replace .bash with your shell if you’re not using bash.
#Possible values are: setup.bash, setup.sh, setup.zsh.
cd ~/ros2_ws 
colcon build --packages-select determ_rviz_viewer --cmake-args -Dflowbit-server_DIR=/path/to/your/flowbit-server_DIR
# It is the directory where the unzip fold is.
# For example, if the fold download zip is unziped in /home/determ/Determ/Library, then 
# colcon build --packages-select determ_rviz_viewer --cmake-args -Dflowbit-server_DIR=/home/determ/Determ/Library/determ_flowbit_server/cmake

source /opt/ros/humble/setup.bash  # Source the Humble setup file in the correct path
# Note: Replace .bash with your shell if you’re not using bash.
#Possible values are: setup.bash, setup.sh, setup.zsh.
cd ~/ros2_ws 
colcon build --packages-select determ_rviz_viewer --cmake-args -Dflowbit-server_DIR=/path/to/your/flowbit-server_DIR
# It is the directory where the unzip fold is.
# For example, if the fold download zip is unziped in /home/determ/Determ/Library, then 
# colcon build --packages-select determ_rviz_viewer --cmake-args -Dflowbit-server_DIR=/home/determ/Determ/Library/determ_flowbit_server/cmake

source /opt/ros/humble/setup.bash  # Source the Humble setup file in the correct path
# Note: Replace .bash with your shell if you’re not using bash.
#Possible values are: setup.bash, setup.sh, setup.zsh.
cd ~/ros2_ws 
colcon build --packages-select determ_rviz_viewer --cmake-args -Dflowbit-server_DIR=/path/to/your/flowbit-server_DIR
# It is the directory where the unzip fold is.
# For example, if the fold download zip is unziped in /home/determ/Determ/Library, then 
# colcon build --packages-select determ_rviz_viewer --cmake-args -Dflowbit-server_DIR=/home/determ/Determ/Library/determ_flowbit_server/cmake

Usage

1. Launch the Viewer

# Source the workspace
source ~/ros2_ws/install/setup.bash

# Launch with default configuration
#The default configuration is in home/determ/ros2_ws/src/determ_rviz_viewer/launch/determ_manager.launch.py. User could modify the setting and rebuild the project with 
#colcon build --packages-select determ_rviz_viewer --cmake-args -Dflowbit-server_DIR=/path/to/your/flowbit-server_DIR

ros2 launch determ_rviz_viewer determ_manager.launch.py 

# Or specify a custom config path

# Source the workspace
source ~/ros2_ws/install/setup.bash

# Launch with default configuration
#The default configuration is in home/determ/ros2_ws/src/determ_rviz_viewer/launch/determ_manager.launch.py. User could modify the setting and rebuild the project with 
#colcon build --packages-select determ_rviz_viewer --cmake-args -Dflowbit-server_DIR=/path/to/your/flowbit-server_DIR

ros2 launch determ_rviz_viewer determ_manager.launch.py 

# Or specify a custom config path

# Source the workspace
source ~/ros2_ws/install/setup.bash

# Launch with default configuration
#The default configuration is in home/determ/ros2_ws/src/determ_rviz_viewer/launch/determ_manager.launch.py. User could modify the setting and rebuild the project with 
#colcon build --packages-select determ_rviz_viewer --cmake-args -Dflowbit-server_DIR=/path/to/your/flowbit-server_DIR

ros2 launch determ_rviz_viewer determ_manager.launch.py 

# Or specify a custom config path

2. Working with Determ Devices

Open another terminal and run the commands as follows:

# Source the Humble setup file
source /opt/ros/humble/setup.bash  
# Source the workspace
source ~/ros2_ws/install/setup.bash

# List Available Clients. 
ros2 service call /determ/list_clients std_srvs/srv/Trigger # The servive call will list all the clients in the ternimal lauching RViz window.

# Set the client ID parameter with the client you want to connect
ros2 param set /determ_manager selected_client_id "SZINDS-000000-XXXXXX-192168-150102"
# Connect to the client
ros2 service call /determ/connect_client std_srvs/srv/Trigger
# Start the capture
ros2 service call /determ/connect_capture_client std_srvs/srv/Trigger
# Stop the capture 
ros2 service call /determ/disconnect_capture_client std_srvs/srv/Trigger
#disconnect to the client
ros2 service call /determ/disconnect_client std_srvs/srv/Trigger
############################################################################
# Work on all clients
# Connect all clients available
ros2 service call /determ/connect_all_clients std_srvs/srv/Trigger
# Capture all clients connected
ros2 service call /determ/connect_all_capture_clients std_srvs/srv/Trigger
# Disconnect all clients
ros2 service call /determ/disconnect_all_clients std_srvs/srv/Trigger
# Check the topics list
ros2 topic list | grep

# Source the Humble setup file
source /opt/ros/humble/setup.bash  
# Source the workspace
source ~/ros2_ws/install/setup.bash

# List Available Clients. 
ros2 service call /determ/list_clients std_srvs/srv/Trigger # The servive call will list all the clients in the ternimal lauching RViz window.

# Set the client ID parameter with the client you want to connect
ros2 param set /determ_manager selected_client_id "SZINDS-000000-XXXXXX-192168-150102"
# Connect to the client
ros2 service call /determ/connect_client std_srvs/srv/Trigger
# Start the capture
ros2 service call /determ/connect_capture_client std_srvs/srv/Trigger
# Stop the capture 
ros2 service call /determ/disconnect_capture_client std_srvs/srv/Trigger
#disconnect to the client
ros2 service call /determ/disconnect_client std_srvs/srv/Trigger
############################################################################
# Work on all clients
# Connect all clients available
ros2 service call /determ/connect_all_clients std_srvs/srv/Trigger
# Capture all clients connected
ros2 service call /determ/connect_all_capture_clients std_srvs/srv/Trigger
# Disconnect all clients
ros2 service call /determ/disconnect_all_clients std_srvs/srv/Trigger
# Check the topics list
ros2 topic list | grep

# Source the Humble setup file
source /opt/ros/humble/setup.bash  
# Source the workspace
source ~/ros2_ws/install/setup.bash

# List Available Clients. 
ros2 service call /determ/list_clients std_srvs/srv/Trigger # The servive call will list all the clients in the ternimal lauching RViz window.

# Set the client ID parameter with the client you want to connect
ros2 param set /determ_manager selected_client_id "SZINDS-000000-XXXXXX-192168-150102"
# Connect to the client
ros2 service call /determ/connect_client std_srvs/srv/Trigger
# Start the capture
ros2 service call /determ/connect_capture_client std_srvs/srv/Trigger
# Stop the capture 
ros2 service call /determ/disconnect_capture_client std_srvs/srv/Trigger
#disconnect to the client
ros2 service call /determ/disconnect_client std_srvs/srv/Trigger
############################################################################
# Work on all clients
# Connect all clients available
ros2 service call /determ/connect_all_clients std_srvs/srv/Trigger
# Capture all clients connected
ros2 service call /determ/connect_all_capture_clients std_srvs/srv/Trigger
# Disconnect all clients
ros2 service call /determ/disconnect_all_clients std_srvs/srv/Trigger
# Check the topics list
ros2 topic list | grep

After connecting and capturing clients, you can manually add point cloud, color, and depth images in RViz 2, as shown in the screenshot below.

A Python script is provided to automatically load all connected and available clients:

cd ~/ros2_ws/src/determ_rviz_viewer
python3 generate_rviz_config.py

cd ~/ros2_ws/src/determ_rviz_viewer
python3 generate_rviz_config.py

cd ~/ros2_ws/src/determ_rviz_viewer
python3 generate_rviz_config.py

This will:

Call /determ/list_clients service
Parse all available client IDs
Generate RViz config with displays for ALL of them
Save to ~/.rviz2/determ_auto_generated.rviz

rviz2 -d ~/.rviz2/determ_auto_generated.rviz

rviz2 -d ~/.rviz2/determ_auto_generated.rviz

rviz2 -d ~/.rviz2/determ_auto_generated.rviz

All client point clouds will be visualized in RViz.

If your clients are calibrated, place the calibration JSON file in ~/ros2_ws/src/determ_rviz_viewer/config as calibration.json. An example calibration file is included in the folder. All extrinsic matrix parameters will be loaded for all clients.

When running the commands above, please review the outputs in both terminals to check the status and any errors. Security is built into every layer.

The Package

1. ROS 2 Topics

The package publishes the following topics:

Topic	Type	Description
`/determ/status`	`std_msgs/msg/String`	Status messages from the system
`/determ/<client_id>/pointcloud`	`sensor_msgs/msg/PointCloud2`	Point cloud data from a specific client
`/determ/<client_id>/rgb`	`sensor_msgs/msg/Image`	RGB image from a specific client
`/determ/<client_id>/depth`	`sensor_msgs/msg/Image`	Depth image from a specific client
`/determ/markers`	`visualization_msgs/msg/MarkerArray`	Markers for client visualization
`/determ/transforms`	`geometry_msgs/msg/TransformStamped`	Transform data for the frame hierarchy

2. ROS 2 Services

The package provides the following services:

Service	Type	Description
`/determ/list_clients`	`std_srvs/srv/Trigger`	List all available clients
`/determ/connect_client`	`std_srvs/srv/Trigger`	Connect to the selected client
`/determ/disconnect_client`	`std_srvs/srv/Trigger`	Disconnect the selected client
`/determ/connect_capture_client`	`std_srvs/srv/Trigger`	Start capture for the selected client
`/determ/disconnect_capture_client`	`std_srvs/srv/Trigger`	Stop capture for the selected client
`/determ/shutdown`	`std_srvs/srv/Trigger`	Shutdown the application
`/determ/connect_all_clients`	`std_srvs/srv/Trigger`	Connect all the client available
`/determ/connect_all_capture_clients`	`std_srvs/srv/Trigger`	Capture all the clients connected
`/determ/disconnect_all_clients`	`std_srvs/srv/Trigger`	Disconnect all the clients

3. Parameters

The package uses the following parameters:

Parameter	Type	Description
`config_path`	string	Path to the Determ configuration directory
`publish_pointclouds`	bool	Whether to publish point clouds as ROS messages
`publish_images`	bool	Whether to publish RGB and depth images as ROS messages
`update_rate`	double	Update rate in Hz for the main processing loop
`world_frame_id`	string	Frame ID for the world frame
`selected_client_id`	string	Client ID for client-specific operations

Support

For further assistance, please contact the Determ support team.

Flowbit Unreal Windows Usage Guide

On this page

Title