aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorJuneChul Roh2020-11-19 23:07:22 -0600
committerJuneChul Roh2020-11-19 23:07:22 -0600
commit0b0999bf73853b766d63c2120e02d743051960d3 (patch)
treedd13078b610441bad512f96c00ad04bdc4d2f28d
parentad09f354a81ebb395d7d34f8c73732ff5ced288d (diff)
downloadjacinto_ros_perception-0b0999bf73853b766d63c2120e02d743051960d3.tar.gz
jacinto_ros_perception-0b0999bf73853b766d63c2120e02d743051960d3.tar.xz
jacinto_ros_perception-0b0999bf73853b766d63c2120e02d743051960d3.zip
REL.00.01.00.02. Compatible with: J7 PSDK-RTOS 07_01_00_09, tidl-semseg-model_1.3.0.11.tar.gz, ros-bag_2020_0914.tar.gz
-rw-r--r--README.md44
-rw-r--r--cmake/common.cmake85
-rw-r--r--docker/Makefile50
-rw-r--r--docker/README.md150
-rw-r--r--docker/docs/tiovx_ros_demo_diagram.pngbin0 -> 52350 bytes
-rw-r--r--docker/docs/tiovx_ros_diagram.pngbin68419 -> 0 bytes
-rw-r--r--docker/docs/tiovx_ros_sw_stack.pngbin0 -> 36505 bytes
-rwxr-xr-xdocker/entrypoint.sh20
-rw-r--r--docker/setup_ti_processor_sdk.sh13
-rw-r--r--nodes/ti_sde/CMakeLists.txt21
-rw-r--r--nodes/ti_sde/README.md50
-rw-r--r--nodes/ti_sde/config/params.yaml5
-rw-r--r--nodes/ti_sde/launch/bag_sde.launch4
-rw-r--r--nodes/ti_sde/launch/rviz.launch (renamed from nodes/ti_viz_color_conversion/launch/yuv_to_rgb.launch)12
-rw-r--r--nodes/ti_sde/rviz/sde.rviz136
-rwxr-xr-xnodes/ti_sde/src/sde.h2
-rw-r--r--nodes/ti_semseg_cnn/CMakeLists.txt19
-rw-r--r--nodes/ti_semseg_cnn/README.md34
-rw-r--r--nodes/ti_semseg_cnn/launch/bag_semseg_cnn.launch2
-rw-r--r--nodes/ti_semseg_cnn/launch/rviz.launch18
-rw-r--r--nodes/ti_semseg_cnn/rviz/semseg_cnn.rviz123
-rw-r--r--nodes/ti_viz_color_conversion/CMakeLists.txt14
-rw-r--r--nodes/ti_viz_disparity/CMakeLists.txt15
-rw-r--r--nodes/ti_viz_disparity/README.md28
-rw-r--r--nodes/ti_viz_disparity/launch/viz_disparity.launch16
-rw-r--r--nodes/ti_viz_disparity/package.xml47
-rw-r--r--nodes/ti_viz_nodes/CMakeLists.txt51
-rw-r--r--nodes/ti_viz_nodes/README.md29
-rw-r--r--nodes/ti_viz_nodes/launch/viz_disparity.launch14
-rw-r--r--nodes/ti_viz_nodes/launch/viz_semseg.launch (renamed from nodes/ti_viz_semseg/launch/viz_semseg.launch)13
-rw-r--r--nodes/ti_viz_nodes/launch/yuv_to_rgb.launch15
-rw-r--r--nodes/ti_viz_nodes/package.xml (renamed from nodes/ti_viz_color_conversion/package.xml)7
-rw-r--r--nodes/ti_viz_nodes/src/ros_app_color_conversion_node.cpp (renamed from nodes/ti_viz_color_conversion/src/ros_app_color_conversion_node.cpp)0
-rw-r--r--nodes/ti_viz_nodes/src/viz_disparity_node.cpp (renamed from nodes/ti_viz_disparity/src/viz_disparity_node.cpp)0
-rw-r--r--nodes/ti_viz_nodes/src/viz_semseg_node.cpp (renamed from nodes/ti_viz_semseg/src/viz_semseg_node.cpp)14
-rw-r--r--nodes/ti_viz_semseg/CMakeLists.txt14
-rw-r--r--nodes/ti_viz_semseg/README.md29
-rw-r--r--nodes/ti_viz_semseg/package.xml47
-rwxr-xr-xsetup_env_pc.sh11
39 files changed, 716 insertions, 436 deletions
diff --git a/README.md b/README.md
index 3386e43..70cc275 100644
--- a/README.md
+++ b/README.md
@@ -1,16 +1,44 @@
1TI OpenVX + ROS Framework & Applications 1TI OpenVX + ROS Framework & Applications
2======================================== 2========================================
3 3
4## Introduction to TI OpenVX + ROS Framework 4### Introduction to TI OpenVX + ROS Development Framework
5[To be added]
6 5
7![TI OpenVX + ROS Example Diagram](docker/docs/tiovx_ros_diagram.png) 6<figure class="image">
7 <center><img src="docker/docs/tiovx_ros_sw_stack.png"/></center>
8 <figcaption> <center>Figure 1. TI OpenVX + ROS Framework: Software Stack </center></figcaption>
9</figure>
8 10
9## How to Set Up TI OpenVX + ROS Environment 11The TI OpenVX + ROS development framework is enabled in a Docker container environment on J7 Processor SDK Linux. We provide detailed steps for setting a Docker container environment for ROS Melodic together with the TI Vision Apps Library (see next section). The TI OpenVX + ROS development framework allows:
10See [this link](docker/README.md)
11 12
12## TI OpenVX + ROS Example Applications 13- Optimized software implementation of computation-intensive software blocks (including deep-learning, vision, perception, and ADAS) on deep-learning core (C7x/MMA), DSP cores, hardware accelerators built-in on the Jacinto 7 processor
14- Application softwares can be complied directly on the Jacinto 7 processor in a Docker container using APIs optimized on Jacinto 7 processor along with many open-source libraries and packages including, for example. OpenCV and Point-Cloud Library (PCL).
13 15
14### [Stereo Vision](nodes/ti_sde/README.md) 16Figure below is a representative vision application that can be developed in TI OpenVX + ROS framework.
15 17
16### [CNN Semantic Segmentation](nodes/ti_semseg_cnn/README.md) 18<figure class="image">
19 <center><img src="docker/docs/tiovx_ros_demo_diagram.png" style="width: 800px;"/></center>
20 <figcaption> <center>Figure 2. Example Application in TI OpenVX + ROS Framework </center></figcaption>
21</figure>
22
23### TI Vision Apps Library
24The TI Vision Apps Library is a set of APIs for the target deployment that are derived from the Jacinto 7 Processor SDK RTOS, which includes:
25- TI OpenVX kernels and infrastructure
26- TI deep learning (TIDL) applications
27- Imaging and vision applications
28- Advanced driver-assistance systems (ADAS) applications
29- Perception applications
30
31## How to Set Up TI OpenVX + ROS Docker Container Environment on J7 Target
32See [LINK](docker/README.md)
33
34## TI OpenVX + ROS Demo Applications
35
36### [Stereo Vision Processing Node Accelerated on LDC and SDE](nodes/ti_sde/README.md)
37
38### [CNN Semantic Segmentation Node with TIDL Running on C7x/MMA](nodes/ti_semseg_cnn/README.md)
39
40## Known Issues
41
421. Display is not enabled from a Docker container on J7.
432. The inference-time accuracy of `ti_semseg_cnn` semantic segmentation CNN is not great. This is expected since the CNN network was trained with Cityscapes dataset, and was not re-trained to further optimize the CNN model on the camera data that played back from the ROSBAG file.
443. Ctrl+C terminal of a ROS node or a ROS launch session may be slow and sometimes does not fully clean up OpenVX part of implementation, causing VX_ERROR in the next launching. When this happens, it is recommended to reboot the J7 EVM. \ No newline at end of file
diff --git a/cmake/common.cmake b/cmake/common.cmake
index 568812d..31cff81 100644
--- a/cmake/common.cmake
+++ b/cmake/common.cmake
@@ -11,16 +11,18 @@ catkin_package(CATKIN_DEPENDS nodelet roscpp cv_bridge image_transport sensor_ms
11 11
12find_package(OpenCV REQUIRED) 12find_package(OpenCV REQUIRED)
13 13
14include_directories(include) 14include_directories(
15include_directories(${catkin_INCLUDE_DIRS}) 15 include
16include_directories(${OpenCV_INCLUDE_DIRS}) 16 ${catkin_INCLUDE_DIRS}
17 ${OpenCV_INCLUDE_DIRS}
18)
17 19
18# PSDKRA base folder location 20# PSDKRA base folder location
19set(PSDK_DIR $ENV{PSDK_BASE_PATH}) 21set(PSDK_DIR $ENV{PSDK_BASE_PATH})
20 22
21set(CGT7X_ROOT ti-cgt-c7000_1.4.0.LTS) 23set(CGT7X_ROOT ti-cgt-c7000_1.4.0.LTS)
22set(PDK_PACKAGE_ROOT pdk/packages/ti) 24set(PDK_PACKAGE_ROOT pdk/packages/ti)
23set(TIDL_PACKAGE_ROOT tidl_j7_01_03_00_07/ti_dl) 25set(TIDL_PACKAGE_ROOT tidl_j7_01_03_00_11/ti_dl)
24set(MMALIB_PACKAGE_ROOT mmalib_01_03_00_06) 26set(MMALIB_PACKAGE_ROOT mmalib_01_03_00_06)
25set(TIADALG_PATH tiadalg) 27set(TIADALG_PATH tiadalg)
26 28
@@ -63,33 +65,31 @@ unset(CMAKE_BUILD_TYPE CACHE)
63 65
64# pass the macros 66# pass the macros
65if(${TARGET_PLATFORM} STREQUAL "PC") 67if(${TARGET_PLATFORM} STREQUAL "PC")
66 add_definitions(-DTARGET_CPU=${TARGET_CPU} 68 add_definitions(
67 -DTARGET_OS=${TARGET_OS} 69 -DTARGET_CPU=${TARGET_CPU}
68 -DPROFILE=${PROFILE} 70 -DTARGET_OS=${TARGET_OS}
69 -DBUILD_EMULATION_MODE=${BUILD_EMULATION_MODE} 71 -DPROFILE=${PROFILE}
70 -DPC 72 -DBUILD_EMULATION_MODE=${BUILD_EMULATION_MODE}
71 # -DTARGET_CPU_PC=true 73 -DPC
72 ) 74 )
73else() # J7 75else() # J7
74 add_definitions(-DTARGET_CPU=${TARGET_CPU} 76 add_definitions(
75 -DTARGET_OS=${TARGET_OS} 77 -DTARGET_CPU=${TARGET_CPU}
76 -DPROFILE=${PROFILE} 78 -DTARGET_OS=${TARGET_OS}
77 -DBUILD_EMULATION_MODE=${BUILD_EMULATION_MODE} 79 -DPROFILE=${PROFILE}
80 -DBUILD_EMULATION_MODE=${BUILD_EMULATION_MODE}
78 ) 81 )
79endif() 82endif()
80 83
81# TIOVX, VISION_APPS, PTK_DEMOS: include folders 84# TIOVX, VISION_APPS, PTK_DEMOS: include folders
82set(TIOVX_INCLUDE_DIRS 85set(TIOVX_INCLUDE_DIRS
83 ${PSDK_DIR}/tiovx/include 86 ${PSDK_DIR}/tiovx/include
87 ${PSDK_DIR}/tiovx/kernels_j7/include
84 ${PSDK_DIR}/tiovx/tiovx_dev/kernels_j7/include 88 ${PSDK_DIR}/tiovx/tiovx_dev/kernels_j7/include
85 ${PSDK_DIR}/tiovx/utils/include/ 89 ${PSDK_DIR}/tiovx/utils/include/
86 ${PSDK_DIR}/tiovx/kernels/include/ 90 ${PSDK_DIR}/tiovx/kernels/include/
87) 91)
88 92
89# set(PDK_INCLUDE_DIRS
90# ${PSDK_DIR}/${PDK_PACKAGE_ROOT}/drv/udma
91# )
92
93set(PTK_INCLUDE_DIRS 93set(PTK_INCLUDE_DIRS
94 ${PSDK_DIR}/perception/include 94 ${PSDK_DIR}/perception/include
95) 95)
@@ -133,14 +133,15 @@ set(OTHER_INCLUDE_DIRS
133 ${PSDK_DIR}/ivision 133 ${PSDK_DIR}/ivision
134) 134)
135 135
136include_directories(${TIOVX_INCLUDE_DIRS} 136include_directories(
137 ${PTK_INCLUDE_DIRS} 137 ${TIOVX_INCLUDE_DIRS}
138 ${VISION_APP_INCLUDE_DIRS} 138 ${PTK_INCLUDE_DIRS}
139 ${VISION_APP_KERNEL_INCLUDE_DIRS} 139 ${VISION_APP_INCLUDE_DIRS}
140 ${MMALIB_INCLUDE_DIRS} 140 ${VISION_APP_KERNEL_INCLUDE_DIRS}
141 ${TIDL_INCLUDE_DIRS} 141 ${MMALIB_INCLUDE_DIRS}
142 ${TIADALG_INCLUDE_DIRS} 142 ${TIDL_INCLUDE_DIRS}
143 ${OTHER_INCLUDE_DIRS} 143 ${TIADALG_INCLUDE_DIRS}
144 ${OTHER_INCLUDE_DIRS}
144) 145)
145 146
146# TIOVX, VISION_APPS, PTK_DEMOS: define lib variables 147# TIOVX, VISION_APPS, PTK_DEMOS: define lib variables
@@ -191,20 +192,22 @@ else() # static
191 ${PTHREAD_LIB} 192 ${PTHREAD_LIB}
192 ${ION_LIB} # only for J7 193 ${ION_LIB} # only for J7
193 ${CMAKE_DL_LIBS} 194 ${CMAKE_DL_LIBS}
194 -Wl,--end-group) 195 -Wl,--end-group
195 196 )
196 link_directories(${TIOVX_LIBS_DIR} 197
197 ${VISION_APPS_LIBS_DIR} 198 link_directories(
198 ${PDK_LIBS_DIR} 199 ${TIOVX_LIBS_DIR}
199 ${PTK_LIBS_DIR} 200 ${VISION_APPS_LIBS_DIR}
200 ${IMAGING_LIBS_DIR} 201 ${PDK_LIBS_DIR}
201 ${CGT7X_LIBS_DIR} 202 ${PTK_LIBS_DIR}
202 ${TIDL_LIBS_DIR} 203 ${IMAGING_LIBS_DIR}
203 ${MMA_LIBS_DIR} 204 ${CGT7X_LIBS_DIR}
204 ${TIADALG_LIBS_DIR} 205 ${TIDL_LIBS_DIR}
205 ${J7_CMODEL_LIBS_DIR} 206 ${MMA_LIBS_DIR}
206 ${PTHREAD_LIB_DIR} 207 ${TIADALG_LIBS_DIR}
207 ${ION_LIB_DIR} # only for J7 208 ${J7_CMODEL_LIBS_DIR}
208 ) 209 ${PTHREAD_LIB_DIR}
210 ${ION_LIB_DIR} # only for J7
211 )
209endif() 212endif()
210 213
diff --git a/docker/Makefile b/docker/Makefile
index 9eb12f4..465e84f 100644
--- a/docker/Makefile
+++ b/docker/Makefile
@@ -1,13 +1,15 @@
1#!make 1#!make
2# Dependency: J7 Processor SDK-RTOS 07_01_00_11
3
2## Independent variables 4## Independent variables
3# Dockerfile 5# Dockerfile
4DOCKERFILE_NAME = Dockerfile 6DOCKERFILE_NAME = Dockerfile
5# Docker image tag 7# Docker image tag
6DOCKER_TAG = j7ros 8DOCKER_TAG = j7ros
7# TI Vision Apps Lib version 9# TI Vision Apps Lib version
8LIB_VER = 7.1.0-r0.0 10TIVA_LIB_VER = 7.1.0
9# TIDL version 11# TIDL version
10TIDL_VER = 1.3.0.7 12TIDL_VER = 1.3.0.11
11# ROS bag date 13# ROS bag date
12ROSBAG_DATE = 2020_0914 14ROSBAG_DATE = 2020_0914
13 15
@@ -27,36 +29,20 @@ SCRIPTS_DIR = $(REPO_PATH)/docker/scripts
27ENV_FILE = env_list.txt 29ENV_FILE = env_list.txt
28ENV_FILE_PATH = $(DOCKER_DIR)/$(ENV_FILE) 30ENV_FILE_PATH = $(DOCKER_DIR)/$(ENV_FILE)
29 31
30# Tarball URLs: tentative 32# Tarball URLs
31OPKG_REPO_URL = https://txn.box.com/shared/static/uav8998htorweblm6se4o0uveepejymr 33TIDL_MODEL_URL = https://tidrive.ext.ti.com/fss/public/link/public/stream/read/tidl-semseg-model_1.3.0.11.tar.gz?linkToken=xeW1RA_ltU_X5bVF&itemName=aabc76cb-ae69-451c-97b9-11bf0b959563
32TIDL_MODEL_URL = https://txn.box.com/shared/static/2u7ypmbvy9mlk1hq1wq9ym88edk7rk9b 34ROSBAG_URL = https://tidrive.ext.ti.com/fss/public/link/public/stream/read/ros-bag_2020_0914.tar.gz?linkToken=rRCsIGcQrCe_EKwh&itemName=3b002324-6c8b-43b4-b4b3-c797d3852ff3
33ROSBAG_URL = https://txn.box.com/shared/static/oyz3syp7up8rtfhif3akxiw1tac065ys
34 35
35# Tarball filenames 36# Tarball filenames
36OPKG_REPO_FILE = opkg-repo_$(LIB_VER).tar.gz
37TIDL_MODEL_FILE = tidl-semseg-model_$(TIDL_VER).tar.gz 37TIDL_MODEL_FILE = tidl-semseg-model_$(TIDL_VER).tar.gz
38ROSBAG_FILE = ros-bag_$(ROSBAG_DATE).tar.gz 38ROSBAG_FILE = ros-bag_$(ROSBAG_DATE).tar.gz
39 39
40# IP address 40# IP address
41J7_IP_ADDR := $(shell ifconfig | grep -A 1 'eth0' | tail -1 | cut -d ':' -f 2 | cut -d ' ' -f 1) 41J7_IP_ADDR := $(shell ifconfig | grep -A 1 'eth0' | tail -1 | cut -d ':' -f 2 | cut -d ' ' -f 1)
42 42
43## Set up environments on J7 host linux
44host_setup: opkg_config opkg_repo_download ipk_install
45
46opkg_config:
47 $(SCRIPTS_DIR)/config_opkg_repo.sh
48
49ipk_install:
50 $(SCRIPTS_DIR)/install_tivision_apps_ipk.sh
51
52ip_show: 43ip_show:
53 @echo "$(J7_IP_ADDR)" 44 @echo "$(J7_IP_ADDR)"
54 45
55opkg_repo_download: check_data_dir
56 wget -O $(DATA_DIR)/$(OPKG_REPO_FILE) $(OPKG_REPO_URL)
57 rm -rf $(OPKG_REPO_DIR)
58 tar xzf $(DATA_DIR)/$(OPKG_REPO_FILE) -C /
59
60## Docker 46## Docker
61scripts: docker_build_sh docker_run_sh 47scripts: docker_build_sh docker_run_sh
62 48
@@ -67,7 +53,6 @@ docker_build_sh:
67 @echo "cd $(WORK_PATH_HOST)" >> $(DOCKER_BUILD_SH) 53 @echo "cd $(WORK_PATH_HOST)" >> $(DOCKER_BUILD_SH)
68 @echo "docker build -t \$$DOCKER_TAG -f \$$DOCKER_DIR/$(DOCKERFILE_NAME) ." >> $(DOCKER_BUILD_SH) 54 @echo "docker build -t \$$DOCKER_TAG -f \$$DOCKER_DIR/$(DOCKERFILE_NAME) ." >> $(DOCKER_BUILD_SH)
69 @chmod +x $(DOCKER_BUILD_SH) 55 @chmod +x $(DOCKER_BUILD_SH)
70 @cp $(DOCKER_BUILD_SH) $(DOCKER_DIR)/$(DOCKER_BUILD_SH)
71 @echo "$(DOCKER_BUILD_SH) is generated" 56 @echo "$(DOCKER_BUILD_SH) is generated"
72 57
73docker_run_sh: 58docker_run_sh:
@@ -75,6 +60,7 @@ docker_run_sh:
75 @echo "DOCKER_TAG=$(DOCKER_TAG)" >> $(DOCKER_RUN_SH) 60 @echo "DOCKER_TAG=$(DOCKER_TAG)" >> $(DOCKER_RUN_SH)
76 @echo "DOCKER_DIR=$(DOCKER_DIR)" >> $(DOCKER_RUN_SH) 61 @echo "DOCKER_DIR=$(DOCKER_DIR)" >> $(DOCKER_RUN_SH)
77 @echo "IP_ADDR=\$$(ifconfig | grep -A 1 'eth0' | tail -1 | cut -d ':' -f 2 | cut -d ' ' -f 1)" >> $(DOCKER_RUN_SH) 62 @echo "IP_ADDR=\$$(ifconfig | grep -A 1 'eth0' | tail -1 | cut -d ':' -f 2 | cut -d ' ' -f 1)" >> $(DOCKER_RUN_SH)
63 @echo "TIVA_LIB_VER=$(TIVA_LIB_VER)" >> $(DOCKER_RUN_SH)
78 @echo "if [ \"\$$#\" -lt 1 ]; then" >> $(DOCKER_RUN_SH) 64 @echo "if [ \"\$$#\" -lt 1 ]; then" >> $(DOCKER_RUN_SH)
79 @echo " COMMAND=/bin/bash" >> $(DOCKER_RUN_SH) 65 @echo " COMMAND=/bin/bash" >> $(DOCKER_RUN_SH)
80 @echo "else" >> $(DOCKER_RUN_SH) 66 @echo "else" >> $(DOCKER_RUN_SH)
@@ -88,42 +74,38 @@ docker_run_sh:
88 @echo " --network host \\" >> $(DOCKER_RUN_SH) 74 @echo " --network host \\" >> $(DOCKER_RUN_SH)
89 @echo " --device-cgroup-rule='c 238:* rmw' \\" >> $(DOCKER_RUN_SH) 75 @echo " --device-cgroup-rule='c 238:* rmw' \\" >> $(DOCKER_RUN_SH)
90 @echo " --env J7_IP_ADDR=\$$IP_ADDR \\" >> $(DOCKER_RUN_SH) 76 @echo " --env J7_IP_ADDR=\$$IP_ADDR \\" >> $(DOCKER_RUN_SH)
77 @echo " --env TIVA_LIB_VER=\$$TIVA_LIB_VER \\" >> $(DOCKER_RUN_SH)
91 @echo " --env-file \$$DOCKER_DIR/$(ENV_FILE) \\" >> $(DOCKER_RUN_SH) 78 @echo " --env-file \$$DOCKER_DIR/$(ENV_FILE) \\" >> $(DOCKER_RUN_SH)
92 @echo " \$$DOCKER_TAG \$$COMMAND" >> $(DOCKER_RUN_SH) 79 @echo " \$$DOCKER_TAG \$$COMMAND" >> $(DOCKER_RUN_SH)
93 @chmod +x $(DOCKER_RUN_SH) 80 @chmod +x $(DOCKER_RUN_SH)
94 @cp $(DOCKER_RUN_SH) $(DOCKER_DIR)/$(DOCKER_RUN_SH)
95 @echo "$(DOCKER_RUN_SH) is generated" 81 @echo "$(DOCKER_RUN_SH) is generated"
96 82
97docker_build:
98 ./docker_build.sh
99
100docker_run:
101 ./docker_run.sh
102
103## data files 83## data files
104data_download: tidl_net_download rosbag_download 84data_download: tidl_net_download rosbag_download
105 85
106tidl_net_download: check_data_dir 86tidl_net_download: check_data_dir
107 wget -O $(DATA_DIR)/$(TIDL_MODEL_FILE) $(TIDL_MODEL_URL) 87 wget -O $(DATA_DIR)/$(TIDL_MODEL_FILE) "$(TIDL_MODEL_URL)"
108 rm -rf $(DATA_DIR)/tidl_semseg_model 88 rm -rf $(DATA_DIR)/tidl_semseg_model
109 tar xzf $(DATA_DIR)/$(TIDL_MODEL_FILE) -C $(DATA_DIR) 89 tar xzf $(DATA_DIR)/$(TIDL_MODEL_FILE) -C $(DATA_DIR)
90 rm -f $(DATA_DIR)/$(TIDL_MODEL_FILE)
110 91
111rosbag_download: check_data_dir 92rosbag_download: check_data_dir
112 wget -O $(DATA_DIR)/$(ROSBAG_FILE) $(ROSBAG_URL) 93 wget -O $(DATA_DIR)/$(ROSBAG_FILE) "$(ROSBAG_URL)"
113 rm -rf $(DATA_DIR)/ros_bag 94 rm -rf $(DATA_DIR)/ros_bag
114 tar xzf $(DATA_DIR)/$(ROSBAG_FILE) -C $(DATA_DIR) 95 tar xzf $(DATA_DIR)/$(ROSBAG_FILE) -C $(DATA_DIR)
96 rm -f $(DATA_DIR)/$(ROSBAG_FILE)
115 97
116check_data_dir: 98check_data_dir:
117 @if [ ! -d $(DATA_DIR) ]; then \ 99 @if [ ! -d $(DATA_DIR) ]; then \
118 mkdir -p $(DATA_DIR); \ 100 mkdir -p $(DATA_DIR); \
119 echo "mkdir -p $(DATA_DIR)"; \ 101 echo "mkdir -p $(DATA_DIR)"; \
120 else \ 102 else \
121 echo "Ok,.$(DATA_DIR) exists"; \ 103 echo "Ok, $(DATA_DIR) exists"; \
122 fi 104 fi
123 105
124data_clean: 106data_clean:
125 rm -rf $(DATA_DIR) 107 rm -rf $(DATA_DIR)
126 108
127## .PHONY 109## .PHONY
128.PHONY: docker_build_sh docker_run_sh opkg_config ipk_install ip_show \ 110.PHONY: docker_build_sh docker_run_sh ip_show \
129 opkg_repo_download tidl_net_download rosbag_download check_data_dir 111 tidl_net_download rosbag_download check_data_dir
diff --git a/docker/README.md b/docker/README.md
index c60e16d..b0b73e9 100644
--- a/docker/README.md
+++ b/docker/README.md
@@ -1,30 +1,43 @@
1How to Set Up TI OpenVX + ROS Environment 1How to Set Up TI OpenVX + ROS Environment
2========================================= 2=========================================
3 3
4## Prerequisites 4## Prerequisites & Dependency
5 5
6### Hardward: J7/TDA4x Family Processors 6### Hardward: J7/TDA4x Family Processors
7https://www.ti.com/processors/automotive-processors/featured-platform.html 7https://www.ti.com/processors/automotive-processors/featured-platform.html
8 8
9### J7 Host Linux: Jacinto Processor SDK Linux Version 7.1.0 or Later 9### J7 Processor SDK RTOS Version 7.1.0
10https://software-dl.ti.com/jacinto7/esd/processor-sdk-linux-jacinto7/latest/exports/docs/linux/index.html 10Currently testing with Prebuilt Package of J7 Processor SDK RTOS 7.1.0 RC9 [LINK](http://gtweb.dal.design.ti.com/nightly_builds/PSDKRA_INSTALLER/57-2020-11-19_11-41-23/artifacts/output/webgen/publish/PROCESSOR-SDK-RTOS-J721E/07_01_00_09/)
11
12Download [Pre-built Package](http://gtweb.dal.design.ti.com/nightly_builds/PSDKRA_INSTALLER/57-2020-11-19_11-41-23/artifacts/output/webgen/publish/PROCESSOR-SDK-RTOS-J721E/07_01_00_09/exports/ti-processor-sdk-rtos-j721e-evm-07_01_00_09-prebuilt.tar.gz) and instrall to a SD card by referring to the instruction on this page [LINK](http://gtweb.dal.design.ti.com/nightly_builds/PSDKRA_INSTALLER/57-2020-11-19_11-41-23/artifacts/output/webgen/publish/PROCESSOR-SDK-RTOS-J721E/07_01_00_09/exports/docs/psdk_rtos/docs/user_guide/out_of_box_j721e.html)
13
14### Terminal Session to J7
15
16Once finding the IP address assigned to J7 EVM (e.g., using a serial port communications program, for example, `minicom`), connect to J7 Linux with SSH:
17
18```
19ssh root@<J7_IP_address>
20```
21
22**Note**: It is highly recommended to use a "static IP" for the J7 EVM to make ROS network setting easy.
11 23
12Currently testing with pre-lease version of PSDKLA 7.1.0
13 24
14<!-- ================================================================================= --> 25<!-- ================================================================================= -->
15## Clone Git Repository 26## Clone Git Repository
27
16```sh 28```sh
17# Working directory 29# project home directory
18WORK_DIR=$HOME/j7ros_home 30WORK_DIR=$HOME/j7ros_home
19# catkin workspace 31# catkin workspace
20CATKIN_WS=$WORK_DIR/catkin_ws 32CATKIN_WS=$WORK_DIR/catkin_ws
21
22mkdir -p $CATKIN_WS/src 33mkdir -p $CATKIN_WS/src
23cd $CATKIN_WS/src 34cd $CATKIN_WS/src
24 35
25# git clone 36# clone the project GIT repository
26# Not yet active >>> TODO 37# Below is accessible only in TI network:
27git clone https://git.ti.com/git/processor-sdk-vision/jacinto_ros_perception.git jacinto_ros_perception 38git clone ssh://git@bitbucket.itg.ti.com/processor-sdk-vision/jacinto_ros_perception.git
39# To be be available on the following GIT repository:
40# git clone https://git.ti.com/git/processor-sdk-vision/jacinto_ros_perception.git
28``` 41```
29 42
30## Configure J7 Host Linux & Install TI Vision Apps Library 43## Configure J7 Host Linux & Install TI Vision Apps Library
@@ -32,32 +45,17 @@ git clone https://git.ti.com/git/processor-sdk-vision/jacinto_ros_perception.git
32For convenience, set up following soft-links: 45For convenience, set up following soft-links:
33```sh 46```sh
34cd $WORK_DIR 47cd $WORK_DIR
35ln -s $CATKIN_WS/src/jacinto_ros_perception/docker 48ln -s $CATKIN_WS/src/jacinto_ros_perception/docker/Makefile
36ln -s docker/Makefile 49ln -s docker/Makefile
37``` 50```
38 51
39### Configure Host Linux
40For configuring the host Linux, run the following in `$WORK_DIR`:
41```sh
42make opkg_config
43```
44
45### Install TI Vision Apps Library
46
47To install the TI Vision Apps Library, run the following in `$WORK_DIR`:
48```sh
49make opkg_repo_download
50make ipk_install
51```
52
53<!-- ================================================================================= --> 52<!-- ================================================================================= -->
54## Build Docker Image 53## Build Docker Image
55 54
56The IP address that is assigned to the J7 host Linux will be required in setting network for the ROS applications. The IP address is automatically parsed in `Makefile`, to check: 55The IP address that is assigned to the J7 host Linux will be required in setting network for the ROS applications. The IP address is automatically parsed in `Makefile`. To check the IP address:
57``` 56```
58make ip_show 57make ip_show
59``` 58```
60**Note**: It is highly recommended to use a "static IP" for the J7 EVM to make ROS network setting easy.
61 59
62### Generate Scripts 60### Generate Scripts
63To generate bash scripts for building and running a Docker image for the project: 61To generate bash scripts for building and running a Docker image for the project:
@@ -97,6 +95,8 @@ j7ros latest 34ef46e34368 2 minutes ago
97arm64v8/ubuntu 18.04 0ccb47f043f5 3 months ago 57.8MB 95arm64v8/ubuntu 18.04 0ccb47f043f5 3 months ago 57.8MB
98``` 96```
99 97
98**Note**: The Docker image that is built using `Makefile` provided in the GIT repository will include minimal number of ROS packages on which the ROS package(s) under `$CATKIN_WS/src` have dependency. In case when more ROS package(s) are added under the folder, it is required to re-build a Docker image using the `Makefile`.
99
100<!-- ================================================================================= --> 100<!-- ================================================================================= -->
101## Run Docker Image 101## Run Docker Image
102To run the docker image: 102To run the docker image:
@@ -104,40 +104,62 @@ To run the docker image:
104./docker_run.sh 104./docker_run.sh
105``` 105```
106 106
107For reference, below shows `docker_run.sh`: 107To exit from inside the Docker container, type `exit` at the command line.
108
109<!-- ================================================================================= -->
110## Setting Up Remote PC for Visualization
111
112Open another terminal on Ubuntu PC to set up environments for visualization.
113
114### Clone GIT repository
108```sh 115```sh
109root@j7-evm:~/j7ros_home# cat docker_run.sh 116# catkin workspace
110#!/bin/bash 117CATKIN_WS=$HOME/j7ros_home/catkin_ws
111DOCKER_TAG=j7ros
112DOCKER_DIR=/home/root/j7ros_home/catkin_ws/src/jacinto_ros_perception/docker
113IP_ADDR=$(ifconfig | grep -A 1 'eth0' | tail -1 | cut -d ':' -f 2 | cut -d ' ' -f 1)
114if [ "$#" -lt 1 ]; then
115 COMMAND=/bin/bash
116else
117 COMMAND="$@"
118fi
119docker run -it --rm \
120 -v /home/root/j7ros_home:/root/j7ros_home \
121 -v /usr:/host/usr:ro \
122 -v /dev:/dev \
123 --privileged \
124 --network host \
125 --device-cgroup-rule='c 238:* rmw' \
126 --env J7_IP_ADDR=$IP_ADDR \
127 --env-file $DOCKER_DIR/env_list.txt \
128 $DOCKER_TAG $COMMAND
129```
130
131Some explanation:
132
133* `-v /home/root/j7ros_home:/root/j7ros_home`: `$WORK_DIR` is volume-mapped to `/root/j7ros_home` in the Docker container, under the folder all the source codes for the project will be available. Therefore, any changes made to `$WORK_DIR` will be accessible both inside the container or on the host Linux
134
135* `--env J7_IP_ADDR=$IP_ADDR`: IP address is passed as an environment variable `J7_IP_ADDR` to the container.
136
137* `--env-file $DOCKER_DIR/env_list.txt`: pass other environment variables that are specified in `$DOCKER_DIR/env_list.txt` for the ROS applications are passed to the container.
138 118
139To exit from inside the Docker container, type `exit` at the command line. 119mkdir -p $CATKIN_WS/src
120cd $CATKIN_WS/src
140 121
122# git clone
123# Below is accessible only in TI network:
124git clone ssh://git@bitbucket.itg.ti.com/processor-sdk-vision/jacinto_ros_perception.git
125# To be be available on the following GIT repository:
126# git clone https://git.ti.com/git/processor-sdk-vision/jacinto_ros_perception.git
127```
128
129### Build ROS nodes
130```
131cd $CATKIN_WS
132catkin_make
133```
134
135### PC Environment Setting
136For convenience, set up following soft-links:
137```sh
138cd $CATKIN_WS
139ln -s src/jacinto_ros_perception/setup_env_pc.sh
140```
141
142Update the following lines in `setup_env_pc.sh` for setting the ROS network:
143```
144PC_IP_ADDR=<PC_IP_address>
145J7_IP_ADRR=<J7_IP_address>
146```
147
148`<J7_IP_address>` can be found by running the following on the J7 terminal:
149```
150make ip_show
151```
152
153To set up the PC environment, run the following:
154```
155source setup_env_pc.sh
156```
157
158### Checking ROS topics
159After launching ROS nodes on the J7, when the network is set correctly, we can check the all the topics published with:
160```
161rostopic list
162```
141 163
142<!-- ================================================================================= --> 164<!-- ================================================================================= -->
143## ROS Applications 165## ROS Applications
@@ -151,9 +173,6 @@ Several tarballs are downloaded and uncompressed under `$WORK_DIR/data`, with fo
151 173
152``` 174```
153$WORK_DIR/data 175$WORK_DIR/data
154├── opkg-repo_7.1.0-r0.0.tar.gz
155├── ros-bag_2020_0914.tar.gz
156├── tidl-semseg-model_1.3.0.0.tar.gz
157├── ros_bag 176├── ros_bag
158│   └── 2020-09-14-12-31-43.bag 177│   └── 2020-09-14-12-31-43.bag
159└── tidl_semseg_model 178└── tidl_semseg_model
@@ -181,6 +200,11 @@ Alternatively, you can run the following `roslaunch` command **inside** the Dock
181roslaunch ti_sde bag_sde.launch 200roslaunch ti_sde bag_sde.launch
182``` 201```
183 202
203For visualization, on the PC (see the above for setting PC environment):
204```
205roslaunch ti_sde rviz.launch
206```
207
184### Run Semantic Segmentation CNN App 208### Run Semantic Segmentation CNN App
185To launch `ti_semseg_cnn` along with `rosbag play <ROSBAG>`, run the following in `$WORK_DIR` on the J7 host Linux: 209To launch `ti_semseg_cnn` along with `rosbag play <ROSBAG>`, run the following in `$WORK_DIR` on the J7 host Linux:
186```sh 210```sh
@@ -191,3 +215,9 @@ Alternatively, you can run the following `roslaunch` command **inside** the Dock
191```sh 215```sh
192roslaunch ti_semseg_cnn bag_semseg_cnn.launch 216roslaunch ti_semseg_cnn bag_semseg_cnn.launch
193``` 217```
218
219For visualization, on the PC (see the above for setting PC environment):
220```
221roslaunch ti_semseg_cnn rviz.launch
222```
223
diff --git a/docker/docs/tiovx_ros_demo_diagram.png b/docker/docs/tiovx_ros_demo_diagram.png
new file mode 100644
index 0000000..b4618db
--- /dev/null
+++ b/docker/docs/tiovx_ros_demo_diagram.png
Binary files differ
diff --git a/docker/docs/tiovx_ros_diagram.png b/docker/docs/tiovx_ros_diagram.png
deleted file mode 100644
index b67aca9..0000000
--- a/docker/docs/tiovx_ros_diagram.png
+++ /dev/null
Binary files differ
diff --git a/docker/docs/tiovx_ros_sw_stack.png b/docker/docs/tiovx_ros_sw_stack.png
new file mode 100644
index 0000000..ac014a3
--- /dev/null
+++ b/docker/docs/tiovx_ros_sw_stack.png
Binary files differ
diff --git a/docker/entrypoint.sh b/docker/entrypoint.sh
index 4dcb86e..7052217 100755
--- a/docker/entrypoint.sh
+++ b/docker/entrypoint.sh
@@ -4,18 +4,14 @@ set -e
4# set up ROS environment 4# set up ROS environment
5source "/opt/ros/$ROS_DISTRO/setup.bash" 5source "/opt/ros/$ROS_DISTRO/setup.bash"
6 6
7# set up TI Processor SDK environment 7# setup TI Processor SDK environment
8ln -s /host/usr/lib/libtivision_apps.so.7.1.0 /usr/lib/libtivision_apps.so.7.1.0 8SRC_PATH=$WORK_DIR/catkin_ws/src/jacinto_ros_perception/docker
9ln -s /usr/lib/libtivision_apps.so.7.1.0 /usr/lib/libtivision_apps.so 9FILE="$SRC_PATH/setup_ti_processor_sdk.sh"
10ln -s /host/usr/include/processor_sdk /usr/include/processor_sdk 10if [ -f $FILE ]; then
11ln -s /host/usr/lib/libion.so.1.0.0 /usr/lib/libion.so.1.0.0 11 source $FILE
12ln -s /usr/lib/libion.so.1.0.0 /usr/lib/libion.so 12else
13ln -s /host/usr/lib/libgbm.so.1.0.0 /usr/lib/libgbm.so.1.0.0 13 echo "$FILE does not exist"
14ln -s /usr/lib/libgbm.so.1.0.0 /usr/lib/libgbm.so.1 14fi
15ln -s /usr/lib/libgbm.so.1.0.0 /usr/lib/libgbm.so
16ln -s /host/usr/lib/libti_rpmsg_char.so.0.1.0 /usr/lib/libti_rpmsg_char.so.0.1.0
17ln -s /usr/lib/libti_rpmsg_char.so.0.1.0 /usr/lib/libti_rpmsg_char.so.0
18ln -s /usr/lib/libti_rpmsg_char.so.0.1.0 /usr/lib/libti_rpmsg_char.so
19 15
20# ROS network settings 16# ROS network settings
21export ROS_MASTER_URI=http://$J7_IP_ADDR:11311 17export ROS_MASTER_URI=http://$J7_IP_ADDR:11311
diff --git a/docker/setup_ti_processor_sdk.sh b/docker/setup_ti_processor_sdk.sh
new file mode 100644
index 0000000..90693a8
--- /dev/null
+++ b/docker/setup_ti_processor_sdk.sh
@@ -0,0 +1,13 @@
1#!/bin/bash
2# set up TI Processor SDK environment
3ln -s /host/usr/lib/libtivision_apps.so.$TIVA_LIB_VER /usr/lib/libtivision_apps.so.$TIVA_LIB_VER
4ln -s /usr/lib/libtivision_apps.so.$TIVA_LIB_VER /usr/lib/libtivision_apps.so
5ln -s /host/usr/include/processor_sdk /usr/include/processor_sdk
6ln -s /host/usr/lib/libion.so.1.0.0 /usr/lib/libion.so.1.0.0
7ln -s /usr/lib/libion.so.1.0.0 /usr/lib/libion.so
8ln -s /host/usr/lib/libgbm.so.1.0.0 /usr/lib/libgbm.so.1.0.0
9ln -s /usr/lib/libgbm.so.1.0.0 /usr/lib/libgbm.so.1
10ln -s /usr/lib/libgbm.so.1.0.0 /usr/lib/libgbm.so
11ln -s /host/usr/lib/libti_rpmsg_char.so.0.2.0 /usr/lib/libti_rpmsg_char.so.0.2.0
12ln -s /usr/lib/libti_rpmsg_char.so.0.2.0 /usr/lib/libti_rpmsg_char.so.0
13ln -s /usr/lib/libti_rpmsg_char.so.0.2.0 /usr/lib/libti_rpmsg_char.so
diff --git a/nodes/ti_sde/CMakeLists.txt b/nodes/ti_sde/CMakeLists.txt
index c49906e..ede37be 100644
--- a/nodes/ti_sde/CMakeLists.txt
+++ b/nodes/ti_sde/CMakeLists.txt
@@ -1,14 +1,17 @@
1cmake_minimum_required(VERSION 3.0.2) 1cmake_minimum_required(VERSION 3.0.2)
2project(ti_sde) 2project(ti_sde)
3 3
4include(${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/common.cmake) 4# Build only on J7
5if(${CMAKE_SYSTEM_PROCESSOR} STREQUAL "aarch64")
6 include(${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/common.cmake)
5 7
6# LDC+SDE example 8 # LDC+SDE example
7set(APP_SDE_SRCS 9 set(APP_SDE_SRCS
8 src/sde_main.cpp 10 src/sde_main.cpp
9 src/sde_node.cpp 11 src/sde_node.cpp
10 src/sde.cpp 12 src/sde.cpp
11 src/sde_manage_graph.cpp) 13 src/sde_manage_graph.cpp)
12 14
13# Build nodes 15 # Build nodes
14build_node(sde APP_SDE_SRCS) 16 build_node(sde APP_SDE_SRCS)
17endif()
diff --git a/nodes/ti_sde/README.md b/nodes/ti_sde/README.md
index 76afddb..eebca6b 100644
--- a/nodes/ti_sde/README.md
+++ b/nodes/ti_sde/README.md
@@ -3,16 +3,16 @@ Stereo Vision Application on ROS
3 3
4It demonstrates the stereo application that uses J7 LDC (Lense Distortion Correction) and DMPAC SDE (Stereo Depth Engine) HWAs. 4It demonstrates the stereo application that uses J7 LDC (Lense Distortion Correction) and DMPAC SDE (Stereo Depth Engine) HWAs.
5 5
6LDC first rectifies the unrectified input stereo images using the separte rectifiation tables for lett and right cameras, respectively, in the format that LDC supports. 6LDC first rectifies the unrectified input stereo images using the separate rectification tables for lett and right cameras, respectively, in the format that LDC supports.
7 7
8Then SDE produces disparity map from the rectified stereo images. Two different disparity estimation modes are supported in the application. One is a single-layer SDE mode, which outputs the raw disparity map from SDE without post processing. The other is a mulit-layer SDE refinement mode, which combines the disparity maps produced by SDE at different layers with post processings. Up to 3 layers are supported and these are configuarable. The overall application data flow is shown below. 8Then SDE produces disparity map from the rectified stereo images. Two different disparity estimation modes are supported in the application. One is a single-layer SDE mode, which outputs the raw disparity map from SDE without post processing. The other is a multi-layer SDE refinement mode, which combines the disparity maps produced by SDE at different layers with post processing. Up to 3 layers are supported and these are configurable. The overall application data flow is shown below.
9 9
10<figure class="image"> 10<figure class="image">
11 <center><img src="./docs/ti_sde_data_flow.png"></center> 11 <center><img src="./docs/ti_sde_data_flow.png"></center>
12 <figcaption> <center>Figure 1. Overall stereo application flow </center></figcaption> 12 <figcaption> <center>Figure 1. Overall stereo application flow </center></figcaption>
13</figure> 13</figure>
14 14
15# Introduction 15## Introduction
16 16
17`ti_sde` directory structure is shown below: 17`ti_sde` directory structure is shown below:
18``` 18```
@@ -43,34 +43,50 @@ This application can be run by launching sde.launch file, i.e.,
43``` 43```
44roslaunch ti_sde sde.launch 44roslaunch ti_sde sde.launch
45``` 45```
46It is recommended to launch bag_sde.launch file if a rosbag file needs to be played as well. 46It is recommended to launch bag_sde.launch file if a ROSBAG file needs to be played as well.
47 47
48sde.launch file specifies the followings: 48sde.launch file specifies the followings:
49* YAML file that includes algorithm configuration parameters. For the decriptions of them, please take a look at this file. 49* YAML file that includes algorithm configuration parameters. For the descriptions of important parameters, refer to Parameter section below. For the descriptions of all parameters, please see a yaml file.
50* Left input topic name to read left images from a stereo camera. 50* Left input topic name to read left images from a stereo camera.
51* Right input topic name to read right images from a stereo camera. 51* Right input topic name to read right images from a stereo camera.
52* Output disparity topic name to publish raw disparity maps. 52* Output disparity topic name to publish raw disparity maps.
53 53
54This application publishes the raw disparity map and the color-coded disparity map image and confidence map image, which can be dispalyed by RViz, are created by launcing the ti_viz_disparity application, i.e., 54This application publishes the raw disparity map and the color-coded disparity map image and confidence map image, which can be displayed by RViz, are created by launching the ti_viz_nodes application, i.e.,
55``` 55```
56roslaunch ti_viz_disparity viz_disparity.launch 56roslaunch ti_viz_nodes viz_disparity.launch
57``` 57```
58 58
59# LDC (Lense Distortion Correction) 59## Parameters
60 60
61As shown in Figure 1, We use the LDC HWA to rectify left and right images. In order to use LDC, the recfification tables should be provided in the format that LDC support. It is a two-step process to create the rectification table in the LDC format. 61 Parameter | Description | Value
62--------------------------|------------------------------------------------------------------------------|----------
63 left_lut_file_path | LDC rectification table path for left image | String
64 right_lut_file_path | LDC rectification table path for right image | String
65 input_format | Input image format, 0: U8, 1: YUV422 | 0, 1
66 width | Input image width | Integer
67 height | Input image height | Integer
68 sde_algo_type | SDE algorithm type, 0: sinlge-layer SDE, 1: multi-layer SDE | 0, 1
69 num_layers | Number of layers in multi-layer SDE | 2, 3
70 sde_confidence_threshold | Disparity with confidence less than this value is invalidated | 0 ~ 7
71 disparity_min | Minimum disparity to search, 0: 0, 1: -3 | 0, 1
72 disparity_max | Maximum disparity to search, 0: 63, 1: 127, 2: 191 | 0 ~ 2
73 pipeline_depth | OpenVX grpah pipeline depth | 1 ~ 4
74
75## LDC (Lense Distortion Correction)
76
77As shown in Figure 1, We use the LDC HWA to rectify left and right images. In order to use LDC, the rectification tables should be provided in the format that LDC support. It is a two-step process to create the rectification table in the LDC format.
62 78
631. Generation of raw rectification table 791. Generation of raw rectification table
64 80
65 A raw look-up table has `width x height x 2` entries in it, where width and height are the horizontal and vertical sizes of an image, to specify the horizonal and vertical pixel position in a source image that every pixel in a target image maps to. It may consist of two look-up tables of `width x height` for hosizontal position and vertical position, respectively. A target image (i.e. rectified image) is created by fetching the pixel in a source image (i.e. unrectified image), which is spcified by a raw look up table, for every pixel. For example, OpenCV stereo rectification function generates such a raw rectification table for given camera parameters. 81 A raw look-up table has `width x height x 2` entries in it, where width and height are the horizontal and vertical sizes of an image, to specify the horizontal and vertical pixel position in a source image that every pixel in a target image maps to. It may consist of two look-up tables of `width x height` for horizontal position and vertical position, respectively. A target image (i.e. rectified image) is created by fetching the pixel in a source image (i.e. unrectified image), which is specified by a raw look up table, for every pixel. For example, OpenCV stereo rectification function generates such a raw rectification table for given camera parameters.
66 82
672. Convertion of raw rectification table to LDC format 832. Convention of raw rectification table to LDC format
68 84
69 A raw retification table is converted to the LDC format by the following pseudo code. 85 A raw rectification table is converted to the LDC format by the following pseudo code.
70 86
71 ``` 87 ```
72 // mapX is a raw LUT for horizontal pixel position in Q3 format. Its size is width x height 88 // mapX is a raw LUT for horizontal pixel position in Q3 format. Its size is width x height
73 // mapY is a raw LUT for vertial pixel positin in Q3 format. Its size is width x height 89 // mapY is a raw LUT for vertical pixel position in Q3 format. Its size is width x height
74 // ldcLUT is an output LDC format look-up table 90 // ldcLUT is an output LDC format look-up table
75 91
76 LDC_DS_FACTOR = 4 92 LDC_DS_FACTOR = 4
@@ -113,17 +129,17 @@ As shown in Figure 1, We use the LDC HWA to rectify left and right images. In or
113 ``` 129 ```
114 130
115 131
116# SDE (Stereo Depth Engine) 132## SDE (Stereo Depth Engine)
117 133
118When `sde_algo_type = 0` in params.yaml, the output disparity map is simply the disparity map generated by the SDE HWA without any post processing. 134When `sde_algo_type = 0` in params.yaml, the output disparity map is simply the disparity map generated by the SDE HWA without any post processing.
119 135
120# Multi-Layer SDE Refinement 136## Multi-Layer SDE Refinement
121 137
122When `sde_algo_type = 1` in params.yaml, the mullti-layer SDE refinement is applied with post processings to improve the disparity map quality. The number of layers is configured by `num_layers` in params.yaml and it should be 2 or 3. 138When `sde_algo_type = 1` in params.yaml, the multi-layer SDE refinement is applied with post processing to improve the disparity map quality. The number of layers is configured by `num_layers` in params.yaml and it should be 2 or 3.
123 139
124<figure class="image"> 140<figure class="image">
125 <center><img src="./docs/sde_multilayer_data_flow.png"></center> 141 <center><img src="./docs/sde_multilayer_data_flow.png"></center>
126 <figcaption> <center>Figure 2. Multi-layer SDE refinement </center></figcaption> 142 <figcaption> <center>Figure 2. Multi-layer SDE refinement </center></figcaption>
127</figure> 143</figure>
128 144
129Figure 2 shows the overal block diagram of the multi-layer SDE Refinment approach. The rectified stereo pair at full resolution are down-sampled by half to smaller resolutions by the MSC (Multi -Scaler) HWA. The stereo pair at each layers are provided to SDEs to produce the disparity maps at different resolutions. Then the low-resolution disparity map is up-sampled and merged with the high-resolution disparity map successively. The merged disparity map is further processed by hole filling algorithm. 145Figure 2 shows the overal block diagram of the multi-layer SDE refinement approach. The rectified stereo pair at full resolution are down-sampled by half to smaller resolutions by the MSC (Multi-Scaler) HWA. The stereo pair at each layers are provided to SDEs to produce the disparity maps at different resolutions. Then the low-resolution disparity map is up-sampled and merged with the high-resolution disparity map successively. The merged disparity map is further processed by hole filling algorithm.
diff --git a/nodes/ti_sde/config/params.yaml b/nodes/ti_sde/config/params.yaml
index f33a5d0..6daa349 100644
--- a/nodes/ti_sde/config/params.yaml
+++ b/nodes/ti_sde/config/params.yaml
@@ -14,7 +14,6 @@ width: 1280
14# input data height in lines 14# input data height in lines
15height: 720 15height: 720
16 16
17
18# SDE type 17# SDE type
19# 0: Single-layer SDE, 1: Multi-layer SDE 18# 0: Single-layer SDE, 1: Multi-layer SDE
20sde_algo_type: 0 19sde_algo_type: 0
@@ -24,7 +23,7 @@ sde_algo_type: 0
24num_layers: 2 23num_layers: 2
25 24
26# For single-layer SDE only 25# For single-layer SDE only
27# disparity thrshold 26# disparity threshold
28sde_confidence_threshold: 1 27sde_confidence_threshold: 1
29 28
30####### SDE params BEGIN ####### 29####### SDE params BEGIN #######
@@ -46,7 +45,7 @@ texture_filter_enable: 0
46# <0 - 255>, scaled threshold for texture based filtering 45# <0 - 255>, scaled threshold for texture based filtering
47threshold_texture: 0 46threshold_texture: 0
48 47
49# <0 - 127>, Optimization penalty constant for smalle disparity change 48# <0 - 127>, Optimization penalty constant for small disparity change
50aggregation_penalty_p1: 32 49aggregation_penalty_p1: 32
51 50
52# <0 - 255>, Optimization penalty constant for large disparity change 51# <0 - 255>, Optimization penalty constant for large disparity change
diff --git a/nodes/ti_sde/launch/bag_sde.launch b/nodes/ti_sde/launch/bag_sde.launch
index 6ed2472..2326a84 100644
--- a/nodes/ti_sde/launch/bag_sde.launch
+++ b/nodes/ti_sde/launch/bag_sde.launch
@@ -1,9 +1,9 @@
1<launch> 1<launch>
2 2
3 <!-- ti_sde --> 3 <!-- ti_sde -->
4 <include file="$(find ti_sde)/launch/sde.launch"/> 4 <include file="$(find ti_sde)/launch/sde.launch"/>
5 5
6 <!-- rosbag play --> 6 <!-- rosbag play -->
7 <arg name="bagfile" default="$(env WORK_DIR)/data/ros_bag/2020-09-14-12-31-43.bag"/> 7 <arg name="bagfile" default="$(env WORK_DIR)/data/ros_bag/2020-09-14-12-31-43.bag"/>
8 <arg name="ratefactor" default="1.0"/> 8 <arg name="ratefactor" default="1.0"/>
9 <node name="rosbag_play" pkg="rosbag" type="play" output="screen" 9 <node name="rosbag_play" pkg="rosbag" type="play" output="screen"
diff --git a/nodes/ti_viz_color_conversion/launch/yuv_to_rgb.launch b/nodes/ti_sde/launch/rviz.launch
index cd27c3f..3598df8 100644
--- a/nodes/ti_viz_color_conversion/launch/yuv_to_rgb.launch
+++ b/nodes/ti_sde/launch/rviz.launch
@@ -1,17 +1,21 @@
1<launch> 1<launch>
2 2
3 <node pkg="ti_viz_color_conversion" type="viz_color_conversion" name="viz_color_conversion_node" output="screen" args="" required="true"> 3 <!-- color conversion from YUV422 to RGB -->
4 <node pkg="ti_viz_nodes" type="viz_color_conversion" name="viz_color_conversion_node" output="screen" args="" required="true">
4 <param name="width" value="1280"/> 5 <param name="width" value="1280"/>
5 <param name="height" value="720"/> 6 <param name="height" value="720"/>
6
7 <!-- Input YUV image topic name to subscribe to --> 7 <!-- Input YUV image topic name to subscribe to -->
8 <param name= "input_yuv_topic" value = "camera/right/image_raw"/> 8 <param name= "input_yuv_topic" value = "camera/right/image_raw"/>
9
10 <!-- Output RGB image topic name to publish to --> 9 <!-- Output RGB image topic name to publish to -->
11 <param name= "output_rgb_topic" value = "camera/right/image_raw_rgb"/> 10 <param name= "output_rgb_topic" value = "camera/right/image_raw_rgb"/>
12
13 <!-- Input YUV image format, YUV420 or YUV422 --> 11 <!-- Input YUV image format, YUV420 or YUV422 -->
14 <param name= "yuv_format" value = "YUV422"/> 12 <param name= "yuv_format" value = "YUV422"/>
15 </node> 13 </node>
16 14
15 <!-- generate disparity map and confidence map from disparity_raw -->
16 <include file="$(find ti_viz_nodes)/launch/viz_disparity.launch"/>
17
18 <!-- rviz -->
19 <node pkg="rviz" type="rviz" name="rviz_sde" args="-d $(find ti_sde)/rviz/sde.rviz" />
20
17</launch> 21</launch>
diff --git a/nodes/ti_sde/rviz/sde.rviz b/nodes/ti_sde/rviz/sde.rviz
new file mode 100644
index 0000000..928abb9
--- /dev/null
+++ b/nodes/ti_sde/rviz/sde.rviz
@@ -0,0 +1,136 @@
1Panels:
2 - Class: rviz/Displays
3 Help Height: 78
4 Name: Displays
5 Property Tree Widget:
6 Expanded:
7 - /Global Options1
8 - /Status1
9 - /Image1
10 - /Image2
11 - /Image3
12 Splitter Ratio: 0.5
13 Tree Height: 363
14 - Class: rviz/Selection
15 Name: Selection
16 - Class: rviz/Tool Properties
17 Expanded:
18 - /2D Pose Estimate1
19 - /2D Nav Goal1
20 - /Publish Point1
21 Name: Tool Properties
22 Splitter Ratio: 0.5886790156364441
23 - Class: rviz/Time
24 Experimental: false
25 Name: Time
26 SyncMode: 0
27 SyncSource: Image
28Preferences:
29 PromptSaveOnExit: true
30Toolbars:
31 toolButtonStyle: 2
32Visualization Manager:
33 Class: ""
34 Displays:
35 - Class: rviz/Image
36 Enabled: true
37 Image Topic: /camera/right/image_raw_rgb
38 Max Value: 1
39 Median window: 5
40 Min Value: 0
41 Name: Image
42 Normalize Range: true
43 Queue Size: 2
44 Transport Hint: raw
45 Unreliable: false
46 Value: true
47 - Class: rviz/Image
48 Enabled: true
49 Image Topic: /camera/disparity/ccDisparity
50 Max Value: 1
51 Median window: 5
52 Min Value: 0
53 Name: Image
54 Normalize Range: true
55 Queue Size: 2
56 Transport Hint: raw
57 Unreliable: false
58 Value: true
59 - Class: rviz/Image
60 Enabled: true
61 Image Topic: /camera/disparity/ccConfidence
62 Max Value: 1
63 Median window: 5
64 Min Value: 0
65 Name: Image
66 Normalize Range: true
67 Queue Size: 2
68 Transport Hint: raw
69 Unreliable: false
70 Value: true
71 Enabled: true
72 Global Options:
73 Background Color: 48; 48; 48
74 Default Light: true
75 Fixed Frame: map
76 Frame Rate: 30
77 Name: root
78 Tools:
79 - Class: rviz/Interact
80 Hide Inactive Objects: true
81 - Class: rviz/MoveCamera
82 - Class: rviz/Select
83 - Class: rviz/FocusCamera
84 - Class: rviz/Measure
85 - Class: rviz/SetInitialPose
86 Theta std deviation: 0.2617993950843811
87 Topic: /initialpose
88 X std deviation: 0.5
89 Y std deviation: 0.5
90 - Class: rviz/SetGoal
91 Topic: /move_base_simple/goal
92 - Class: rviz/PublishPoint
93 Single click: true
94 Topic: /clicked_point
95 Value: true
96 Views:
97 Current:
98 Class: rviz/Orbit
99 Distance: 14.049280166625977
100 Enable Stereo Rendering:
101 Stereo Eye Separation: 0.05999999865889549
102 Stereo Focal Distance: 1
103 Swap Stereo Eyes: false
104 Value: false
105 Focal Point:
106 X: 0
107 Y: 0
108 Z: 0
109 Focal Shape Fixed Size: true
110 Focal Shape Size: 0.05000000074505806
111 Invert Z Axis: false
112 Name: Current View
113 Near Clip Distance: 0.009999999776482582
114 Pitch: 1.1503978967666626
115 Target Frame: <Fixed Frame>
116 Value: Orbit (rviz)
117 Yaw: 0.8753975629806519
118 Saved: ~
119Window Geometry:
120 Displays:
121 collapsed: false
122 Height: 1123
123 Hide Left Dock: false
124 Hide Right Dock: false
125 Image:
126 collapsed: false
127 QMainWindow State: 000000ff00000000fd0000000400000000000006190000025afc0200000008fb0000001200530065006c0065006300740069006f006e00000001ec0000025a0000005c00fffffffb0000001e0054006f006f006c002000500072006f007000650072007400690065007302000000cf000000ad0000019f00000216fb000000120056006900650077007300200054006f006f02000001df000002110000018500000122fb000000200054006f006f006c002000500072006f0070006500720074006900650073003203000002880000011d000002210000017afb000000100044006900730070006c00610079007302000000a900000094000001c70000041cfb0000002000730065006c0065006300740069006f006e00200062007500660066006500720200000138000000aa0000023a00000294fb00000014005700690064006500530074006500720065006f02000000e6000000d2000003ee0000030bfb0000000c004b0069006e0065006300740200000186000001060000030c0000026100000001000006190000025afc0200000004fb0000000a0049006d00610067006501000001ec0000025a0000001600fffffffb0000001e0054006f006f006c002000500072006f00700065007200740069006500730100000041000000780000000000000000fb0000000a00560069006500770073000000003d000002b00000000000000000fb0000001200530065006c0065006300740069006f006e010000025a000000b20000000000000000000000020000063f000001d0fc0100000003fb0000000a0049006d00610067006501000000000000030d0000005e00fffffffb0000000a0049006d00610067006501000003130000032c0000005e00fffffffb0000000a00560069006500770073030000004e00000080000002e10000019700000003000008720000003efc0100000002fb0000000800540069006d0065000000000000000872000002eb00fffffffb0000000800540069006d00650100000000000004500000000000000000000000200000025a00000004000000040000000800000008fc0000000100000002000000010000000a0054006f006f006c00730000000000ffffffff0000000000000000
128 Selection:
129 collapsed: false
130 Time:
131 collapsed: false
132 Tool Properties:
133 collapsed: false
134 Width: 1599
135 X: 213
136 Y: 139
diff --git a/nodes/ti_sde/src/sde.h b/nodes/ti_sde/src/sde.h
index c4598d4..1451d5a 100755
--- a/nodes/ti_sde/src/sde.h
+++ b/nodes/ti_sde/src/sde.h
@@ -194,7 +194,7 @@ typedef struct
194 uint8_t inputFormat; 194 uint8_t inputFormat;
195 195
196 /** Application interactive status, 0=non-interactive, 1=interactive. */ 196 /** Application interactive status, 0=non-interactive, 1=interactive. */
197 uint32_t is_interactive; 197 uint8_t is_interactive;
198 198
199 SDELDCAPPLIB_createParams sdeLdcCreateParams; 199 SDELDCAPPLIB_createParams sdeLdcCreateParams;
200 SDELDCAPPLIB_Handle sdeLdcHdl; 200 SDELDCAPPLIB_Handle sdeLdcHdl;
diff --git a/nodes/ti_semseg_cnn/CMakeLists.txt b/nodes/ti_semseg_cnn/CMakeLists.txt
index 04c4d28..93497bc 100644
--- a/nodes/ti_semseg_cnn/CMakeLists.txt
+++ b/nodes/ti_semseg_cnn/CMakeLists.txt
@@ -1,13 +1,16 @@
1cmake_minimum_required(VERSION 3.0.2) 1cmake_minimum_required(VERSION 3.0.2)
2project(ti_semseg_cnn) 2project(ti_semseg_cnn)
3 3
4include(${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/common.cmake) 4# Build only on J7
5if(${CMAKE_SYSTEM_PROCESSOR} STREQUAL "aarch64")
6 include(${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/common.cmake)
5 7
6# CNN based Semantic Segmentation example 8 # CNN based Semantic Segmentation example
7set(APP_SEMSEG_CNN_SRCS 9 set(APP_SEMSEG_CNN_SRCS
8 src/semseg_cnn_main.cpp 10 src/semseg_cnn_main.cpp
9 src/semseg_cnn_node.cpp 11 src/semseg_cnn_node.cpp
10 src/semseg_cnn.cpp) 12 src/semseg_cnn.cpp)
11 13
12# Build nodes 14 # Build nodes
13build_node(semseg_cnn APP_SEMSEG_CNN_SRCS) 15 build_node(semseg_cnn APP_SEMSEG_CNN_SRCS)
16endif() \ No newline at end of file
diff --git a/nodes/ti_semseg_cnn/README.md b/nodes/ti_semseg_cnn/README.md
index 29b8c21..3fcf61c 100644
--- a/nodes/ti_semseg_cnn/README.md
+++ b/nodes/ti_semseg_cnn/README.md
@@ -1,14 +1,14 @@
1Semantic Segmentation Application on ROS 1Semantic Segmentation Application on ROS
2======================================== 2========================================
3 3
4It demonstrates the semantic segmenation application using a TIDL (TI Deep Learning) CNN network running on J7 C7x/MMA. However, as shown in Figure 1, this application consists of multiple processing blocks that consumes other J7 HWAs and processors besides C7x/MMA. 4It demonstrates the semantic segmentation application using a TIDL (TI Deep Learning) CNN network running on J7 C7x/MMA. However, as shown in Figure 1, this application consists of multiple processing blocks that consumes other J7 HWAs and processors besides C7x/MMA.
5 5
6<figure class="image"> 6<figure class="image">
7 <center><img src="./docs/ti_semseg_cnn_data_flow.png"></center> 7 <center><img src="./docs/ti_semseg_cnn_data_flow.png"></center>
8 <figcaption> <center>Figure 1. Overall semantic segmentation application flow </center></figcaption> 8 <figcaption> <center>Figure 1. Overall semantic segmentation application flow </center></figcaption>
9</figure> 9</figure>
10 10
11# Introduction 11## Introduction
12 12
13`ti_semseg_cnn` directory structure is shown below: 13`ti_semseg_cnn` directory structure is shown below:
14``` 14```
@@ -36,21 +36,41 @@ This application can be run by launching semseg_cnn.launch file, i.e.,
36``` 36```
37roslaunch ti_semseg_cnn semseg_cnn.launch 37roslaunch ti_semseg_cnn semseg_cnn.launch
38``` 38```
39It is recommended to launch bag_semseg_cnn.launch file if a rosbag file needs to be played as well. 39It is recommended to launch bag_semseg_cnn.launch file if a ROSBAG file needs to be played as well.
40 40
41semseg_cnn.launch file specifies the followings: 41semseg_cnn.launch file specifies the followings:
42* YAML file that includes algorithm configuration parameters. For the descriptions of them, please take a look at this file. 42* YAML file that includes algorithm configuration parameters. For the descriptions of important parameters, refer to Parameter section below. For the description of all parameters, please see a yaml file.
43* Input topic name to read input images. 43* Input topic name to read input images.
44* Output undistorted or rectified image topic name. 44* Output undistorted or rectified image topic name.
45* Output semantic segmentation image topic name when an color-coded semantic segmentation map is published. 45* Output semantic segmentation image topic name when an color-coded semantic segmentation map is published.
46* Flag that indicates the color-coded semantic segmentation map is published in RGB format. If this flag is false, it is published in YUV420 format. 46* Flag that indicates the color-coded semantic segmentation map is published in RGB format. If this flag is false, it is published in YUV420 format.
47* Output semantic segmentation tensor topic name when the output tensor is published. 47* Output semantic segmentation tensor topic name when the output tensor is published.
48 48
49When the semantic segmentation output tensor is published by the application, the color-coded semantic segmentation can be created alternatively by launching the ti_viz_semseg application, i.e., 49When the semantic segmentation output tensor is published by the application, the color-coded semantic segmentation can be created alternatively by launching the ti_viz_nodes application, i.e.,
50``` 50```
51roslaunch ti_viz_semseg viz_semseg.launch 51roslaunch ti_viz_nodes viz_semseg.launch
52``` 52```
53# Processing Blocks 53
54## Parameters
55
56 Parameter | Description | Value
57--------------------------|------------------------------------------------------------------------------|----------
58 lut_file_path | LDC rectification table path | String
59 tidl_config_file_path | TIDL config file path | String
60 tidl_network_file_path | TIDL network file path | String
61 tidl_network_file_path | TIDL network file path | String
62 width | Input image width | Integer
63 height | Input image height | Integer
64 dl_width | Image width to TIDL network | Integer
65 dl_height | Image height to TIDL network | Integer
66 out_width | Output semantic segmentation output (tensor) width | Integer
67 out_height | Output semantic segmentation output (tensor) height | Integer
68 num_classes | Number of semantic segmentation classes | 0 ~ 20
69 enable_post_proc | Flag to indicate if the post processing of th TIDL output should be enabled | 0, 1
70 pipeline_depth | OpenVX graph pipeline depth | 1 ~ 4
71
72
73## Processing Blocks
54 74
55Please refer to Figure 1 for the following descriptions of the processing blocks implemented for this application. 75Please refer to Figure 1 for the following descriptions of the processing blocks implemented for this application.
56 76
diff --git a/nodes/ti_semseg_cnn/launch/bag_semseg_cnn.launch b/nodes/ti_semseg_cnn/launch/bag_semseg_cnn.launch
index 1e6bb04..242e7aa 100644
--- a/nodes/ti_semseg_cnn/launch/bag_semseg_cnn.launch
+++ b/nodes/ti_semseg_cnn/launch/bag_semseg_cnn.launch
@@ -1,7 +1,7 @@
1<launch> 1<launch>
2 2
3 <!-- ti_semseg_cnn --> 3 <!-- ti_semseg_cnn -->
4 <include file="$(find ti_semseg_cnn)/launch/semseg_cnn.launch"/> 4 <include file="$(find ti_semseg_cnn)/launch/semseg_cnn.launch"/>
5 5
6 <!-- rosbag play --> 6 <!-- rosbag play -->
7 <arg name="bagfile" default="$(env WORK_DIR)/data/ros_bag/2020-09-14-12-31-43.bag"/> 7 <arg name="bagfile" default="$(env WORK_DIR)/data/ros_bag/2020-09-14-12-31-43.bag"/>
diff --git a/nodes/ti_semseg_cnn/launch/rviz.launch b/nodes/ti_semseg_cnn/launch/rviz.launch
new file mode 100644
index 0000000..86bbd28
--- /dev/null
+++ b/nodes/ti_semseg_cnn/launch/rviz.launch
@@ -0,0 +1,18 @@
1<launch>
2
3 <!-- color conversion for input image_raw for visualization -->
4 <node pkg="ti_viz_nodes" type="viz_color_conversion" name="viz_color_conversion_node" output="screen" args="" required="true">
5 <param name="width" value="1280"/>
6 <param name="height" value="720"/>
7 <param name="input_yuv_topic" value="camera/right/image_raw"/>
8 <param name="output_rgb_topic" value="camera/right/image_raw_rgb"/>
9 <param name="yuv_format" value="YUV422"/>
10 </node>
11
12 <!-- viz_semseg_cnn -->
13 <include file="$(find ti_viz_nodes)/launch/viz_semseg.launch"/>
14
15 <!-- rviz -->
16 <node pkg="rviz" type="rviz" name="rviz_semseg" args="-d $(find ti_semseg_cnn)/rviz/semseg_cnn.rviz" />
17
18</launch>
diff --git a/nodes/ti_semseg_cnn/rviz/semseg_cnn.rviz b/nodes/ti_semseg_cnn/rviz/semseg_cnn.rviz
new file mode 100644
index 0000000..51dc01a
--- /dev/null
+++ b/nodes/ti_semseg_cnn/rviz/semseg_cnn.rviz
@@ -0,0 +1,123 @@
1Panels:
2 - Class: rviz/Displays
3 Help Height: 78
4 Name: Displays
5 Property Tree Widget:
6 Expanded:
7 - /Global Options1
8 - /Status1
9 - /Image1
10 - /Image2
11 Splitter Ratio: 0.5
12 Tree Height: 363
13 - Class: rviz/Selection
14 Name: Selection
15 - Class: rviz/Tool Properties
16 Expanded:
17 - /2D Pose Estimate1
18 - /2D Nav Goal1
19 - /Publish Point1
20 Name: Tool Properties
21 Splitter Ratio: 0.5886790156364441
22 - Class: rviz/Time
23 Experimental: false
24 Name: Time
25 SyncMode: 0
26 SyncSource: Image
27Preferences:
28 PromptSaveOnExit: true
29Toolbars:
30 toolButtonStyle: 2
31Visualization Manager:
32 Class: ""
33 Displays:
34 - Class: rviz/Image
35 Enabled: true
36 Image Topic: /camera/right/image_raw_rgb
37 Max Value: 1
38 Median window: 5
39 Min Value: 0
40 Name: Image
41 Normalize Range: true
42 Queue Size: 2
43 Transport Hint: raw
44 Unreliable: false
45 Value: true
46 - Class: rviz/Image
47 Enabled: true
48 Image Topic: semseg_cnn/out_image
49 Max Value: 1
50 Median window: 5
51 Min Value: 0
52 Name: Image
53 Normalize Range: true
54 Queue Size: 2
55 Transport Hint: raw
56 Unreliable: false
57 Value: true
58 Enabled: true
59 Global Options:
60 Background Color: 48; 48; 48
61 Default Light: true
62 Fixed Frame: map
63 Frame Rate: 30
64 Name: root
65 Tools:
66 - Class: rviz/Interact
67 Hide Inactive Objects: true
68 - Class: rviz/MoveCamera
69 - Class: rviz/Select
70 - Class: rviz/FocusCamera
71 - Class: rviz/Measure
72 - Class: rviz/SetInitialPose
73 Theta std deviation: 0.2617993950843811
74 Topic: /initialpose
75 X std deviation: 0.5
76 Y std deviation: 0.5
77 - Class: rviz/SetGoal
78 Topic: /move_base_simple/goal
79 - Class: rviz/PublishPoint
80 Single click: true
81 Topic: /clicked_point
82 Value: true
83 Views:
84 Current:
85 Class: rviz/Orbit
86 Distance: 14.049280166625977
87 Enable Stereo Rendering:
88 Stereo Eye Separation: 0.05999999865889549
89 Stereo Focal Distance: 1
90 Swap Stereo Eyes: false
91 Value: false
92 Focal Point:
93 X: 0
94 Y: 0
95 Z: 0
96 Focal Shape Fixed Size: true
97 Focal Shape Size: 0.05000000074505806
98 Invert Z Axis: false
99 Name: Current View
100 Near Clip Distance: 0.009999999776482582
101 Pitch: 1.1503978967666626
102 Target Frame: <Fixed Frame>
103 Value: Orbit (rviz)
104 Yaw: 0.8753975629806519
105 Saved: ~
106Window Geometry:
107 Displays:
108 collapsed: false
109 Height: 794
110 Hide Left Dock: false
111 Hide Right Dock: false
112 Image:
113 collapsed: false
114 QMainWindow State: 000000ff00000000fd0000000400000000000006190000025afc0200000008fb0000001200530065006c0065006300740069006f006e00000001ec0000025a0000005c00fffffffb0000001e0054006f006f006c002000500072006f007000650072007400690065007302000000cf000000ad0000019f00000216fb000000120056006900650077007300200054006f006f02000001df000002110000018500000122fb000000200054006f006f006c002000500072006f0070006500720074006900650073003203000002880000011d000002210000017afb000000100044006900730070006c00610079007302000000a900000094000001c70000041cfb0000002000730065006c0065006300740069006f006e00200062007500660066006500720200000138000000aa0000023a00000294fb00000014005700690064006500530074006500720065006f02000000e6000000d2000003ee0000030bfb0000000c004b0069006e0065006300740200000186000001060000030c0000026100000001000006190000021ffc0200000003fb0000001e0054006f006f006c002000500072006f00700065007200740069006500730100000041000000780000000000000000fb0000000a00560069006500770073000000003d000002b00000000000000000fb0000001200530065006c0065006300740069006f006e010000025a000000b2000000000000000000000002000006c8000002cafc0100000004fb0000000a0049006d0061006700650100000000000003600000005e00fffffffb0000000a0049006d0061006700650100000366000003620000005e00fffffffb0000000a0049006d00610067006501000003130000032c0000000000000000fb0000000a00560069006500770073030000004e00000080000002e10000019700000003000008720000003efc0100000002fb0000000800540069006d0065000000000000000872000002eb00fffffffb0000000800540069006d00650100000000000004500000000000000000000006c80000001700000004000000040000000800000008fc0000000100000002000000010000000a0054006f006f006c00730000000000ffffffff0000000000000000
115 Selection:
116 collapsed: false
117 Time:
118 collapsed: false
119 Tool Properties:
120 collapsed: false
121 Width: 1736
122 X: 499
123 Y: 97
diff --git a/nodes/ti_viz_color_conversion/CMakeLists.txt b/nodes/ti_viz_color_conversion/CMakeLists.txt
deleted file mode 100644
index f399d88..0000000
--- a/nodes/ti_viz_color_conversion/CMakeLists.txt
+++ /dev/null
@@ -1,14 +0,0 @@
1cmake_minimum_required(VERSION 3.0.2)
2project(ti_viz_color_conversion)
3
4include(${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/common.cmake)
5
6# Visualization nodes only built on PC
7if(${TARGET_PLATFORM} STREQUAL "PC")
8
9 # yuv420 to RGB conversion
10 set(VIZ_COL_CONV_SRCS src/ros_app_color_conversion_node.cpp)
11 build_node(viz_color_conversion VIZ_COL_CONV_SRCS)
12
13endif()
14
diff --git a/nodes/ti_viz_disparity/CMakeLists.txt b/nodes/ti_viz_disparity/CMakeLists.txt
deleted file mode 100644
index bfa8434..0000000
--- a/nodes/ti_viz_disparity/CMakeLists.txt
+++ /dev/null
@@ -1,15 +0,0 @@
1cmake_minimum_required(VERSION 3.0.2)
2project(ti_viz_disparity)
3
4include(${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/common.cmake)
5
6# Visualization nodes only built on PC
7if(${TARGET_PLATFORM} STREQUAL "PC")
8 set(VIZ_DISP_SRCS src/viz_disparity_node.cpp)
9 build_node(viz_disparity VIZ_DISP_SRCS)
10 add_dependencies(viz_disparity ${catkin_EXPORTED_TARGETS})
11endif()
12
13
14
15
diff --git a/nodes/ti_viz_disparity/README.md b/nodes/ti_viz_disparity/README.md
deleted file mode 100644
index c415888..0000000
--- a/nodes/ti_viz_disparity/README.md
+++ /dev/null
@@ -1,28 +0,0 @@
1Disparity Map Visualization
2===========================
3
4This application subscribes to a raw disparity map topic, which is published by ti_sde, creates a color-coded disparity map and a confidence map from the raw disparity map and publish them.
5
6
7`ti_viz_disparity` directory structure is shown below:
8```
9ti_viz_disparity
10.
11├── CMakeLists.txt
12├── launch
13│   └── viz_disparity.launch
14├── package.xml
15├── README.md
16└── src
17 └── viz_disparity_node.cpp
18
19```
20
21We can run this application by launching viz_disparity.launch file, i.e.,
22```
23roslaunch ti_viz_disparity viz_disparity.launch
24```
25viz_disparity.launch file specifies the followings:
26* Input topic name to read a raw disparity map.
27* Output topic names to publish a color-coded disparity map and confidence map, respectively, which can be dispalyed directly by RViz.
28
diff --git a/nodes/ti_viz_disparity/launch/viz_disparity.launch b/nodes/ti_viz_disparity/launch/viz_disparity.launch
deleted file mode 100644
index 20ac4bc..0000000
--- a/nodes/ti_viz_disparity/launch/viz_disparity.launch
+++ /dev/null
@@ -1,16 +0,0 @@
1<launch>
2
3 <node pkg="ti_viz_disparity" type="viz_disparity" name="viz_disparity" output="screen" args="" required="true">
4 <param name="width" value="1280"/>
5 <param name="height" value="720"/>
6
7 <!-- Input raw dispairty topic to subscribe to -->
8 <param name= "disparity_topic" value = "camera/disparity/raw"/>
9 <!-- Output color coded disparity map topic to publish to -->
10 <param name = "cc_disparity_topic" value = "camera/disparity/ccDisparity"/>
11 <!-- Output color coded confidence map topic to publish to -->
12 <param name = "cc_confidence_topic" value = "camera/disparity/ccConfidence"/>
13
14 </node>
15
16</launch>
diff --git a/nodes/ti_viz_disparity/package.xml b/nodes/ti_viz_disparity/package.xml
deleted file mode 100644
index 4ce9959..0000000
--- a/nodes/ti_viz_disparity/package.xml
+++ /dev/null
@@ -1,47 +0,0 @@
1<?xml version="1.0"?>
2<package format="2">
3 <name>ti_viz_disparity</name>
4 <version>0.1.0</version>
5 <description>The ti_viz_disparity package</description>
6
7 <!-- One maintainer tag required, multiple allowed, one person per tag -->
8 <!-- Example: -->
9 <!-- <maintainer email="jane.doe@example.com">Jane Doe</maintainer> -->
10 <maintainer email="d-kwon@ti.com">d.kwon</maintainer>
11
12 <!-- One license tag required, multiple allowed, one license per tag -->
13 <!-- Commonly used license strings: -->
14 <!-- BSD, MIT, Boost Software License, GPLv2, GPLv3, LGPLv2.1, LGPLv3 -->
15 <license>TODO</license>
16
17 <buildtool_depend>catkin</buildtool_depend>
18 <build_depend>geometry_msgs</build_depend>
19 <build_depend>roscpp</build_depend>
20 <build_depend>nodelet</build_depend>
21 <build_depend>cv_bridge</build_depend>
22 <build_depend>image_transport</build_depend>
23 <build_depend>sensor_msgs</build_depend>
24 <build_depend>message_generation</build_depend>
25 <!-- <build_depend>ti_sde</build_depend> -->
26 <build_export_depend>geometry_msgs</build_export_depend>
27 <build_export_depend>roscpp</build_export_depend>
28 <build_export_depend>nodelet</build_export_depend>
29 <build_export_depend>cv_bridge</build_export_depend>
30 <build_export_depend>image_transport</build_export_depend>
31 <build_export_depend>sensor_msgs</build_export_depend>
32 <build_export_depend>message_runtime</build_export_depend>
33 <!-- <build_export_depend>ti_sde</build_export_depend> -->
34 <exec_depend>geometry_msgs</exec_depend>
35 <exec_depend>roscpp</exec_depend>
36 <exec_depend>nodelet</exec_depend>
37 <exec_depend>cv_bridge</exec_depend>
38 <exec_depend>image_transport</exec_depend>
39 <exec_depend>sensor_msgs</exec_depend>
40 <exec_depend>message_runtime</exec_depend>
41 <!-- <exec_depend>ti_sde</exec_depend> -->
42
43 <!-- The export tag contains other, unspecified, tags -->
44 <export>
45 <!-- Other tools can request additional information be placed here -->
46 </export>
47</package>
diff --git a/nodes/ti_viz_nodes/CMakeLists.txt b/nodes/ti_viz_nodes/CMakeLists.txt
new file mode 100644
index 0000000..59ed65b
--- /dev/null
+++ b/nodes/ti_viz_nodes/CMakeLists.txt
@@ -0,0 +1,51 @@
1cmake_minimum_required(VERSION 3.0.2)
2project(ti_viz_nodes)
3
4add_compile_options(-std=c++14)
5
6find_package(catkin REQUIRED COMPONENTS roscpp cv_bridge image_transport sensor_msgs std_msgs message_generation common_msgs)
7
8catkin_package(CATKIN_DEPENDS roscpp cv_bridge image_transport sensor_msgs message_runtime)
9
10find_package(OpenCV REQUIRED)
11
12include_directories(
13 include
14 ${catkin_INCLUDE_DIRS}
15 ${OpenCV_INCLUDE_DIRS}
16)
17
18link_directories(${catkin_LIBRARY_DIRS})
19
20# Function for building a node:
21# ARG0: app name
22# ARG1: source list
23function(build_node)
24 set(app ${ARGV0})
25 set(src ${ARGV1})
26 add_executable(${app} ${${src}})
27 target_link_libraries(${app} ${catkin_LIBRARIES})
28 add_dependencies(${app} ${common_msgs_EXPORTED_TARGETS})
29endfunction()
30
31# Visualization nodes only built on PC
32if (${CMAKE_SYSTEM_PROCESSOR} STREQUAL "x86_64")
33 # viz_disparity
34 set(VIZ_DISP_SRCS src/viz_disparity_node.cpp)
35 build_node(viz_disparity VIZ_DISP_SRCS)
36 add_dependencies(viz_disparity ${catkin_EXPORTED_TARGETS})
37
38 # YUV to RGB conversion
39 set(VIZ_COL_CONV_SRCS src/ros_app_color_conversion_node.cpp)
40 build_node(viz_color_conversion VIZ_COL_CONV_SRCS)
41
42 # viz_semseg
43 set(VIZ_SEMSEG_SRCS src/viz_semseg_node.cpp)
44 build_node(viz_semseg VIZ_SEMSEG_SRCS)
45 add_dependencies(viz_semseg ${catkin_EXPORTED_TARGETS})
46
47endif()
48
49
50
51
diff --git a/nodes/ti_viz_nodes/README.md b/nodes/ti_viz_nodes/README.md
new file mode 100644
index 0000000..a3b2d55
--- /dev/null
+++ b/nodes/ti_viz_nodes/README.md
@@ -0,0 +1,29 @@
1Visualization Nodes
2===================
3
4## Disparity Map Visualization
5
6This application subscribes to a raw disparity map topic, which is published by ti_sde, creates a color-coded disparity map and a confidence map from the raw disparity map and publish them.
7
8We can run this application by launching `viz_disparity.launch` file, i.e.,
9```
10roslaunch ti_viz_nodes viz_disparity.launch
11```
12viz_disparity.launch file specifies the followings:
13* Input topic name to read a raw disparity map.
14* Output topic names to publish a color-coded disparity map and confidence map, respectively, which can be displayed directly by RViz.
15
16<!-- =================================================================================================== -->
17## Semantic Segmentation Map Visualization
18
19This application subscribes to two topics, one is an input image topic, on which class dependent colors will be overlaid, and the other is a semantic segmentation CNN output tensor topic, which has class information for all pixels. It creates a color-coded semantic segmentation map from them, which is published as an output.
20
21We can run this application by launching `viz_semseg.launch` file, i.e.,
22```
23roslaunch ti_viz_nodes viz_semseg.launch
24```
25
26`viz_semseg.launch` file specifies the followings:
27* Input image topic name to read input images.
28* CNN tensor topic name to read output tensors from a CNN network.
29* Output topic name to publish a color-coded semantic segmentation map, which can be displayed directly by RViz.
diff --git a/nodes/ti_viz_nodes/launch/viz_disparity.launch b/nodes/ti_viz_nodes/launch/viz_disparity.launch
new file mode 100644
index 0000000..7dae506
--- /dev/null
+++ b/nodes/ti_viz_nodes/launch/viz_disparity.launch
@@ -0,0 +1,14 @@
1<launch>
2
3 <node pkg="ti_viz_nodes" type="viz_disparity" name="viz_disparity" output="screen" args="" required="true">
4 <param name="width" value="1280"/>
5 <param name="height" value="720"/>
6 <!-- Input raw dispairty topic to subscribe to -->
7 <param name="disparity_topic" value="camera/disparity/raw"/>
8 <!-- Output color coded disparity map topic to publish to -->
9 <param name="cc_disparity_topic" value="camera/disparity/ccDisparity"/>
10 <!-- Output color coded confidence map topic to publish to -->
11 <param name="cc_confidence_topic" value="camera/disparity/ccConfidence"/>
12 </node>
13
14</launch>
diff --git a/nodes/ti_viz_semseg/launch/viz_semseg.launch b/nodes/ti_viz_nodes/launch/viz_semseg.launch
index f5fd19b..6a63265 100644
--- a/nodes/ti_viz_semseg/launch/viz_semseg.launch
+++ b/nodes/ti_viz_nodes/launch/viz_semseg.launch
@@ -1,21 +1,16 @@
1<launch> 1<launch>
2 2
3 <node pkg="ti_viz_semseg" type="viz_semseg" name="viz_semseg" output="screen" args="" required="true"> 3 <node pkg="ti_viz_nodes" type="viz_semseg" name="viz_semseg" output="screen" args="" required="true">
4
5 <param name="width" value="1280"/> 4 <param name="width" value="1280"/>
6 <param name="height" value="720"/> 5 <param name="height" value="720"/>
7 <param name="tensor_width" value="768"/> 6 <param name="tensor_width" value="768"/>
8 <param name="tensor_height" value="432"/> 7 <param name="tensor_height" value="432"/>
9
10 <!-- Input recitified image topic name to subscribe to --> 8 <!-- Input recitified image topic name to subscribe to -->
11 <param name = "rectified_image_topic" value = "camera/right/image_rect_mono"/> 9 <param name="rectified_image_topic" value="camera/right/image_rect_mono"/>
12
13 <!-- Input CNN tensor topic name to subscribe to --> 10 <!-- Input CNN tensor topic name to subscribe to -->
14 <param name = "semseg_cnn_tensor_topic" value = "semseg_cnn/tensor"/> 11 <param name="semseg_cnn_tensor_topic" value="semseg_cnn/tensor"/>
15
16 <!-- Output semantic segmentation image topic name to publish to --> 12 <!-- Output semantic segmentation image topic name to publish to -->
17 <param name = "semseg_cnn_image_topic" value = "semseg_cnn/out_image"/> 13 <param name="semseg_cnn_image_topic" value="semseg_cnn/out_image"/>
18
19 </node> 14 </node>
20 15
21</launch> 16</launch>
diff --git a/nodes/ti_viz_nodes/launch/yuv_to_rgb.launch b/nodes/ti_viz_nodes/launch/yuv_to_rgb.launch
new file mode 100644
index 0000000..d60714d
--- /dev/null
+++ b/nodes/ti_viz_nodes/launch/yuv_to_rgb.launch
@@ -0,0 +1,15 @@
1<launch>
2
3 <node pkg="ti_viz_nodes" type="viz_color_conversion"
4 name="viz_color_conversion" output="screen" args="" required="true">
5 <param name="width" value="1280"/>
6 <param name="height" value="720"/>
7 <!-- Input YUV image topic name to subscribe to -->
8 <param name="input_yuv_topic" value="camera/right/image_raw"/>
9 <!-- Output RGB image topic name to publish to -->
10 <param name="output_rgb_topic" value="camera/right/image_raw_rgb"/>
11 <!-- Input YUV image format, YUV420 or YUV422 -->
12 <param name="yuv_format" value="YUV422"/>
13 </node>
14
15</launch>
diff --git a/nodes/ti_viz_color_conversion/package.xml b/nodes/ti_viz_nodes/package.xml
index dfbc5fb..e2dec76 100644
--- a/nodes/ti_viz_color_conversion/package.xml
+++ b/nodes/ti_viz_nodes/package.xml
@@ -1,8 +1,8 @@
1<?xml version="1.0"?> 1<?xml version="1.0"?>
2<package format="2"> 2<package format="2">
3 <name>ti_viz_color_conversion</name> 3 <name>ti_viz_nodes</name>
4 <version>0.1.0</version> 4 <version>0.1.0</version>
5 <description>The ti_viz_color_conversion package</description> 5 <description>The ti_viz_nodes package</description>
6 6
7 <!-- One maintainer tag required, multiple allowed, one person per tag --> 7 <!-- One maintainer tag required, multiple allowed, one person per tag -->
8 <!-- Example: --> 8 <!-- Example: -->
@@ -17,21 +17,18 @@
17 <buildtool_depend>catkin</buildtool_depend> 17 <buildtool_depend>catkin</buildtool_depend>
18 <build_depend>geometry_msgs</build_depend> 18 <build_depend>geometry_msgs</build_depend>
19 <build_depend>roscpp</build_depend> 19 <build_depend>roscpp</build_depend>
20 <build_depend>nodelet</build_depend>
21 <build_depend>cv_bridge</build_depend> 20 <build_depend>cv_bridge</build_depend>
22 <build_depend>image_transport</build_depend> 21 <build_depend>image_transport</build_depend>
23 <build_depend>sensor_msgs</build_depend> 22 <build_depend>sensor_msgs</build_depend>
24 <build_depend>message_generation</build_depend> 23 <build_depend>message_generation</build_depend>
25 <build_export_depend>geometry_msgs</build_export_depend> 24 <build_export_depend>geometry_msgs</build_export_depend>
26 <build_export_depend>roscpp</build_export_depend> 25 <build_export_depend>roscpp</build_export_depend>
27 <build_export_depend>nodelet</build_export_depend>
28 <build_export_depend>cv_bridge</build_export_depend> 26 <build_export_depend>cv_bridge</build_export_depend>
29 <build_export_depend>image_transport</build_export_depend> 27 <build_export_depend>image_transport</build_export_depend>
30 <build_export_depend>sensor_msgs</build_export_depend> 28 <build_export_depend>sensor_msgs</build_export_depend>
31 <build_export_depend>message_runtime</build_export_depend> 29 <build_export_depend>message_runtime</build_export_depend>
32 <exec_depend>geometry_msgs</exec_depend> 30 <exec_depend>geometry_msgs</exec_depend>
33 <exec_depend>roscpp</exec_depend> 31 <exec_depend>roscpp</exec_depend>
34 <exec_depend>nodelet</exec_depend>
35 <exec_depend>cv_bridge</exec_depend> 32 <exec_depend>cv_bridge</exec_depend>
36 <exec_depend>image_transport</exec_depend> 33 <exec_depend>image_transport</exec_depend>
37 <exec_depend>sensor_msgs</exec_depend> 34 <exec_depend>sensor_msgs</exec_depend>
diff --git a/nodes/ti_viz_color_conversion/src/ros_app_color_conversion_node.cpp b/nodes/ti_viz_nodes/src/ros_app_color_conversion_node.cpp
index 9932b1a..9932b1a 100644
--- a/nodes/ti_viz_color_conversion/src/ros_app_color_conversion_node.cpp
+++ b/nodes/ti_viz_nodes/src/ros_app_color_conversion_node.cpp
diff --git a/nodes/ti_viz_disparity/src/viz_disparity_node.cpp b/nodes/ti_viz_nodes/src/viz_disparity_node.cpp
index fc8e7ff..fc8e7ff 100644
--- a/nodes/ti_viz_disparity/src/viz_disparity_node.cpp
+++ b/nodes/ti_viz_nodes/src/viz_disparity_node.cpp
diff --git a/nodes/ti_viz_semseg/src/viz_semseg_node.cpp b/nodes/ti_viz_nodes/src/viz_semseg_node.cpp
index 0583670..9dd469a 100644
--- a/nodes/ti_viz_semseg/src/viz_semseg_node.cpp
+++ b/nodes/ti_viz_nodes/src/viz_semseg_node.cpp
@@ -11,13 +11,14 @@
11using namespace sensor_msgs; 11using namespace sensor_msgs;
12using namespace message_filters; 12using namespace message_filters;
13 13
14 14// OpenCV uses BGR format
15static const uint8_t color_map[20][3] = 15static const uint8_t color_map[20][3] =
16{{128, 64, 128},{244, 35, 232},{ 70, 70, 70},{102, 102, 156},{190, 153, 153}, 16{{128, 64, 128},{244, 35, 232},{ 70, 70, 70},{102, 102, 156},{190, 153, 153},
17 {153, 153, 153},{250, 170, 30},{220, 220, 0},{107, 142, 35},{152, 251, 152}, 17 {153, 153, 153},{250, 170, 30},{220, 220, 0},{107, 142, 35},{152, 251, 152},
18 { 70, 130, 180},{220, 20, 60},{255, 0, 0},{ 0, 0, 142},{ 0, 0, 70}, 18 { 70, 130, 180},{220, 20, 60},{255, 0, 0},{ 0, 0, 142},{ 0, 0, 70},
19 { 0, 60, 100},{ 0, 80, 100},{ 0, 0, 230},{119, 11, 32},{128, 128, 128}}; 19 { 0, 60, 100},{ 0, 80, 100},{ 0, 0, 230},{119, 11, 32},{128, 128, 128}};
20 20
21
21namespace ros_app_viz 22namespace ros_app_viz
22{ 23{
23 /** 24 /**
@@ -81,13 +82,16 @@ namespace ros_app_viz
81 // Semantic class 82 // Semantic class
82 uint8_t classId; 83 uint8_t classId;
83 84
85 // blending factor
86 float bf = 0.5;
87
84 int32_t stride = m_width * 3; 88 int32_t stride = m_width * 3;
85 89
86 float horScale = m_tensorWidth *1.0/m_width; 90 float horScale = m_tensorWidth *1.0/m_width;
87 float verScale = m_tensorHeight*1.0/m_height; 91 float verScale = m_tensorHeight*1.0/m_height;
88 92
89 cv_bridge::CvImagePtr cv_ssPtr; 93 cv_bridge::CvImagePtr cv_ssPtr;
90 cv_ssPtr = cv_bridge::toCvCopy(imagePtr, sensor_msgs::image_encodings::BGR8); 94 cv_ssPtr = cv_bridge::toCvCopy(imagePtr, sensor_msgs::image_encodings::RGB8);
91 95
92 for (j = 0; j < m_height; j++) 96 for (j = 0; j < m_height; j++)
93 { 97 {
@@ -100,11 +104,11 @@ namespace ros_app_viz
100 classId = tensorPtr->data[y*m_tensorWidth + x]; 104 classId = tensorPtr->data[y*m_tensorWidth + x];
101 105
102 cv_ssPtr->image.data[j*stride + i*3] = 106 cv_ssPtr->image.data[j*stride + i*3] =
103 0.5*cv_ssPtr->image.data[j*stride + i*3] + 0.5*color_map[classId][0]; 107 (1-bf)*cv_ssPtr->image.data[j*stride + i*3] + bf*color_map[classId][0];
104 cv_ssPtr->image.data[j*stride + i*3 + 1] = 108 cv_ssPtr->image.data[j*stride + i*3 + 1] =
105 0.5*cv_ssPtr->image.data[j*stride + i*3 + 1] + 0.5*color_map[classId][1]; 109 (1-bf)*cv_ssPtr->image.data[j*stride + i*3 + 1] + bf*color_map[classId][1];
106 cv_ssPtr->image.data[j*stride + i*3 + 2] = 110 cv_ssPtr->image.data[j*stride + i*3 + 2] =
107 0.5*cv_ssPtr->image.data[j*stride + i*3 + 2] + 0.5*color_map[classId][2]; 111 (1-bf)*cv_ssPtr->image.data[j*stride + i*3 + 2] + bf*color_map[classId][2];
108 } 112 }
109 } 113 }
110 114
diff --git a/nodes/ti_viz_semseg/CMakeLists.txt b/nodes/ti_viz_semseg/CMakeLists.txt
deleted file mode 100644
index e462b21..0000000
--- a/nodes/ti_viz_semseg/CMakeLists.txt
+++ /dev/null
@@ -1,14 +0,0 @@
1cmake_minimum_required(VERSION 3.0.2)
2project(ti_viz_semseg)
3
4include(${CMAKE_CURRENT_SOURCE_DIR}/../../cmake/common.cmake)
5
6# Visualization nodes only built on PC
7if(${TARGET_PLATFORM} STREQUAL "PC")
8 set(VIZ_SEMSEG_SRCS src/viz_semseg_node.cpp)
9 build_node(viz_semseg VIZ_SEMSEG_SRCS)
10 add_dependencies(viz_semseg ${catkin_EXPORTED_TARGETS})
11endif()
12
13
14
diff --git a/nodes/ti_viz_semseg/README.md b/nodes/ti_viz_semseg/README.md
deleted file mode 100644
index d0ef2d8..0000000
--- a/nodes/ti_viz_semseg/README.md
+++ /dev/null
@@ -1,29 +0,0 @@
1Semantic Segmentation Map Visualization
2=======================================
3
4This application subscribes to two topics, one is an input image topic, on which class dependent colors will be overlaid, and the other is a semantic segmentation CNN output tensor topic, which has class information for all pixels. It creates a color-coded semantic segmentation map from them, which is published as an output.
5
6
7`ti_viz_semseg` directory structure is shown below:
8```
9ti_viz_semseg
10.
11├── CMakeLists.txt
12├── launch
13│   └── viz_semseg.launch
14├── package.xml
15├── README.md
16└── src
17 └── viz_semseg_node.cpp
18
19```
20
21We can run this application by launching viz_semseg.launch file, i.e.,
22```
23roslaunch ti_viz_semseg viz_semseg.launch
24```
25viz_semseg.launch file specifies the followings:
26* Input image topic name to read input images.
27* CNN tensor topic name to read output tensors from a CNN network.
28* Output topic name to publish a color-coded semantic segmentation map, which can be displayed directly by RViz.
29
diff --git a/nodes/ti_viz_semseg/package.xml b/nodes/ti_viz_semseg/package.xml
deleted file mode 100644
index 093e7b3..0000000
--- a/nodes/ti_viz_semseg/package.xml
+++ /dev/null
@@ -1,47 +0,0 @@
1<?xml version="1.0"?>
2<package format="2">
3 <name>ti_viz_semseg</name>
4 <version>0.1.0</version>
5 <description>The ti_viz_semseg package</description>
6
7 <!-- One maintainer tag required, multiple allowed, one person per tag -->
8 <!-- Example: -->
9 <!-- <maintainer email="jane.doe@example.com">Jane Doe</maintainer> -->
10 <maintainer email="d-kwon@ti.com">d.kwon</maintainer>
11
12 <!-- One license tag required, multiple allowed, one license per tag -->
13 <!-- Commonly used license strings: -->
14 <!-- BSD, MIT, Boost Software License, GPLv2, GPLv3, LGPLv2.1, LGPLv3 -->
15 <license>TODO</license>
16
17 <buildtool_depend>catkin</buildtool_depend>
18 <build_depend>geometry_msgs</build_depend>
19 <build_depend>roscpp</build_depend>
20 <build_depend>nodelet</build_depend>
21 <build_depend>cv_bridge</build_depend>
22 <build_depend>image_transport</build_depend>
23 <build_depend>sensor_msgs</build_depend>
24 <build_depend>message_generation</build_depend>
25 <!-- <build_depend>ti_semseg_cnn</build_depend> -->
26 <build_export_depend>geometry_msgs</build_export_depend>
27 <build_export_depend>roscpp</build_export_depend>
28 <build_export_depend>nodelet</build_export_depend>
29 <build_export_depend>cv_bridge</build_export_depend>
30 <build_export_depend>image_transport</build_export_depend>
31 <build_export_depend>sensor_msgs</build_export_depend>
32 <build_export_depend>message_runtime</build_export_depend>
33 <!-- <build_export_depend>ti_semseg_cnn</build_export_depend> -->
34 <exec_depend>geometry_msgs</exec_depend>
35 <exec_depend>roscpp</exec_depend>
36 <exec_depend>nodelet</exec_depend>
37 <exec_depend>cv_bridge</exec_depend>
38 <exec_depend>image_transport</exec_depend>
39 <exec_depend>sensor_msgs</exec_depend>
40 <exec_depend>message_runtime</exec_depend>
41 <!-- <exec_depend>ti_semseg_cnn</exec_depend> -->
42
43 <!-- The export tag contains other, unspecified, tags -->
44 <export>
45 <!-- Other tools can request additional information be placed here -->
46 </export>
47</package>
diff --git a/setup_env_pc.sh b/setup_env_pc.sh
new file mode 100755
index 0000000..1f57e6b
--- /dev/null
+++ b/setup_env_pc.sh
@@ -0,0 +1,11 @@
1# NOTE: Update the following lines based on your setting
2export J7_IP_ADDR=192.168.1.32
3export PC_IP_ADDR=192.168.1.20
4export CATKIN_WS=$HOME/j7ros_home/catkin_ws
5
6# ROS network setting
7export ROS_MASTER_URI=http://$J7_IP_ADDR:11311
8export ROS_IP=$PC_IP_ADDR
9
10source /opt/ros/melodic/setup.bash
11source $CATKIN_WS/devel/setup.bash