Documentation Guide

Advanced Autonomous Vehicle Vision System

Getting Started

Overview

VisionSense is a comprehensive ROS2-based computer vision system designed for autonomous vehicles running on NVIDIA Jetson platforms with JetPack 6.2. It provides a complete perception pipeline with real-time object detection, lane detection, traffic sign recognition, stereo depth estimation, and driver monitoring capabilities.

Features

Feature	Description	Model/Method
Object Detection	Detect vehicles, pedestrians, cyclists, traffic signs/lights	`YOLOv8 + TensorRT`
Multi-Object Tracking	Track objects across frames with unique IDs	`BYTE Tracker + Kalman Filter`
Lane Detection	Segment and detect lane lines	`Neural Network + TensorRT`
Traffic Sign Recognition	Classify 50+ traffic sign types	`YOLOv8 Classifier + TensorRT`
Stereo Depth Estimation	Dense depth maps from stereo camera	`LightStereo + TensorRT`
Driver Monitoring	Face detection and gaze estimation	`YOLOv11 + ResNet18 + TensorRT`
Data Fusion GUI	Real-time visualization of all perception data	`OpenCV + X11`
Web Dashboard	Remote monitoring interface	`HTTP Server`

System Requirements

Hardware: NVIDIA Jetson Orin Nano/NX/AGX
OS: Ubuntu 22.04 (JetPack 6.2)
ROS2: Humble Hawksbill

CUDA: 12.6+
TensorRT: 10.x
OpenCV: 4.x with CUDA support

Installation

Step 1: Clone the Repository

git clone https://github.com/connected-wise/VisionSense.git
cd VisionSense

Step 2: Install OpenCV with CUDA Support

sudo bash install_opencv_cuda_orin.sh

This process takes 2-3 hours. Configures OpenCV with CUDA 12.6 for Jetson Orin (compute capability 8.7).

Step 3: Install ROS2 and Dependencies

sudo bash install_all_deps.sh

Installs ROS2 Humble, jetson-inference, build tools, and all required libraries.

Step 4: Build VisionSense

source /opt/ros/humble/setup.bash
colcon build --packages-select visionconnect

Usage

Desktop Launcher

Double-click the VisionSense icon on the desktop to launch the application.

Command Line

source /opt/ros/humble/setup.bash
cd ~/VisionSense && source install/setup.bash
ros2 launch visionconnect visionsense.launch.py

Individual Nodes

ros2 run visionconnect camera
ros2 run visionconnect detect
ros2 run visionconnect gui

Need More Help?

Can't find what you're looking for? Check out our community forum or contact our support team.

Get Support Community Forum