CUDA-Accelerated GStreamer Pipeline for GigE Thermal Camera Integration Using RAPIDSEA GStreamer Plugins

CUDA-Accelerated GStreamer Pipeline for GigE Thermal Camera Integration Using RAPIDSEA GStreamer Plugins

Snapshot

RAPIDSEA supported an ADAS vision systems developer in building a CUDA-accelerated GStreamer pipeline for thermal camera integration on an embedded Linux platform. By deploying the RAPIDSEA GST-CUDA Color Manipulation and Color Space Conversion plugins, the team eliminated custom GPU shader development entirely — delivering a validated, real-time thermal imaging pipeline with hardware-accelerated frame processing in 7 weeks, with processing latency well within the 33-millisecond budget required for 30fps ADAS vision applications.

Customer Profile

Embedded vision systems developer building ADAS perception modules for commercial vehicle OEMs and aftermarket fleet safety integrators. Product line covers forward collision warning, pedestrian detection, and driver monitoring systems. Engineering teams carry deep expertise in computer vision algorithms and sensor fusion but operate lean on GPU pipeline infrastructure development.

Business Context

Thermal cameras are increasingly adopted in ADAS applications for detecting pedestrians, animals, and road hazards in conditions where RGB cameras fail — low light, rain, fog, and glare. The customer's thermal camera — a GigE Vision device delivering 14-bit radiometric frames over Gigabit Ethernet — needed a processing pipeline covering three stages: raw frame acquisition via GigE Vision transport, pixel format conversion from 14-bit greyscale to 8-bit YUV for algorithm input, and false-colour mapping for the driver display overlay. Building these stages as custom GStreamer plugins with CUDA acceleration from scratch was estimated at eight to ten weeks of dedicated GPU pipeline engineering.

Key Challenges

•  14-Bit Thermal Frame Processing: Standard GStreamer colour space conversion plugins handle 8-bit and 10-bit pixel formats natively but have no support for 14-bit radiometric thermal data. Custom bit-depth handling was required before any standard pipeline element could process the frames.
•  Real-Time CUDA Acceleration Requirement: Processing thermal frames on the CPU at 30fps — particularly the colour mapping stage — consumed sufficient CPU cycles to starve the vision algorithm inference thread. GPU offload via CUDA was mandatory, not optional.
•  False-Colour Mapping for Driver Display: The driver display overlay required false-colour mapping — converting greyscale thermal intensity values to a heat-map colour palette. Implementing a configurable colour lookup table on the GPU with sub-millisecond per-frame latency required CUDA kernel expertise the customer's team did not carry.
•  GigE Vision Transport Integration: The thermal camera used the GigE Vision standard transport protocol. Integrating GigE Vision frame acquisition as a GStreamer source element — with proper buffer management for large 14-bit frame payloads over Gigabit Ethernet — required careful pipeline buffer architecture to prevent frame drops under network jitter conditions.
•  Pipeline Latency Budget: The end-to-end pipeline had a hard 33-millisecond budget at 30fps. Any pipeline stage introducing CPU-GPU synchronisation overhead or unnecessary memory copies risked blowing this budget and causing algorithm input starvation.

Target Platform

NVIDIA Jetson AGX Orin — NVIDIA's high-performance embedded AI computing platform combining an Arm Cortex-A78 CPU cluster with an Ampere GPU and hardware-accelerated video encode/decode engines. Running embedded Linux with GStreamer 1.20 as the media pipeline framework. RAPIDSEA GStreamer CUDA plugins deployed as standard GStreamer plugin shared libraries — loaded by the GStreamer plugin registry with no modification to existing pipeline orchestration code.

Why RAPIDSEA

•  Production-Ready CUDA Kernels Without GPU Development Effort: RAPIDSEA GST-CUDA plugins ship as production-validated GStreamer elements with CUDA kernels already implemented and optimised for Jetson-class GPU architectures. Pipeline behaviour configured through GStreamer element properties — no CUDA kernel authoring, no GPU memory management code, no CUDA stream synchronisation handling required.
•  Zero-Copy GPU Buffer Architecture: RAPIDSEA CUDA plugins use CUDA unified memory and GStreamer CUDA buffer pools to pass frame data between pipeline stages without CPU-GPU memory copies. Each frame moves through acquisition, colour space conversion, and colour manipulation entirely on the GPU memory bus.
•  Configurable Colour Lookup Tables: GST-CUDA Color Manipulation plugin supports runtime-configurable colour lookup tables — false-colour palette selected or updated via GStreamer element properties without pipeline reconstruction. Iron, rainbow, and greyscale palettes pre-validated; custom palette loading supported through the plugin's LUT import interface.

Solution: How to Build a GStreamer CUDA Pipeline for Thermal Camera ADAS Integration

GigE Vision Source Integration and Buffer Architecture

Pipeline constructed with a GigE Vision GStreamer source element — built using the Aravis GigE Vision library as the acquisition backend — feeding raw 14-bit greyscale frames into the pipeline at 30fps. Source element configured with a four-frame acquisition buffer pool, absorbing network jitter. Frames immediately transferred to CUDA device memory at the source element output pad.

RAPIDSEA GST-CUDA Color Space Conversion: 14-Bit to 8-Bit YUV

RAPIDSEA GST-CUDA Color Space Conversion plugin inserted as first processing stage, converting 14-bit greyscale thermal frames to 8-bit YUV420. Plugin's CUDA kernel applied a configurable normalisation range. Conversion throughput on Jetson AGX Orin: 847fps at target resolution — 28x the required 30fps throughput.

RAPIDSEA GST-CUDA Color Manipulation: False-Colour Driver Display Overlay

RAPIDSEA GST-CUDA Color Manipulation plugin inserted in parallel via a GStreamer tee element — producing the false-colour display overlay independently of the algorithm input branch. Iron palette lookup table applied per-pixel on the GPU in a single CUDA kernel pass. Output pad delivered RGB24 frames directly to the display compositor's input buffer.

Pipeline Latency Optimisation

End-to-end pipeline latency measured using GStreamer's built-in latency tracer across fifty thermal scene captures. Zero-copy GPU buffer architecture contributed directly to a measured end-to-end latency of 18.4 milliseconds — 14.6 milliseconds within the 33ms budget.

Vision Algorithm Integration

8-bit YUV420 output from the colour space conversion branch consumed by the customer's TensorRT-based pedestrian detection model through a GStreamer appsink element. False-colour RGB24 output fed the display compositor overlay. Both branches operated from the same source frame with no duplication of acquisition or GPU transfer overhead.

Engineering Impact

Conclusion

Building a CUDA-accelerated GStreamer pipeline for GigE thermal camera integration in an ADAS application demands GPU pipeline expertise that most embedded vision teams do not carry as a core competency. The RAPIDSEA GST-CUDA Color Manipulation and Color Space Conversion plugins gave this vision systems developer production-validated CUDA processing stages — with zero-copy buffer architecture and runtime-configurable colour tables — that delivered an 18.4-millisecond end-to-end pipeline on Jetson AGX Orin without a single CUDA kernel written by the customer's team.

Connect with us to explore RAPIDSEA for your embedded vision applications.