Page 55 - Read Online

Zhuang et al. Intell Robot 2024;4(3):276-92 Intelligence & Robotics
DOI: 10.20517/ir.2024.18

Research Article Open Access

An in-vehicle real-time infrared object detection sys-
tem based on deep learning with resource-constrained
hardware

1
2
3
Tingting Zhuang , Xunru Liang , Bohuan Xue , Xiaoyu Tang 1,2
1 School of Data Science and Engineering, Xingzhi College, South China Normal University, Shanwei 516600, Guangdong, China.
2 School of Physics, South China Normal University, Guangzhou 510006, Guangdong, China.
3 Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong 999077, China.

Correspondence to: Prof. Xiaoyu Tang, School of Data Science and Engineering, Xingzhi College, South China Normal University,
Ma Gong Street, Shanwei 516600, Guangdong, China. E-mail: tangxy@scnu.edu.cn
How to cite this article: Zhuang T, Liang X, Xue B, Tang X. An in-vehicle real-time infrared object detection system based on deep
learning with resource-constrained hardware. Intell Robot 2024;4(3):276-92. http://dx.doi.org/10.20517/ir.2024.18
Received: 13 May 2024 First Decision: 7 Aug 2024 Revised: 6 Sep 2024 Accepted: 11 Sep 2024 Published: 24 Sep 2024

Academic Editor: Simon X. Yang Copy Editor: Dong-Li Li Production Editor: Dong-Li Li

Abstract
Advanceddriver assistancesystemsprimarily rely onvisible images forinformation. However, in low-visibility weather
conditions, such as heavy rain or fog, visible images struggle to capture road conditions accurately. In contrast, in-
frared (IR) images can overcome this limitation, providing reliable information regardless of external lighting. Ad-
dressing this problem, we propose an in-vehicle IR object detection system. We optimize the you only look once
(YOLO) v4 object detection algorithm by replacing its original backbone with MobileNetV3, a lightweight feature ex-
traction network, resulting in the MobileNetV3-YOLOv4 model. Furthermore, we replace traditional pre-processing
methods with an Image Enhancement Conditional Generative Adversarial Network inversion algorithm to enhance
the pre-processing of the input IR images. Finally, we deploy the model on the Jetson Nano, an edge device with con-
strained hardware resources. Our proposed method achieves an 82.7% mean Average Precision and a frame rate of
55.9 frames per second on the FLIR dataset, surpassing state-of-the-art methods. The experimental results confirm
that our approach provides outstanding real-time detection performance while maintaining high precision.

Keywords: Infrared object detection, in-vehicle system, lightweight, limited hardware resources, real-time

© The Author(s) 2024. Open Access This article is licensed under a Creative Commons Attribution 4.0
International License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, shar-
ing, adaptation, distribution and reproduction in any medium or format, for any purpose, even commercially, as long as you
give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate
if changes were made.

www.oaepublish.com/ir