Probabilistic model for sensor fusion and data association in pedestrian tracking

(under review process)

Abstract: Pedestrian detection and tracking have been the focus of interest due to their high demand and applications. A plethora of work has been done in this field in the last decades; however, due to the dynamic environments, the data obtained from a single visual sensor for these operations cannot suffice to cover all the environmental conditions. Hence, the use of multiple, heterogeneous visual sensors in such applications is indispensable. This research is focused on pedestrian detection and tracking using sequential importance resampling particle filters based on the fused data from thermal and colour images condition on environmental conditions that affect luminance change in both modalities. The proposed method uses Naive Bayes and Bayesian inference probabilistic models for late-fusion and tracking of pedestrians in the sequences. The results in this paper demonstrate that the fused data for pedestrian detection and tracking outperforms other existing state-of-the-art solutions, especially when the environmental information is taken into the account in combination with RGB-T (Red Green Blue – Thermal) sensor data.

Probabilistic Fusion for Pedestrian Detection from Thermal and Colour Images

MDPI Sensors, vol. 22, no. 22, 2022

Abstract: Pedestrian detection is an important research domain due to its relevance for autonomous and assisted driving, as well as its applications in security and industrial automation. Often, more than one type of sensor is used to cover a broader range of operating conditions than a single-sensor system would allow. However, it remains difficult to make pedestrian detection systems perform well in highly dynamic environments, often requiring extensive retraining of the algorithms for specific conditions to reach satisfactory accuracy, which, in turn, requires large, annotated datasets captured in these conditions. In this paper, we propose a probabilistic decision-level sensor fusion method based on naive Bayes to improve the efficiency of the system by combining the output of available pedestrian detectors for colour and thermal images without retraining. The results in this paper, obtained through long-term experiments, demonstrate the efficacy of our technique, its ability to work with non-registered images, and its adaptability to cope with situations when one of the sensors fails. The results also show that our proposed technique improves the overall accuracy of the system and could be very useful in several applications.

Automatic annotation of pedestrians in thermal images using b-f segmentation for training deep neural networks

IEEE Symposium Series on Computational Intelligence, Australia, 2020

Abstract: Deep Neural Networks for object detection have become of significant interest with the substantial improvement in their efficiency and increase in their applications. However, training such a network requires a large annotated dataset, which is very expensive in term of time and human effort. In this context, several automatic image annotation solutions have been proposed which are dependent on rich visual features and thus suitable for color images only. On the other hand, recent experiments have proven that color cameras alone are not enough for pedestrian related applications and there is a need to use other visual sensors as well, such as thermal cameras. In this paper, we propose an automatic image annotation technique for pedestrian detection in thermal images using an adaptive background/foreground estimation model to train Faster-RCNN. The results presented in this paper, obtained through long-term experiments demonstrate the efficacy of our technique. The results also show that our proposed technique is be very useful to generate image annotations automatically and to train a deep neural network without having a manually annotated dataset for cameras with different modalities.