Predicting Vehicle Collision Using Transformer Network with Multi-Modal Data

Road accidents pose a significant threat to human life, causing numerous injuries, fatalities, and economic damage worldwide. Recently, there has been growing interest in leveraging Artificial Intelligence (AI) to create systems that can predict vehicle crashes. This research focuses on vehicle collision prediction and aims to develop a solution combining pre-trained Convolutional Neural Networks (CNN) and transformer network to mitigate the occurrence of such accidents. By leveraging advanced deep learning techniques, this research addresses the limitations of traditional crash analysis methods. The Car Learning to Act (CARLA) simulator was used for data gathering, with an ego-vehicle attached with RGB and RGB-Depth cameras. Four pre-trained CNNs were used for feature extraction. With those extracted features, a transformer network was employed to train a model. After model training and testing, it was observed that the transformer model trained with VGG16-based feature extraction performs better than other methods.