Multi-Model Traffic Congestion Prediction Using Diffusion Convolutional Recurrent Neural Networks with Transformer Based Fusion and Dual-Channel CNN for Multimodal Learning

Abstract

Traffic congestion prediction focuses on evaluations upcoming road traffic states by analyzing past and real-time data such as vehicle velocity, traffic throughput and road occupancy. However, many existing models have limited adaptability to rapidly changing urban traffic conditions. To address these limitations, the proposed framework introduces a multimodal traffic prediction mechanism that integrates sensor time-series data with surveillance video streams. Initially, multi-source data are collected where traffic sensors provide time-series values (vehicle count, speed, and density) and roadside cameras provide continuous video frames. The sensors data are pre-processed using ImputeGAN (IGAN) for missing value imputation, Quantile Transformation for Normalization (QTN) and Time2Vec encoding (T2VE) to capture rush-hour patterns. Video frames are enhanced using Deep Unsupervised Blind Image Denoising (DUBID) to remove noise and Self-supervised Low-Light Enhancement (SSLLE) to improve visual quality. After preprocessing, both modalities are temporally aligned using a Sliding Window Synchronization (SWS). Feature-level multi-modal representations are learned using a Dual-Channel Convolutional Neural Network (DCCNN), where ResNet50 extracts detailed traffic density patterns and MobileNetV4 learns lightweight structural features. The combined features are fused through a Transformer-Based Fusion (TBF), to model relationships between numerical trends and vehicle congestion. Finally, the fused representation is fed into a Diffusion Convolutional Recurrent Neural Network (DCRNN) to predict the traffic state and the system classifies the road condition into four levels: low, normal, high, and heavy congestion. By using this methodology achieves the accuracy of 98.22%, precision of 96.48%, and the recall of 96.53%, demonstrating improved prediction efficiency, reliability with sparse data and responsiveness to real-world traffic dynamics.