Intelligent transportation requires traffic flow prediction. This aids congestion control, route planning, and infrastructure optimisation. Correct short- and long-term forecasting helps urban transportation and the environment. ARIMA models struggles with nonlinear and nonstationary traffic data. While LSTM networks improve temporal modelling, their sequential structure makes it difficult to express long-range dependencies and slows computation. Recent transformer topologies show promise. They represent global dependencies through self-attention. To simplify networks, this study uses an Encoder-Only Transformer framework to predict traffic flow in several phases without a decoder. The suggested method outperforms LSTM for long-horizon forecasting and decreases inference time on benchmark datasets. However, an encoder-only architecture has drawbacks. It has one attention pathway, no weighting, and is unreliable in traffic. Hyperparameter tuning via grid search increases processing costs and may not optimise performance. The Hybrid Dual-Attention Fusion Framework in this study handles this. An Encoder-Only Transformer, a BiLSTM network, and an APSO algorithm are used. Temporal Multi-Head Self-Attention and Adaptive Feature Recalibration Attention are used. The unique Dual-Attention process uses Temporal Multi-Head Self-Attention and Adaptive Feature Recalibration Attention. The BiLSTM module captures dependencies in both directions to boost sequential context learning. In contrast, the APSO-based tuning module optimises the embedding dimension, the number of attention heads, and the learning rate. Experimental results reveal that prediction accuracy is more consistent and generalizable across datasets.