A Multimodal Deep Fusion Method for Stock Movement Prediction Using Heterogeneous Data Source

Stock movement prediction remains challenging due to the complexity of integrating heterogeneous data sources and capturing intricate temporal-semantic relationships. Existing approaches suffer from incomplete semantic preservation, shallow fusion mechanisms, and inadequate multi-scale temporal modeling. In this paper, we propose a multimodal deep fusion framework that addresses these limitations through three synergistic components: 1) An SRTE module based on pretrained BERT that preserves 100% of textual content by segmenting long sequences and applying average pooling, recovering 17.82% more tokens than conventional truncation; 2) An MsLSTM architecture that simultaneously captures short-term price dynamics and long-term market trends through parallel processing of original and temporally smoothed sequences, and; 3) A CMCF mechanism that performs dual-level integration via across-temporal interval fusion and along-temporal step aggregation. We compare our proposed method against baseline models on the StockNet dataset, achieving superior predictive performance. Additionally, we conduct comprehensive ablation studies, back-testing analysis, long-text modeling comparisons, and interpretability analysis. The results validate the effectiveness of deep multimodal fusion for financial forecasting and provide actionable insights for data-driven decision-making in volatile market environments. © 2013 IEEE.