Persistent structure meets dynamic attention: Cross-variable priors for multivariate time series forecasting

Multivariate time series contain two kinds of cross-variable relationships: persistent ones that reflect underlying structure (geographic proximity, shared infrastructure, physical coupling) and dynamic ones that arise from transient conditions in each observation window. Current transformer architectures conflate the two—channel-independent models ignore cross-variable relationships entirely, while cross-variable methods must rediscover all structure from scratch in every forward pass. We propose Factored Topology Bias (FTB), which explicitly separates these concerns by adding a learned low-rank prior P P ⊤ to cross-variable attention logits, encoding time-invariant relationships, while standard attention (QK⊤) captures input-specific dynamics. The factored form requires only O(Cdp) parameters and is interleaved with temporal attention at every encoder layer, controlled by a learned gate. On standard benchmarks spanning 21–862 variables, FTB improves forecasting accuracy by 2–7% on high-dimensional datasets while correctly providing no benefit when persistent structure is absent. Ablation reveals a horizon-dependent complementarity: the structural prior contributes 33% of the gain at short horizons but 88% at long horizons, confirming that time-invariant knowledge compensates as temporal signal fades. We introduce a Params-Per-Pair diagnostic that predicts from dataset properties alone whether structural priors will help.