Self-Supervised Temporal Transformer Learning for Behavioral Segmentation in Digital Banking

Abstract

Digital banking systems continuously record detailed traces of how customers interact with financial services, offering rich opportunities for behavioral analysis. However, using these data for customer segmentation remains difficult due to the heterogeneity of banking events, strong temporal dependencies, and the lack of labeled segment definitions. This paper introduces a sequence-based segmentation approach that models customer behavior as a stream of multi-modal banking events and learns representations directly from raw interaction logs. The proposed framework relies on a temporal Transformer encoder trained with self-supervised objectives, allowing categorical attributes, numerical values, and inter-event time oinformation to be jointly encoded. Masked event modeling and next-event prediction are used to guide representation learning, while a deep embedded clustering objective integrates segmentation directly into the training process. Experiments on a large-scale real-world digital banking dataset show that the proposed method produces more compact and better separated behavioral segments than traditional aggregation-based techniques and neural baselines. In addition, the resulting segments remain stable across monthly evaluation windows and reflect persistent behavioral characteristics related to activity regularity, spending variability, and engagement patterns. In the future, we also plan to extend this work in several directions, such as incorporating the segmentation objectives with downstream outcomes, performing online segmentation, and conducting a deeper causal analysis of segment jumps. Plus, it would be interesting to inject domain knowledge or expert intervention into the segmentation process.