Barycentric Alignment of Mutually Disentangled Modalities

Abstract

Discovering explanatory factors of user preferences behind behavioral data has gained increasing attention. As collected behavioral data is often highly sparse, mining other data modalities, e.g., texts, for interest factors and then correlating them with those from behavioral data could provide a pathway to improve recommendation. Nonetheless, two challenges prevail. For one, the unordered set nature of discovered factors and the unavailability of prior alignment information causes a challenge to align revealed interest factors from two modalities. For another, it demands a tailored method to effectively transfer knowledge between interest factors from mutually related modalities. To resolve this, we regard discovered interest factors from ratings and texts as supporting points of two discrete measures. Then, their alignment is formulated as an optimal transport problem, finding an optimal mapping between two probability masses. Next, the mapping probability serves not only as the prior information but also as input of barycentric strategy to match and fuse interest factors, effectively tranferring user preferences between mutually disentangled modalities. Experiments on real-world datasets verify the advantage of the proposed method over a series of baselines.