Fourier Ordinary Differential Equations

Abstract

Continuous models such as Neural Ordinary Differential Equations (NODEs) are powerful approaches for modeling time series data, known for their ability to capture underlying dynamics and generalization. Current continuous models focus on learning mappings within finite-dimensional Euclidean spaces, raising two critical questions for enhancing their effectiveness. First, Is Euclidean space the optimal representation for capturing the underlying patterns and features in time series data? Second, how can we maintain granularity while benefiting from the generalization capabilities of continuous models? To address the first question, we propose a novel approach for learning dynamics in the Fourier domain. In contrast to Euclidean space, each point in Fourier space summarizes the original signal at a specific frequency, enabling more comprehensive data representations. Additionally, time differentiation in the Fourier domain simplifies the modeling of dynamics as it becomes a multiplication operation. To answer the second question, we introduce element-wise filtering, a method designed to compensate for the bias of continuous models when fitting discrete data points. These techniques culminate in the introduction of a new approach—Fourier Ordinary Differential Equations (FODEs). Our experiments provide compelling evidence of FODEs' superiority in terms of accuracy, efficiency, and generalization capabilities when compared to existing methods across various time series datasets. By offering a novel method for modeling time series data capable of capturing both short-term and long-term patterns, FODEs have the potential to significantly enhance the modeling and prediction of complex dynamic systems.