Time-aware neural ordinary differential equations for incomplete time series modeling

• Main Authors: Cai, Z. (Author), Chang, Z. (Author), Liu, S. (Author), Qiu, R. (Author), Song, S. (Author), Tu, G. (Author)
• Format: Article
• Language: English
• Published: Springer 2023
• Subjects: Classification (of information); Classification tasks; Continuous time systems; Incomplete time series; Learning systems; Mean square error; Missing values; Network architecture; Neural networks; Neural ODE; Neural ODEs; Ordinary differential equation models; Ordinary differential equations; Signal encoding; Tagged data; Time points; Time series; Time-aware encoder; Times series; Times series models
• Online Access: View Fulltext in Publisher; View in Scopus
• ISBN: 09208542 (ISSN)
• DOI: 10.1007/s11227-023-05327-8

Internet of Things realizes the ubiquitous connection of all things, generating countless time-tagged data called time series. However, real-world time series are often plagued with missing values on account of noise or malfunctioning sensors. Existing methods for modeling such incomplete time series typically involve preprocessing steps, such as deletion or missing data imputation using statistical learning or machine learning methods. Unfortunately, these methods unavoidable destroy time information and bring error accumulation to the subsequent model. To this end, this paper introduces a novel continuous neural network architecture, named Time-aware Neural-Ordinary Differential Equations (TN-ODE), for incomplete time data modeling. The proposed method not only supports imputation missing values at arbitrary time points, but also enables multi-step prediction at desired time points. Specifically, TN-ODE employs a time-aware Long Short-Term Memory as an encoder, which effectively learns the posterior distribution from partial observed data. Additionally, the derivative of latent states is parameterized with a fully connected network, thereby enabling continuous-time latent dynamics generation. The proposed TN-ODE model is evaluated on both real-world and synthetic incomplete time-series datasets by conducting data interpolation and extrapolation tasks as well as classification task. Extensive experiments show the TN-ODE model outperforms baseline methods in terms of Mean Square Error for imputation and prediction tasks, as well as accuracy in downstream classification task. © 2023, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.