Note: Causality can be encoded without strict time function choice.
problem Global encoding of causality under natural conditions.
method Observation of weakening causality assumptions in existing results.
result Causality can be encoded without strict time function choice.
We study the use of a time series encoder to learn representations that are useful on data set types with which it has not been trained on. The encoder is formed of a convolutional neural network whose temporal output is summarized by a convolutional attention mechanism. This way, we obtain a compact, fixed-length repr…
Conformer encoder reverses sequence in time dimension, affecting decoder training.
problem Reversal of sequence in Conformer encoder impacts decoder training.
method Analyzed initial behavior of decoder cross-attention and proposed methods to avoid flipping.
result Self-attention module of Conformer starts dominating, allowing only reversed information to pass.
Study infers tree topology from customer data using contrastive learning.
problem Inferring local network topology from customer data.
method Contrastive learning approach for binary event encoding from continuous time series.
result Preliminary results show potential for valuable encoder learning.
Deep learning transforms time series into images for anomaly detection in industrial assets.
problem Detecting anomalies in time series data from industrial assets.
method Transforming time series data into image-like representations and using them as inputs for deep learning models.
result Some encodings provide competitive results for anomaly detection in industrial asset monitoring.
AER combines auto-encoder and LSTM for better time series anomaly detection.
problem Anomaly detection in time series data with limited labeled data and ambiguous definitions.
method AER (Auto-Encoder with Regression) integrates auto-encoder and LSTM for joint predictions and reconstructions.
result AER achieves the highest F1 score across 12 datasets with comparable runtime.
The paper uses neural networks to forecast time series data.
problem Forecasting high-dimensional stationary processes.
method Encoder-decoder neural network structure to model past observations.
result Upper bounds for forecast error under specific assumptions.
TiDE uses MLP for fast, simple long-term time-series forecasting.
problem Long-term time-series forecasting challenges.
method Time-series Dense Encoder (TiDE) based on MLP.
result TiDE matches or outperforms Transformer models while being 5-10x faster.
Proposes a method to improve financial time series forecasting using compact representations and contrastive loss.
problem Financial time series forecasting with small datasets and overfitting issues.
method Class-conditioned latent variable model, mutual information maximization, contrastive loss, deep autoregressive models.
result Empirical experiments show improved performance compared to state-of-the-art methods.
Paper proposes a faster RAE with sequence-aware encoding.
problem Training recurrent autoencoders is challenging and time-consuming.
method Introduces a recurrent autoencoder with sequence-aware encoding using 1D convolutional layers.
result The proposed autoencoder trains faster than standard RAE.
New auto-encoder handles varying noise levels without retraining.
problem Auto-encoders degrade in noisy conditions.
method Formalized auto-encoders as transform learning, derived new architecture.
result Models generalize well to different noise levels.
Nowadays, multivariate time series data are increasingly collected in various real world systems, e.g., power plants, wearable devices, etc. Anomaly detection and diagnosis in multivariate time series refer to identifying abnormal status in certain time steps and pinpointing the root causes. Building such a system, how…
Neural networks encode inputs deterministically and categorically, behaving like hash encoders.
problem Understanding the encoding properties of neural networks.
method Analyzed the input space partitioned by ReLU-like activations in neural networks.
result Neural networks can be represented by unique activation patterns, similar to hash encoders.
Improved OOD detection across various shifts using multi-encoder fusion of RDMs.
problem Out-of-distribution detection across multiple types of distribution shifts.
method Statistical identification of encoder sensitivity, EncMin2L fusion, and Tippett minimum combination.
result Achieves AUROC ≥ 0.94 across four shift types, outperforming state-of-the-art detectors.
Principal components analysis (PCA) is the optimal linear auto-encoder of data, and it is often used to construct features. Enforcing sparsity on the principal components can promote better generalization, while improving the interpretability of the features. We study the problem of constructing optimal sparse linear a…
BinConv improves time series forecasting by preserving ordinal information in a classification framework.
problem Lack of ordinal information in existing classification-based time series forecasting methods.
method Cumulative Binary Encoding (CBE) and BinConv architecture.
result BinConv achieves superior performance in time series forecasting compared to existing methods.
Study combines VICReg and TNC for better encoding of non-stationary seismic signals.
problem Ineffective self-supervised learning on non-stationary time series.
method Combines VICReg and Temporal Neighborhood Coding (TNC).
result Effective for self-supervised learning on non-stationary seismic signals.
In this paper, we consider recommender systems with side information in the form of graphs. Existing collaborative filtering algorithms mainly utilize only immediate neighborhood information and have a hard time taking advantage of deeper neighborhoods beyond 1-2 hops. The main caveat of exploiting deeper graph informa…
Recurrent auto-encoder model summarises sequential data through an encoder structure into a fixed-length vector and then reconstructs the original sequence through the decoder structure. The summarised vector can be used to represent time series features. In this paper, we propose relaxing the dimensionality of the dec…
Time-related features improve time series forecasting models.
problem Lack of explicit time-related encoding in current forecasting models limits their ability to capture cyclical and seasonal trends.
method Introducing Time Stamp Forecaster (TimeSter) to encode time-related features and integrating it with a linear backbone.
result TimeLinear model reduces MSE by 23% on benchmark datasets, improving performance with exceptional efficiency.
Paper proposes GANs for generating business process suffixes and remaining times.
problem Generating accurate suffixes and remaining times for business process models.
method Encoder-decoder GAN architecture with Gumbel-Softmax distribution for categorical sequences.
result The approach outperforms baselines in suffix and remaining time prediction accuracy.
Time-aware deep learning methods improve spatial downscaling of atmospheric pollutants.
problem Transform coarse satellite data of atmospheric pollutants into high-resolution fields.
method Super-resolution deep residual networks and UNet architectures are extended with a temporal module encoding observation time.
result Temporal modules significantly improve downscaling performance and convergence speed.
We propose a conceptual design for a quantum blockchain. Our method involves encoding the blockchain into a temporal GHZ (Greenberger-Horne-Zeilinger) state of photons that do not simultaneously coexist. It is shown that the entanglement in time, as opposed to an entanglement in space, provides the crucial quantum adva…
Paper encodes textile structures and classifies them up to complexity five.
problem Topological classification of textile structures.
method Extends Gauss code to textile structures, introduces a linear time algorithm.
result First classification of all oriented textile structures up to complexity five.
Given a convolutional dictionary underlying a set of observed signals, can a carefully designed auto-encoder recover the dictionary in the presence of noise? We introduce an auto-encoder architecture, termed constrained recurrent sparse auto-encoder (CRsAE), that answers this question in the affirmative. Given an input…
Deep learning models have shown state-of-the-art performance in many inverse reconstruction problems. However, it is not well understood what properties of the latent representation may improve the generalization ability of the network. Furthermore, limited models have been presented for inverse reconstructions over ti…
We present Piano Genie, an intelligent controller which allows non-musicians to improvise on the piano. With Piano Genie, a user performs on a simple interface with eight buttons, and their performance is decoded into the space of plausible piano music in real time. To learn a suitable mapping procedure for this proble…
The paper describes fitting submanifolds to data using Sussmann's orbit theorem.
problem Fitting an immersed submanifold to random samples.
method Uses Sussmann's orbit theorem to ensure submanifold fitting. Reconstruction involves encoding times and decoding via flows of vector fields.
result A high-probability bound on excess risk for the reconstruction error.
Improved anomaly detection in time series data using kervolutional neural networks.
problem Anomaly detection in time series data.
method Mixed kervolutional and convolutional layers in a temporal auto-encoder.
result The mixed model detects anomalies more sensitively in time series data.
Semantic segmentation remains a computationally intensive algorithm for embedded deployment even with the rapid growth of computation power. Thus efficient network design is a critical aspect especially for applications like automated driving which requires real-time performance. Recently, there has been a lot of resea…
Future video prediction is an ill-posed Computer Vision problem that recently received much attention. Its main challenges are the high variability in video content, the propagation of errors through time, and the non-specificity of the future frames: given a sequence of past frames there is a continuous distribution o…
Paper introduces a new method for Transformers with linear complexity.
problem No efficient relative positional encoding for linear Transformer models.
method Stochastic Positional Encoding (SPE) that replaces classical RPE.
result SPE behaves like RPE and performs well on benchmarks.
Study clusters bank customers using LSTM and DTW.
problem Efficiently segmenting bank customers for targeted offers.
method Encoder-decoder LSTM network and Dynamic Time Warping (DTW).
result Hybrid method yields more accurate clusters.
This article presents the use of Answer Set Programming (ASP) to mine sequential patterns. ASP is a high-level declarative logic programming paradigm for high level encoding combinatorial and optimization problem solving as well as knowledge representation and reasoning. Thus, ASP is a good candidate for implementing p…
Model predicts travel time under rare conditions using a vector-space model.
problem Predicting travel time under rare temporal conditions (e.g., holidays, school vacations) is challenging due to limited historical data and other temporal changes.
method Presented a vector-space model for encoding rare temporal conditions, allowing coherent representation learning across different conditions.
result Increased performance for travel time prediction over different baselines when using the vector-space encoding for representing the temporal setting.
The (variational) graph auto-encoder and its variants have been popularly used for representation learning on graph-structured data. While the encoder is often a powerful graph convolutional network, the decoder reconstructs the graph structure by only considering two nodes at a time, thus ignoring possible interaction…
Proposes a THGNN for dynamic financial time series prediction.
problem Challenges in predicting stock market price movements.
method Temporal and heterogeneous graph neural network (THGNN) approach.
result Significantly improved prediction performance compared to state-of-the-art methods.
In this paper we propose a model that combines the strengths of RNNs and SGVB: the Variational Recurrent Auto-Encoder (VRAE). Such a model can be used for efficient, large scale unsupervised learning on time series data, mapping the time series data to a latent vector representation. The model is generative, such that …
RISE framework unifies and improves time series learning with missing data.
problem Learning from time series with missing data.
method RISE framework unifies and improves time series learning with missing data.
result RISE instances always benefit from encoders that learn representations for numerical values.
New approach decodes stock volatility states for S&P500 network.
problem Discovering multiple volatility states in S&P500 stock returns.
method Encoding-and-decoding approach using quantile-based thresholds and change point detection.
result Forecasting stock returns and revealing volatility dynamics.
FraudTransformer detects payment fraud by preserving event order and time gaps.
problem Detecting payment fraud in real-world banking streams with irregular time gaps.
method Augments a GPT-style architecture with a dedicated time encoder and a learned positional encoder.
result FraudTransformer outperforms classical and transformer baselines, achieving highest AUROC and PRAUC on held-out test set.
Financial fraud detection in digital banking requires reasoning over multiple heterogeneous event streams.
problem Financial fraud detection in digital banking requires reasoning over multiple heterogeneous event streams.
method Multi-Stream Fraud Transformer (MSFT) architecture that encodes each event stream with independent Transformer encoders and fuses their representations through configurable mechanisms.
result Sequence models significantly outperform gradient-boosted trees operating on aggregated features.
Quantum models can approximate any function if data encoding allows for a rich enough frequency spectrum.
problem Theoretical properties of quantum machine learning models, particularly their expressive power.
method Investigated how data encoding affects the expressive power of parametrized quantum circuits.
result Quantum models can access increasingly rich frequency spectra by repeating data encoding gates, potentially making them universal function approximators.
CVAE improves stock volume forecasting with advanced input variables.
problem Improving accuracy of daily stock volume forecasts.
method Conditional Variational Auto-Encoder (CVAE) with advanced input variables.
result CVAE generates non-linear forecasts with better accuracy and correlation to actual data.
Quantum computing for option pricing using MPS states.
problem Efficiently generating time series for path-dependent options on quantum computers.
method Proposes a Matrix Product State (MPS) model for time series generation and trains it for the Heston model.
result Demonstrates the MPS model's capability to generate paths in the Heston model for path-dependent option pricing.
The paper introduces walks with jumps for modeling neuron activity in hyperbolic space.
problem Encoding neuron activity sequences in hyperbolic space.
method Introducing walks with jumps in hyperbolic geometry to model neuron activity.
result Endpoints of walks with jumps do not fully encode the sequence of jump times.
A model learns stock trading rules from raw prices using encoder-decoder neural network.
problem Extracting features from long price sequences for profitable trading rules.
method Neural encoder-decoder framework combined with DRL.
result The model outperforms state-of-the-art models in dynamic environments.
New method for Transformer models to encode position information without sequential bias.
problem Lack of flexible and learnable position encoding for Transformer models.
method Continuous dynamical model to learn position encoding.
result Consistent improvements over baselines in various NLP tasks.