XL-Editor improves sentence post-editing using XLNet's variable-length insertion probability.
problem Post-editing sentences to refine generated text.
method XL-Editor trains XLNet to estimate variable-length insertion probabilities and apply post-editing operations.
result XL-Editor outperforms XLNet on text insertion and deletion tasks, and achieves significant style transfer improvements.
A new method predicts protein functions using variable-length sequences.
problem Computational methods for protein function prediction are slow and inaccurate for long sequences.
method Two feature sets: single fixed-sized segments and multi-sized segments, using bi-directional LSTM. Combined with MLDA features.
result Significant improvement in accuracy for long protein sequences.
A new multi-step model improves model-based reinforcement learning efficiency.
problem Expensive environmental interaction in reinforcement learning.
method Proposes a multi-step model for predicting action sequences with variable length.
result Multi-step model outperforms one-step model in preliminary tests.
A deep learning approach for clustering time series of varying lengths.
problem Clustering time series with variable lengths and temporal relations.
method Recurrent Deep Divergence-based Clustering framework.
result Outperforms previous methods on benchmark datasets.
We present the Latent Sequence Decompositions (LSD) framework. LSD decomposes sequences with variable lengthed output units as a function of both the input sequence and the output sequence. We present a training algorithm which samples valid extensions and an approximate decoding algorithm. We experiment with the Wall …
One of the ubiquitous representation of long DNA sequence is dividing it into shorter k-mer components. Unfortunately, the straightforward vector encoding of k-mer as a one-hot vector is vulnerable to the curse of dimensionality. Worse yet, the distance between any pair of one-hot vectors is equidistant. This is partic…
We investigate anomaly detection in an unsupervised framework and introduce Long Short Term Memory (LSTM) neural network based algorithms. In particular, given variable length data sequences, we first pass these sequences through our LSTM based structure and obtain fixed length sequences. We then find a decision functi…
Recurrent neural networks and sequence to sequence models require a predetermined length for prediction output length. Our model addresses this by allowing the network to predict a variable length output in inference. A new loss function with a tailored gradient computation is developed that trades off prediction accur…
This paper explores using a Long short-term memory (LSTM) based sequence autoencoder to learn interesting features for detecting surveillance aircraft using ADS-B flight data. An aircraft periodically broadcasts ADS-B (Automatic Dependent Surveillance - Broadcast) data to ground receivers. The ability of LSTM networks …
We describe our first-place solution to the Animal Behavior Challenge (ABC 2018) on predicting gender of bird from its GPS trajectory. The task consisted in predicting the gender of shearwater based on how they navigate themselves across a big ocean. The trajectories are collected from GPS loggers attached on shearwate…
Tree-based LSTM improves sequential regression with missing data.
problem Regression for variable-length sequential data with missing samples.
method Tree architecture of LSTM networks, selecting LSTM networks based on presence-pattern of previous inputs.
result Significant performance improvements on financial and real-life datasets.
A new kernel Stein test assesses fit for variable-length sequential data.
problem Evaluating goodness of fit for varying-dimensional data like text documents of different lengths.
method Extends kernel Stein discrepancy (KSD) to variable-dimension settings by identifying appropriate Stein operators and proposing a novel KSD goodness-of-fit test.
result The proposed test performs well on discrete sequential data benchmarks.
Universal perturbations misclassify text with high accuracy.
problem Vulnerability of text classifiers to small perturbations.
method Algorithm to compute universal adversarial perturbations.
result Deep neural networks are highly vulnerable to universal adversarial perturbations.
Paper tackles embedding attributed sequences in unsupervised learning.
problem Mining tasks over attributed sequences with dependencies between sequences and attributes.
method Proposes a deep multimodal learning framework, NAS, for unsupervised learning of attributed sequences.
result NAS produces task-independent embeddings for various mining tasks on real-world datasets.
Bayesian learning from variable-length sequences using Gaussian processes with signature covariances.
problem Learning from sequences of varying lengths and complex sequential structures.
method Gaussian processes with signature kernels, sparse variational approach, combining with LSTM/GRU models.
result Effective learning from sequences of different lengths and complex structures.
PRAGMA models financial event sequences for various banking tasks.
problem Handling diverse financial data for multiple applications.
method Pre-training a Transformer model on a large banking event corpus with a self-supervised objective.
result PRAGMA achieves superior performance across multiple financial domains from raw event sequences.
This paper studies mutual information in sequence models and finds Transformers excel in capturing long-range dependencies.
problem Understanding the expressive power of sequence models in capturing temporal dependencies.
method Theoretical and empirical analysis of linear and nonlinear RNNs, including Transformers.
result Transformers can capture long-range mutual information more efficiently than RNNs.
This study provides benchmarks for different implementations of LSTM units between the deep learning frameworks PyTorch, TensorFlow, Lasagne and Keras. The comparison includes cuDNN LSTMs, fused LSTM variants and less optimized, but more flexible LSTM implementations. The benchmarks reflect two typical scenarios for au…
Paper tackles variable-length, incomplete wearable sensor data to improve personalized insights.
problem Variable-length and incomplete time series data from wearable sensors.
method HeartSpace integrates a time series encoding module and pattern aggregation network, along with a Siamese-triplet network for representation learning.
result Empirical evaluation shows significant performance gains in personality prediction, demographics inference, and user identification.
A new smooth edit distance for easier optimization in machine learning.
problem Hard optimization of edit distance for variable-length sequences.
method Soft edit distance (SED) as a differentiable approximation.
result SED can be optimized with gradient methods and used for clustering.
Modelling the real world complexity of music is a challenge for machine learning. We address the task of modeling melodic sequences from the same music genre. We perform a comparative analysis of two probabilistic models; a Dirichlet Variable Length Markov Model (Dirichlet-VMM) and a Time Convolutional Restricted Boltz…
GAN-AE generates synthetic data for imbalanced sequence classification.
problem Imbalanced sequence classification in medical devices.
method Autoencoder and GAN architecture for generating synthetic data.
result GAN-AE outperforms other synthetic data generation methods.
We examine Deep Canonically Correlated LSTMs as a way to learn nonlinear transformations of variable length sequences and embed them into a correlated, fixed dimensional space. We use LSTMs to transform multi-view time-series data non-linearly while learning temporal relationships within the data. We then perform corre…
TREP learns pedestrian trajectories efficiently without needing full datasets.
problem Learning fixed-length vector representations of variable-length trajectories.
method Actor-critic sequence-to-sequence autoencoder with spatial-aware objective function.
result TREP efficiently learns trajectory representations without needing full datasets.
A new method for modeling event sequences with ambiguous timestamps.
problem Handling event sequences with variable timestamps and time-shifts.
method Time-discounting convolution with dynamic pooling.
result Efficiently models event dependencies and robust against timestamp uncertainty.
Unsupervised method learns universal embeddings for variable-length multivariate time series.
problem Challenges in learning representations for time series data due to varying lengths and sparse labeling.
method Combines causal dilated convolutions with triplet loss for time-based negative sampling.
result Demonstrates quality, transferability, and practicability of learned representations.
Introduces BWMD, a new distance measure for DNA and malware clustering.
problem Shortcomings of previous compression-based distance metrics.
method Embeds sequences into a fixed-length feature vector.
result Significantly improved clustering performance on larger malware corpora.
Deep models predict suicidal ideation from EHR and EMA data.
problem Predicting suicidal ideation from mixed data sources.
method Deep sequential models using RNNs for EHR and EMA data, attention schemes, and temporal alignment.
result EMA data improves recall to 67.78% from 48.13% using EHR data alone.
Researchers develop flexible kernels for biological sequences with guaranteed reliability.
problem Challenges in applying machine learning to biological sequences, including unreliable methods.
method Theoretical analysis and development of modified kernels to ensure reliability and accuracy.
result Developed kernels that are universal, characteristic, and metrize the space of distributions for biological sequences.
New linear models improve time series classification efficiency and interpretability.
problem Complex and inefficient classifiers limit interpretability and applicability to variable-length time series.
method Symbolic representations, multi-resolution, multi-domain, linear models.
result mtSS-SEQL+LR achieves similar accuracy to state-of-the-art methods but with lower time and memory usage.
This work learns shared word embeddings for acoustic and phonetic sequences.
problem Mapping variable-length acoustic and phonetic sequences to fixed-dimensional vectors.
method Weak supervision and binary classification task to predict word similarity.
result Best model achieves an F1 score of 0.95 for binary classification.
CompILE learns reusable segments from demonstrations for hierarchical task execution.
problem Learning reusable, variable-length segments of hierarchical behavior from demonstrations.
method Unsupervised, fully-differentiable sequence segmentation module for latent encoding and re-composition.
result Model generalizes to longer sequences and unseen environments, learns task boundaries and event encodings.
SummerTime summarizes variable-length time series for machine learning applications.
problem Classical machine learning methods struggle with variable-length time series data.
method Summarizes time series into a fixed-length feature vector using Gaussian Mixture Models (GMM).
result Improves classification and regression performance in physical activity analysis.
Bayesian method detects Markov order in network paths more reliably.
problem Detecting Markov order in constrained categorical sequences.
method Multi-order Bayesian modelling framework.
result Bayesian method detects correct Markov order more reliably than competing methods.
Attributes, such as metadata and profile, carry useful information which in principle can help improve accuracy in recommender systems. However, existing approaches have difficulty in fully leveraging attribute information due to practical challenges such as heterogeneity and sparseness. These approaches also fail to c…
Paper tackles model failures producing outputs of undesirable length.
problem Model failures producing outputs of undesirable length.
method Develops a differentiable proxy objective and a verification approach.
result Can produce outputs 50 times longer than input and prove output length below a certain size.
Generative model for TPPs using signatures and distributional discrepancies.
problem Limitations of signature methods for TPPs and lack of global sequence-level loss in neural models.
method Introduce interarrival embedding to lift jump paths to continuous paths of bounded variation, enabling signature methods for discrete event sequences. Develop sigTPP, a signature-based generative model trained on path-level loss.
result sigTPP achieves the best average rank across multiple metrics and outperforms or is within a standard error of the strongest baseline in 64% of dataset-metric pairs.
Latent Block-Diffusion Temporal Point Processes (LBDTPP) is a semi-autoregressive framework for generating asynchronous event sequences.
problem Generating asynchronous event sequences
method Latent Block-Diffusion Temporal Point Processes
result Outperforms state-of-the-art TPP baselines in both unconditional and conditional generation tasks
An unsupervised anomaly detection method for irregularly sampled time-series data.
problem Anomaly detection in irregularly sampled or missing valued time-series data.
method Uses LSTM networks with time modulation gates to extract temporal features and SVDD for anomaly labeling.
result Significantly outperforms standard approaches on real-life datasets.
New coding theorem shows achievable rate matches theoretical limit.
problem Unknown existence of encoders and decoders for RDPF.
method Used stochastic, variable-length codes to prove RDPF achievable.
result Achievable rate matches theoretical rate-distortion-perception function.
ALT transforms time series data for better classification.
problem Efficiently classifying time series data with varying temporal scales.
method ALT algorithm using variable-length shifted time windows.
result State-of-the-art performance with minimal computational overhead.
A new method for training generative models with sparse supervision.
problem Training deep generative models with sparse and varying supervision.
method Caffeinated Wake-Sleep (CWS) method, combining reweighted wake-sleep and teacher-forcing.
result The CWS method is robust to variable length supervision and performs well on various datasets.
A sequential classifier minimizes test samples for binary and multi-class classification.
problem Minimizing test samples for sequential classification with unknown distributions.
method Proposes a classifier for binary and multi-class problems, analyzing error probabilities and extending results.
result Significant advantage over non-sequential classifiers, achieving same exponents without rejection option.
Paper proposes FOFE for efficient WSD.
problem Word sense disambiguation (WSD) problem.
method Fixed-size ordinally forgetting encoding (FOFE) combined with FFNN.
result FOFE-based FFNN achieves comparable performance to state-of-the-art at lower cost.
Study on elastic curves with variable stiffness, derived from bending energy.
problem Modeling elastic wires with varying thickness.
method Derive Euler-Lagrange equations for curves with variable bending stiffness.
result Characterizations of elastic curves with variable stiffness.
Neural Diffusion Intensity Models simplify Cox processes inference.
problem Intractable nonparametric estimation and posterior inference of latent stochastic intensity in Cox processes.
method Variational framework using neural SDEs, with theoretical guarantee of ELBO maximization coinciding with maximum likelihood estimation.
result Accurate recovery of latent intensity dynamics and posterior paths with significant speedup.
CoSE models complex drawings by treating strokes as a collection that can be composed.
problem Modeling complex free-form structures like diagrams.
method Generative model using autoencoder to project strokes into a fixed latent space, relational model operates in latent space.
result Model captures both individual strokes and their compositional structure.
We study topology of configuration spaces of planar linkages having one leg of variable length. Such telescopic legs are common in modern robotics where they are used for shock absorbtion and serve a variety of other purposes. Using a Morse theoretic technique, we compute explicitly, in terms of the metric data, the Be…