Meta-learning improves event prediction from short sequences.
problem Predicting events from short sequences is challenging.
method Meta-learning approach using recurrent neural networks and monotonic neural networks.
result Meta-learning enhances long-term prediction performance.
Bandit structured prediction describes a stochastic optimization framework where learning is performed from partial feedback. This feedback is received in the form of a task loss evaluation to a predicted output structure, without having access to gold standard structures. We advance this framework by lifting linear ba…
Improves NMT by sampling context from predicted sequence during training.
problem Error accumulation and overcorrection in NMT due to mismatched training and inference contexts.
method Samples context words from both ground truth and predicted sequences during training.
result Significant improvements on multiple datasets, including Chinese->English and WMT'14 English->German.
A new neural sequence prediction method using error-correcting codes improves accuracy and convergence.
problem Improving neural sequence prediction accuracy and speed.
method Error-correcting output codes, separable code maximization, latent variable mixture sampling.
result Consistent improvements on language modeling datasets and text generation tasks.
As high-throughput biological sequencing becomes faster and cheaper, the need to extract useful information from sequencing becomes ever more paramount, often limited by low-throughput experimental characterizations. For proteins, accurate prediction of their functions directly from their primary amino-acid sequences h…
Machine-assisted treatment recommendations hold a promise to reduce physician time and decision errors. We formulate the task as a sequence-to-sequence prediction model that takes the entire time-ordered medical history as input, and predicts a sequence of future clinical procedures and medications. It is built on the …
We describe a fully data driven model that learns to perform a retrosynthetic reaction prediction task, which is treated as a sequence-to-sequence mapping problem. The end-to-end trained model has an encoder-decoder architecture that consists of two recurrent neural networks, which has previously shown great success in…
Recurrent neural networks and sequence to sequence models require a predetermined length for prediction output length. Our model addresses this by allowing the network to predict a variable length output in inference. A new loss function with a tailored gradient computation is developed that trades off prediction accur…
CTCModel extends Keras for transparent CTC classification.
problem Handling unsegmented input sequences with labels related to subsets of frames.
method Combines Keras and CTC implementation in Tensorflow backend.
result CTCModel predicts sequences of labels from unsegmented input.
Deep network predicts action sequences for complex tasks from a scene image.
problem Scalable task and motion planning from initial scene images.
method Deep convolutional recurrent neural network that predicts action sequences.
result Predicts promising action sequences, reducing motion planning problems.
Predicts cryptocurrency pump probability using sequence-based neural networks.
problem Detecting pump-and-dump schemes in cryptocurrency markets.
method Developed a sequence-based neural network (SNN) that encodes historical P&D events into sequences for prediction.
result SNN improves prediction accuracy by leveraging positional attention to extract useful information.
Improved multi-step prediction of drivable space for autonomous vehicles.
problem Accurate prediction of drivable space for safer, more comfortable navigation.
method Recurrent Neural Network (RNN) architectures trained on KITTI dataset, incorporating motion features.
result Significant improvement in prediction accuracy over state-of-the-art methods.
Improved neural models for diverse user event sequences.
problem Challenges in modeling diverse user event sequences.
method Mixtures of latent embeddings with amortized variational inference.
result Systematic improvements over existing work for various predictive metrics.
New method embeds RNN Seq2Seq models to visualize spatiotemporal data.
problem Visualizing and interpreting spatiotemporal data in sequence prediction tasks.
method Embedding approach to visualize and interpret RNN Seq2Seq model representations.
result Embedding space projections of RNN Seq2Seq models capture spatiotemporal dynamics.
EENNs improve inference efficiency but need nested prediction sets for reliable uncertainty estimates.
problem Non-nested prediction sets from standard uncertainty quantification methods in EENNs.
method Introduced anytime-valid confidence sequences (AVCSs) tailored for EENNs.
result AVCSs generate nested prediction sets across EENN exits, addressing the issue of non-nested sets.
Neural networks predict TED Talk ratings from transcripts, removing bias.
problem Predicting public speaking performance from speech transcripts.
method Causal diagram modeling, word sequence and dependency tree based neural networks.
result Average F-score of 0.77, significantly outperforming baseline methods.
RNA-binding proteins (RBPs) play crucial roles in many biological processes, e.g. gene regulation. Computational identification of RBP binding sites on RNAs are urgently needed. In particular, RBPs bind to RNAs by recognizing sequence motifs. Thus, fast locating those motifs on RNA sequences is crucial and time-efficie…
STConvS2S improves weather forecasting using only convolutional layers.
problem Predicting future weather conditions more accurately.
method Proposes a deep learning architecture combining spatiotemporal convolutional layers.
result Outperforms state-of-the-art architectures for forecasting tasks.
Self-training improves neural sequence generation by correcting incorrect predictions.
problem Improving neural sequence generation models using unlabeled data.
method Injecting pseudo-parallel data (model predictions) into the labeled dataset and using dropout as a regularizer.
result Noisy self-training significantly improves performance on machine translation and text summarization benchmarks.
Model predicts multi-modal sequences using N-curves.
problem Capturing multi-modal data in sequential data.
method Neural network model based on Mixture Density Networks with Bézier curves.
result Smooth multi-mode predictions without Monte Carlo simulation.
The paper improves polyphonic music models by extracting salient features.
problem Improving performance of polyphonic music models.
method Training a neural network to predict both chords and notes, including extra features.
result Training a neural network to predict extra features improves model performance.
Neural network uses local credit assignment to learn distant cause-effect relationships.
problem Learning distant cause-effect relationships in sequential data.
method Sparse coding in a recurrent neural network memory with local and immediate credit assignment.
result Network can predict partially-observable higher-order sequences and navigate mazes.
Tail-GNNs improve protein function prediction using relational reinforcement.
problem Predicting hierarchical protein functions from sequence data.
method Combining Tail-GNNs with dilated convolutional networks for multi-task learning.
result Significant improvement in F_1 score for protein function prediction.
PF-RNNs use particle filtering to model uncertainty in RNNs for better sequential data prediction.
problem Highly variable and noisy sequential data.
method PF-RNNs maintain a latent state distribution as a set of particles, updating with Bayes rule.
result PF-RNNs outperform standard RNNs on various sequence prediction tasks.
DVE models dynamic changes in feature embeddings for better sequence-aware applications.
problem Dynamic characterization of feature variations in time-varying data.
method Dynamic Variational Embedding (DVE) using recurrent neural networks.
result DVE models intrinsic nature and temporal variation of nodes effectively.
Develops a new point process model for detecting neural spike sequences.
problem Detecting sparse sequences of neural spikes in high-dimensional spike trains.
method A point process model that represents sequence occurrences as marked events in continuous time, with learnable time warping parameters.
result Demonstrates improved detection and modeling of neural spike sequences.
In this work, we study the credit assignment problem in reward augmented maximum likelihood (RAML) learning, and establish a theoretical equivalence between the token-level counterpart of RAML and the entropy regularized reinforcement learning. Inspired by the connection, we propose two sequence prediction algorithms, …
Novel neural operator predicts complex spatiotemporal dynamics from partial observations.
problem Capturing complex operator dynamics in infinite-dimensional function spaces.
method Integrates Koopman operator theory with deep neural networks to approximate nonlinear operators between Banach spaces.
result BNO achieves robust zero-shot super-resolution in unsteady flow prediction and outperforms conventional methods.
In this paper, we show that standard feed-forward and recurrent neural networks fail to learn abstract patterns based on identity rules. We propose Relation Based Pattern (RBP) extensions to neural network structures that solve this problem and answer, as well as raise, questions about integrating structures for induct…
A comparison of SLDS and LSTM for pedestrian behavior prediction shows SLDS works better with shorter sequences.
problem Time-critical pedestrian behavior prediction in autonomous vehicles.
method Comparison of a switching linear dynamical system (SLDS) and a three-layered bi-directional LSTM neural network.
result SLDS achieves higher accuracy with shorter sequences (10 samples) compared to LSTM's 80% accuracy with 100 samples.
MutaGAN predicts mutations of evolving protein populations using GANs.
problem Predicting mutations in evolving protein populations.
method Generative adversarial networks (GANs) with recurrent neural networks (RNNs).
result MutaGAN generates complete protein sequences with mutations.
SEISM tests neural network features for regulatory genomics.
problem Testing neural network features for regulatory genomics.
method Selective inference procedure for sequence motifs.
result Sampling under specific parameters characterizes composite null hypothesis.
Paper proposes AR model for graph sequences.
problem Predicting sequences of graphs with variable topology.
method Formalizes AR model for graphs using GNN to learn and predict next graph.
result Significantly better performance on synthetic graph generation problems.
CGAN fails to improve deterministic sequence predictions, revealing a theoretical limitation.
problem Improving deterministic sequence predictions with CGAN.
method Developed an adversarial content loss approach.
result CGAN does not improve deterministic sequence predictions.
THP model captures complex dependencies in event sequences efficiently.
problem Inability of existing models to capture long-term dependencies in event sequences.
method Transformer Hawkes Process (THP) model using self-attention mechanism.
result THP outperforms existing models in likelihood and event prediction accuracy.
Recurrent neural networks (RNNs) are types of artificial neural networks (ANNs) that are well suited to forecasting and sequence classification. They have been applied extensively to forecasting univariate financial time series, however their application to high frequency trading has not been previously considered. Thi…
In this paper, we compare different types of Recurrent Neural Network (RNN) Encoder-Decoders in anomaly detection viewpoint. We focused on finding the model that can learn the same data more effectively. We compared multiple models under the same conditions, such as the number of parameters, optimizer, and learning rat…
Spacetimeformer learns spatiotemporal relationships from data alone.
problem Forecasting multivariate time series with distinct spatial relationships.
method Transformers with dynamic graph connections learning interactions between space, time, and value.
result Competitive results on various time series prediction benchmarks.
QRNN uses quantum neurons to learn sequences efficiently.
problem Efficiently learning sequences with quantum computing.
method Parametrized quantum neurons and amplitude amplification.
result QRNN outperforms classical RNNs on sequence learning tasks.
Neural Physicist learns physical dynamics from images.
problem Learning meaningful physical state representations and accurate state transitions from image sequences.
method Neural Physicist uses VAE for state extraction, NP for parameters, and SSM for dynamics.
result Achieves long-term predictions and identifies system degrees of freedom.
Seq-SetNet processes sequence sets directly, improving protein structure prediction.
problem Processing sequence sets (MSAs) for structural inference without considering sequence order.
method Developed a symmetric function module to integrate features from MSAs.
result Seq-SetNet outperforms state-of-the-art approaches by 3.6% in precision.
In order to alleviate data sparsity and overfitting problems in maximum likelihood estimation (MLE) for sequence prediction tasks, we propose the Generative Bridging Network (GBN), in which a novel bridge module is introduced to assist the training of the sequence prediction model (the generator network). Unlike MLE di…
Stable long-term predictions for fluid flows using neural networks.
problem Predicting complex dynamics of fluid flows with high temporal stability.
method End-to-end trained neural network architecture combining CNN for spatial compression and LSTM for temporal prediction.
result Novel latent space subdivision (LSS) allows stable and controllable long-term predictions.
This paper uses LSTM and GRU models to forecast EBIT/EV for better stock selection.
problem Selecting stocks based on EBIT/EV for long-term investors.
method Sequence prediction models (LSTM and GRU) to forecast EBIT/EV.
result GRU model outperforms LSTM model in forecasting EBIT/EV.
Deep Convolutional Neural Networks (DCNN) has shown excellent performance in a variety of machine learning tasks. This manuscript presents Deep Convolutional Neural Fields (DeepCNF), a combination of DCNN with Conditional Random Field (CRF), for sequence labeling with highly imbalanced label distribution. The widely-us…
CloudLSTM forecasts geospatial point-cloud data streams accurately.
problem Forecasting over geospatial point-cloud data streams.
method Introduces CloudLSTM, a recurrent neural model with a Dynamic Point-cloud Convolution (DConv) operator.
result CloudLSTM outperforms competitor models in long-term predictions for point-cloud data.
Deep neural network translates math formula images to LaTeX sequences.
problem Translating math formula images to LaTeX sequences accurately and efficiently.
method Encoder-decoder architecture with CNN and LSTM, sequence-level training with policy gradient.
result State-of-the-art performance on sequence-based and image-based evaluation metrics.
This paper applies deep learning to ordinal regression, modeling it as a binary search.
problem Ordinal regression with deep learning models.
method Formulated ordinal regression as a binary search problem, using recurrent neural networks.
result Deep learning model shows comparable or better predictive power compared to traditional methods.