Deep learning models optimize protein sequences.
problem Optimizing protein properties through sequence design.
method Deep generative models guided by machine learning.
result Improved protein sequence generation from prior knowledge.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Deep Sequence Models”
Deep learning techniques have been hugely successful for traditional supervised and unsupervised machine learning problems. In large part, these techniques solve continuous optimization problems. Recently however, discrete generative deep learning models have been successfully used to efficiently search high-dimensiona…
Deep generative models have been successfully used to learn representations for high-dimensional discrete spaces by representing discrete objects as sequences and employing powerful sequence-based deep models. Unfortunately, these sequence-based models often produce invalid sequences: sequences which do not represent a…
Paper proposes using LSTM for LSH-based sequence alignment.
problem Sequence alignment using deep learning models.
method Deep bidirectional LSTM for feature learning and LSH-based sequence alignment.
result Higher accuracy achieved with LSTM-based model.
A novel sequence-to-sequence model predicts missing sensor data.
problem Missing sensor data in sequences.
method Formulated a novel sequence-to-sequence model using forward and backward RNNs.
result The model produces the lowest errors in 12% more cases than the current state-of-the-art.
This work compresses sequences by treating them as continuous-time processes, enabling efficient discretization.
problem Efficient compression of sequences, especially with deep learning models that scale with sequence length.
method Treat sequences as continuous-time processes, learn efficient discretization, and decode at different time intervals.
result Automatic bit rate reductions in video and motion capture sequences using learned discretization.
The energy output a photo voltaic(PV) panel is a function of solar irradiation and weather parameters like temperature and wind speed etc. A general measure for solar irradiation called Global Horizontal Irradiance (GHI), customarily reported in Watt/meter, is a generic indicator for this intermittent energy resour…
In this paper, a new deep reinforcement learning based augmented general sequence tagging system is proposed. The new system contains two parts: a deep neural network (DNN) based sequence tagging model and a deep reinforcement learning (DRL) based augmented tagger. The augmented tagger helps improve system performance …
Deep learning model forecasts PV power production with high accuracy.
problem Accurate forecasting of photovoltaic power production is needed for electricity infrastructure.
method Sequence to sequence model with attention mechanism using weather predictions and historical data.
result The model outperforms existing methods in forecast skill score.
Bayesian-Deep Learning model predicts Covid-19 evolution in Spain.
problem Estimating the future evolution of Covid-19 in Spain.
method Combines DL techniques with Bayesian Poisson-Gamma model for counts.
result Allows prediction of future evolution and estimation of scenarios.
Deep metric learning employs deep neural networks to embed instances into a metric space such that distances between instances of the same class are small and distances between instances from different classes are large. In most existing deep metric learning techniques, the embedding of an instance is given by a featur…
This thesis investigates unsupervised time series representation learning for sequence prediction problems, i.e. generating nice-looking input samples given a previous history, for high dimensional input sequences by decoupling the static input representation from the recurrent sequence representation. We introduce thr…
Motivation: Prediction of the interaction affinity between proteins and compounds is a major challenge in the drug discovery process. WideDTA is a deep-learning based prediction model that employs chemical and biological textual sequence information to predict binding affinity. Results: WideDTA uses four text-based inf…
A novel deep probabilistic model for dynamic systems forecasting.
problem Probabilistic forecasting in dynamic systems.
method Combining deep generative models and state space models with recurrent neural networks and variational sequence models.
result Outperforms existing models in system identification benchmarks and real-world centrifugal compressor forecasting.
k-Nearest Neighbors is one of the most fundamental but effective classification models. In this paper, we propose two families of models built on a sequence to sequence model and a memory network model to mimic the k-Nearest Neighbors model, which generate a sequence of labels, a sequence of out-of-sample feature vecto…
Differentiable language model attacks improve adversarial examples for categorical sequence classifiers.
problem Challenges in adversarial attacks for categorical sequence models due to non-differentiability.
method Fine-tuning a language model as a generator of adversarial examples, using a differentiable loss function that combines a surrogate classifier score and approximate edit distance.
result Semantically better and more resistant adversarial examples.
FIGARO generates symbolic music with fine-grained control.
problem Minimal control over generated music sequences.
method Description-to-sequence task, learning conditional distribution of sequences given high-level descriptions.
result State-of-the-art controllable symbolic music generation.
Deep learning models forecast stock market orders over multiple time frames.
problem Forecasting stock market orders over varying time frames.
method Encoder-decoder models with sequence-to-sequence and Attention mechanisms, leveraging Intelligent Processing Units (IPUs) for faster training.
result Multi-horizon forecasting outperforms single-horizon models, especially for long prediction periods.
Learn2Extend: Extending sequences by retaining their statistical properties with mixture modelscs.LG
The paper extends sequences while preserving statistical properties using a mixture model.
problem Extending sequences while retaining their statistical properties.
method Auto-regressive Sequence Extension Mixture Model (SEMM) using deep learning.
result The mixture model outperforms traditional neural networks in sequence extension with statistical property retention.
The availability of large amounts of time series data, paired with the performance of deep-learning algorithms on a broad class of problems, has recently led to significant interest in the use of sequence-to-sequence models for time series forecasting. We provide the first theoretical analysis of this time series forec…
S2P2 model improves predictive likelihoods for MTPPs.
problem Modeling irregular time intervals in event sequences.
method State-space point process model using deep state-space techniques.
result Empirically, S2P2 achieves state-of-the-art predictive likelihoods.
We propose generative neural network methods to generate DNA sequences and tune them to have desired properties. We present three approaches: creating synthetic DNA sequences using a generative adversarial network; a DNA-based variant of the activation maximization ("deep dream") design method; and a joint procedure wh…
In recent times, sequence-to-sequence (seq2seq) models have gained a lot of popularity and provide state-of-the-art performance in a wide variety of tasks such as machine translation, headline generation, text summarization, speech to text conversion, and image caption generation. The underlying framework for all these…
Synergetic Learning of Heterogeneous Temporal Sequences for Multi-Horizon Probabilistic Forecastingcs.LG
Proposes a model for multi-horizon probabilistic forecasting of time series influenced by asynchronous events.
problem Forecasting time series influenced by asynchronous events is challenging.
method Introduces Variational Synergetic Multi-Horizon Network (VSMHN), a deep conditional generative model combining deep point processes and variational recurrent neural networks.
result Produces accurate, sharp, and realistic probabilistic forecasts.
This thesis improves deep sequence models by integrating probabilistic methods for uncertainty quantification.
problem Lack of uncertainty awareness in deep sequence models limits their deployment.
method Develops approximate Bayesian inference methods for Transformers and HiPPOs, leveraging inductive biases.
result Improves predictive and generative performance of deep sequence models by incorporating probabilistic structures.
Survey on modeling event sequences through temporal processes.
problem Modeling phenomena with sequences of events over continuous time.
method Probabilistic models based on point processes, categorized into simple, marked, and spatio-temporal.
result Analysis of existing approaches and their applicability to prediction and modeling.
Proposes a novel BO algorithm for stepwise model selection in sequence prediction.
problem Optimal model selection in sequential settings where model performance varies over time.
method Bayesian optimization with deep kernel learning to handle multiple black-box functions.
result Outperforms standard and multi-objective BO algorithms on sequence prediction tasks.
This paper applies deep learning to ordinal regression, modeling it as a binary search.
problem Ordinal regression with deep learning models.
method Formulated ordinal regression as a binary search problem, using recurrent neural networks.
result Deep learning model shows comparable or better predictive power compared to traditional methods.
Missing value imputation is a fundamental problem in spatiotemporal modeling, from motion tracking to the dynamics of physical systems. Deep autoregressive models suffer from error propagation which becomes catastrophic for imputing long-range sequences. In this paper, we take a non-autoregressive approach and propose …
Deep RL optimizes compiler passes for better performance.
problem Designing optimal compiler optimization sequences is hard and often suboptimal.
method Employed deep reinforcement learning to learn optimal pass orderings.
result Achieved up to 1.32x speedup on previously unseen programs.
Gradient-based adversarial attacks on categorical sequence models via traversing an embedded worldcs.LG
New methods tackle adversarial attacks on categorical sequences, improving model security.
problem Adversarial attacks on categorical sequence models, especially for money transactions and medical fraud.
method Two black-box adversarial attacks: Monte-Carlo and continuous relaxation methods.
result Generated adversarial sequences fool machine learning models but remain close to original ones.
The paper provides a theoretical justification for using stable SSM blocks in deep sequential models.
problem Developing generalization bounds for deep sequential models with varying sequence lengths.
method Using Rademacher contraction and stability constraints, the paper derives a PAC bound that is independent of sequence length.
result The derived PAC bound decreases as the stability of SSM blocks increases, providing theoretical justification for their use.
New deep learning method outperforms random training data.
problem Improving accuracy of deep learning algorithms in high dimensions.
method Training with low-discrepancy sequences instead of random data.
result Significantly outperforms standard deep learning algorithms.
State of the art music recommender systems mainly rely on either matrix factorization-based collaborative filtering approaches or deep learning architectures. Deep learning models usually use metadata for content-based filtering or predict the next user interaction by learning from temporal sequences of user actions. D…
Deep learning model predicts traffic flows across entire network for multiple steps ahead.
problem Accurately forecasting future traffic flows across all network links.
method Spatial-Temporal Sequence to Sequence (STSeq2Seq) model combining seq2seq and graph convolution.
result STSeq2Seq achieves state-of-the-art performance in traffic forecasting.
ATS2S model predicts RUL of industrial equipment using attention mechanism.
problem Accurate estimation of RUL for industrial equipment to improve maintenance schedules and reduce costs.
method ATS2S model that optimizes reconstruction and RUL prediction losses, uses attention mechanism, and integrates encoder and decoder features.
result ATS2S model achieves superior performance over 13 state-of-the-art methods on four real datasets.
Constrained sequence codes have been widely used in modern communication and data storage systems. Sequences encoded with constrained sequence codes satisfy constraints imposed by the physical channel, hence enabling efficient and reliable transmission of coded symbols. Traditional encoding and decoding of constrained …
Applying machine learning models to meteorological data brings many opportunities to the Geosciences field, such as predicting future weather conditions more accurately. In recent years, modeling meteorological data with deep neural networks has become a relevant area of investigation. These works apply either recurren…
We present a new approach to modeling sequential data: the deep equilibrium model (DEQ). Motivated by an observation that the hidden layers of many existing deep sequence models converge towards some fixed point, we propose the DEQ approach that directly finds these equilibrium points via root-finding. Such a method is…
The functions of proteins and RNAs are determined by a myriad of interactions between their constituent residues, but most quantitative models of how molecular phenotype depends on genotype must approximate this by simple additive effects. While recent models have relaxed this constraint to also account for pairwise in…
Sequence-to-Sequence (seq2seq) modeling has rapidly become an important general-purpose NLP tool that has proven effective for many text-generation and sequence-labeling tasks. Seq2seq builds on deep neural language modeling and inherits its remarkable accuracy in estimating local, next-word distributions. In this work…
Computational identification of promoters is notoriously difficult as human genes often have unique promoter sequences that provide regulation of transcription and interaction with transcription initiation complex. While there are many attempts to develop computational promoter identification methods, we have no reliab…
IAE extracts innovations sequences for non-Gaussian processes.
problem Extracting innovations sequences for non-Gaussian processes.
method Causal convolutional neural network.
result IAE effectively detects anomalies in non-Gaussian data.
As deep learning techniques advance more than ever, hyper-parameter optimization is the new major workload in deep learning clusters. Although hyper-parameter optimization is crucial in training deep learning models for high model performance, effectively executing such a computation-heavy workload still remains a chal…
This paper reviews deep time-series forecasting focusing on autocorrelation modeling.
problem Modeling autocorrelation in history and label sequences for time-series forecasting.
method Proposes a novel taxonomy for model architectures and learning objectives.
result Provides a comprehensive review and analysis of deep time-series forecasting.
Deep dynamic generative models are developed to learn sequential dependencies in time-series data. The multi-layered model is designed by constructing a hierarchy of temporal sigmoid belief networks (TSBNs), defined as a sequential stack of sigmoid belief networks (SBNs). Each SBN has a contextual hidden state, inherit…
In this paper we propose a deep neural network model with an encoder-decoder architecture that translates images of math formulas into their LaTeX markup sequences. The encoder is a convolutional neural network (CNN) that transforms images into a group of feature maps. To better capture the spatial relationships of mat…
Deep recurrent neural networks perform well on sequence data and are the model of choice. However, it is a daunting task to decide the structure of the networks, i.e. the number of layers, especially considering different computational needs of a sequence. We propose a layer flexible recurrent neural network with adapt…