arXiv research

Study on continuous sequence classification with distribution uncertainty.

Deep learning models optimize protein sequences.

OptiGAN uses GAN and RL to optimize sequence generation for specific goals.

Deep generative models have been successfully used to learn representations for high-dimensional discrete spaces by representing discrete objects as sequences and employing powerful sequence-based deep models. Unfortunately, these sequence-based models often produce invalid sequences: sequences which do not represent a…

Undirected neural sequence models such as BERT (Devlin et al., 2019) have received renewed interest due to their success on discriminative natural language understanding tasks such as question-answering and natural language inference. The problem of generating sequences directly from these models has received relativel…

FIGARO generates symbolic music with fine-grained control.

Generative adversarial nets (GAN) has been successfully introduced for generating text to alleviate the exposure bias. However, discriminators in these models only evaluate the entire sequence, which causes feedback sparsity and mode collapse. To tackle these problems, we propose a novel mechanism. It first segments th…

ProGen models protein sequences for synthetic biology.

Proves Mayer-Vietoris sequence for diffeological spaces using generating families.

This research proves guarantees on sequence models' generalization to longer and novel sequences.

The problem of universal outlying sequence detection is studied, where the goal is to detect outlying sequences among $M$ sequences of samples. A sequence is considered as outlying if the observations therein are generated by a distribution different from those generating the observations in the majority of the sequenc…

This article studies hypoellipticity on general filtered manifolds. We extend the Rockland criterion to a pseudodifferential calculus on filtered manifolds, construct a parametrix and describe its precise analytic structure. We use this result to study Rockland sequences, a notion generalizing elliptic sequences to fil…

Dynamical system models (including RNNs) often lack the ability to adapt the sequence generation or prediction to a given context, limiting their real-world application. In this paper we show that hierarchical multi-task dynamical systems (MTDSs) provide direct user control over sequence generation, via use of a latent…

We obtain an index of the complexity of a random sequence by allowing the role of the measure in classical probability theory to be played by a function we call the generating mechanism. Typically, this generating mechanism will be a finite automata. We generate a set of biased sequences by applying a finite state auto…

Deep learning techniques have been hugely successful for traditional supervised and unsupervised machine learning problems. In large part, these techniques solve continuous optimization problems. Recently however, discrete generative deep learning models have been successfully used to efficiently search high-dimensiona…

Generative model simulates financial market price variations from order flow.

This thesis investigates unsupervised time series representation learning for sequence prediction problems, i.e. generating nice-looking input samples given a previous history, for high dimensional input sequences by decoupling the static input representation from the recurrent sequence representation. We introduce thr…

Data of practical interest - such as personal records, transaction logs, and medical histories - are sequential collections of events relevant to a particular source entity. Recent studies have attempted to link sequences that represent a common entity across data sets to allow more comprehensive statistical analyses a…

Sequence-to-Sequence (seq2seq) modeling has rapidly become an important general-purpose NLP tool that has proven effective for many text-generation and sequence-labeling tasks. Seq2seq builds on deep neural language modeling and inherits its remarkable accuracy in estimating local, next-word distributions. In this work…

Randomized positional encodings boost transformer performance on longer sequences.

AutoGraph uses transformers to efficiently generate graphs as sequences.

The paper explores spectral sequences of complex manifolds with special metrics.

Generative models have long been the dominant approach for speech recognition. The success of these models however relies on the use of sophisticated recipes and complicated machinery that is not easily accessible to non-practitioners. Recent innovations in Deep Learning have given rise to an alternative - discriminati…

Magnetic resonance imaging (MRI) is being increasingly utilized to assess, diagnose, and plan treatment for a variety of diseases. The ability to visualize tissue in varied contrasts in the form of MR pulse sequences in a single scan provides valuable insights to physicians, as well as enabling automated systems perfor…

In this paper, we give a lower bound of Bergman kernels for a sequence of almost Kähler-Einstein Fano manifolds, or more general, a sequence of Fano manifolds with almost Kähler-Ricci solitons. This generalizes a result by Donaldson-Sun, Tian for Kähler-Einstein manifolds sequence with positive scalar curvature. As an …

Study connections on Lie groupoids and stacks using Atiyah sequences.

A graph embedding is a representation of graph vertices in a low-dimensional space, which approximately preserves properties such as distances between nodes. Vertex sequence-based embedding procedures use features extracted from linear sequences of nodes to create embeddings using a neural network. In this paper, we pr…

Enhances sequence memory capacity in neural networks.

Sequence classification is an important data mining task in many real world applications. Over the past few decades, many sequence classification methods have been proposed from different aspects. In particular, the pattern-based method is one of the most important and widely studied sequence classification methods in …

The celebrated Sequence to Sequence learning (Seq2Seq) technique and its numerous variants achieve excellent performance on many tasks. However, many machine learning tasks have inputs naturally represented as graphs; existing Seq2Seq models face a significant challenge in achieving accurate conversion from graph form …

We study the Thompson sampling algorithm in an adversarial setting, specifically, for adversarial bit prediction. We characterize the bit sequences with the smallest and largest expected regret. Among sequences of length $T$ with $k < \frac{T}{2}$ zeros, the sequences of largest regret consist of alternating zeros and …

For any prime power $q$ and any dimension $s$, a new construction of $(t,s)$-sequences in base $q$ using global function fields is presented. The construction yields an analog of Halton sequences for global function fields. It is the first general construction of $(t,s)$-sequences that is not based on the digital metho…

Generative Adversarial Networks (GANs) have been used in many different applications to generate realistic synthetic data. We introduce a novel GAN with Autoencoder (GAN-AE) architecture to generate synthetic samples for variable length, multi-feature sequence datasets. In this model, we develop a GAN architecture with…

Algorithm optimizes biological sequences using bootstrapped training with a score-conditioned generator.

Given the emerging global threat of antimicrobial resistance, new methods for next-generation antimicrobial design are urgently needed. We report a peptide generation framework PepCVAE, based on a semi-supervised variational autoencoder (VAE) model, for designing novel antimicrobial peptide (AMP) sequences. Our model l…

This paper proposes an EM approach to reduce inference latency in NAR sequence generation.

Study Palais-Smale sequences for Ricci curvature on homogeneous spaces.

Let T be a torus. We present an exact sequence relating the relative equivariant cohomologies of the skeletons of an equivariantly formal T-space. This sequence, which goes back to Atiyah and Bredon, generalizes the so-called Chang-Skjelbred lemma. As coefficients, we allow prime fields and subrings of the rationals, i…

Neural Machine Translation (NMT) generates target words sequentially in the way of predicting the next word conditioned on the context words. At training time, it predicts with the ground truth words as context while at inference it has to generate the entire sequence from scratch. This discrepancy of the fed context l…

Traditional pairwise sequence alignment is based on matching individual samples from two sequences, under time monotonicity constraints. However, in many application settings matching subsequences (segments) instead of individual samples may bring in additional robustness to noise or local non-causal perturbations. Thi…

We study the problem of generating interesting endings for stories. Neural generative models have shown promising results for various text generation problems. Sequence to Sequence (Seq2Seq) models are typically trained to generate a single output sequence for a given input sequence. However, in the context of a story,…

New algorithm minimizes expert selection regret in partial bandit feedback.

Deep conditional generative models are developed to simultaneously learn the temporal dependencies of multiple sequences. The model is designed by introducing a three-way weight tensor to capture the multiplicative interactions between side information and sequences. The proposed model builds on the Temporal Sigmoid Be…

This work compresses sequences by treating them as continuous-time processes, enabling efficient discretization.

Transformers can simulate MLE for Bayesian network sequences.

This paper proposes a generative model, the latent Dirichlet hidden Markov models (LDHMM), for characterizing a database of sequential behaviors (sequences). LDHMMs posit that each sequence is generated by an underlying Markov chain process, which are controlled by the corresponding parameters (i.e., the initial state …

Graphs from van der Corput sequence embed into Chamanara surface.

For any prime power $q$ and any dimension $s \ge 1$, we present a construction of $(t,s)$-sequences in base $q$ with finite-row generating matrices such that, for fixed $q$, the quality parameter $t$ is asymptotically optimal as a function of $s$ as $s \to \infty$. This is the first construction of $(t,s)$-sequences th…