arXiv research

This thesis investigates unsupervised time series representation learning for sequence prediction problems, i.e. generating nice-looking input samples given a previous history, for high dimensional input sequences by decoupling the static input representation from the recurrent sequence representation. We introduce thr…

Anderson and Canary have shown that if the algebraic limit of a sequence of discrete, faithful representations of a finitely generated group into PSL(2,C) does not contain parabolics, then it is also the sequence's geometric limit. We construct examples that demonstrate the failure of this theorem for certain sequences…

Representation learning of pedestrian trajectories transforms variable-length timestamp-coordinate tuples of a trajectory into a fixed-length vector representation that summarizes spatiotemporal characteristics. It is a crucial technique to connect feature-based data mining with trajectory data. Trajectory representati…

Polynomial representations found in surface braid and mapping class groups.

In this paper, we propose a novel neural network model called RNN Encoder-Decoder that consists of two recurrent neural networks (RNN). One RNN encodes a sequence of symbols into a fixed-length vector representation, and the other decodes the representation into another sequence of symbols. The encoder and decoder of t…

Researchers use Gysin sequence to show sl(N) homology of T(2,m) is cohomology of SU(N) representations.

Multimodalities provide promising performance than unimodality in most tasks. However, learning the semantic of the representations from multimodalities efficiently is extremely challenging. To tackle this, we propose the Transformer based Cross-modal Translator (TCT) to learn unimodal sequence representations by trans…

New method learns low-dimensional representations of nonlinear time series without supervision.

We consider exact sequences and lower central series of surface braid groups and we explain how they can prove to be useful for obtaining representations for surface braid groups. In particular, using a completely algebraic framework, we describe the notion of extension of a representation introduced and studied recent…

This paper proposes a self-supervised learning approach for video features that results in significantly improved performance on downstream tasks (such as video classification, captioning and segmentation) compared to existing methods. Our method extends the BERT model for text sequences to the case of sequences of rea…

In this paper, we obtain stability results for martingale representations in a very general framework. More specifically, we consider a sequence of martingales each adapted to its own filtration, and a sequence of random variables measurable with respect to those filtrations. We assume that the terminal values of the m…

This belongs to a series of papers devoted to the study of the cohomology of classifying spaces of Lie groupoids. Our aim here is to introduce and study the notion of representation up to homotopy of Lie groupoids, the resulting derived category, and to show that the adjoint representation is well defined as a represen…

In this paper, we explore the ways to improve POS-tagging using various types of auxiliary losses and different word representations. As a baseline, we utilized a BiLSTM tagger, which is able to achieve state-of-the-art results on the sequence labelling tasks. We developed a new method for character-level word represen…

Let $Γ$ be the fundamental group of a complete hyperbolic $3$-manifold $M$ with toric cusps. We define the $ω$-Borel invariant $β_n^ω(ρ_ω)$ associated to a representation $ρ_ω: Γ\rightarrow SL(n,\mathbb{C}_ω)$, where $\mathbb{C}_ω$ is a field which can be constructed as a quotient of a suitable subset of $\mathbb{C}^\m…

Multimodal machine learning is a core research area spanning the language, visual and acoustic modalities. The central challenge in multimodal learning involves learning representations that can process and relate information from multiple modalities. In this paper, we propose two methods for unsupervised learning of j…

Recurrent models for sequences have been recently successful at many tasks, especially for language modeling and machine translation. Nevertheless, it remains challenging to extract good representations from these models. For instance, even though language has a clear hierarchical structure going from characters throug…

nTreeClus clusters categorical sequences using tree-based learners and k-mers.

Generative Distribution Embeddings learn multiscale representations of distributions.

We show that the Korevaar-Schoen limit of the sequence of equivariant harmonic maps corresponding to a sequence of irreducible $SL_2({\mathbb C})$ representations of the fundamental group of a compact Riemannian manifold is an equivariant harmonic map to an ${\mathbb R}$-tree which is minimal and whose length function …

Let M_g^n be the moduli space of Riemann surfaces of genus g with n labeled marked points. We prove that, for g \geq 2, the cohomology groups {H^i(M_g^n;Q)}_{n=1}^{\infty} form a sequence of Sn representations which is representation stable in the sense of Church-Farb [CF]. In particular this result applied to the triv…

Improves NF for complex data distributions with multiple modes.

This paper clarifies vine copula structures using graph and matrix representations.

Let $Δ_{L,ρ_n}(t)$ be the twisted Alexander polynomial with respect to the representation given by the composition of the lift of the holonomy representation of a certain hyperbolic link $L$ and the $n$-dimensional irreducible complex representation of $\text{SL}(2,\mathbb C)$. We consider a sequence of $Δ_{L,ρ_n}(t)$ …

Meta-learning improves event prediction from short sequences.

Proposes a novel model for healthcare and SME credit risk prediction.

IAE extracts innovations sequences for non-Gaussian processes.

Cataclysm deformations study Anosov representations and their convergence.

The softmax content-based attention mechanism has proven to be very beneficial in many applications of recurrent neural networks. Nevertheless it suffers from two major computational limitations. First, its computations for an attention lookup scale linearly in the size of the attended sequence. Second, it does not enc…

SNNs can represent complex functions efficiently.

Let C_n(M) be the configuration space of n distinct ordered points in M. We prove that if M is any connected orientable manifold (closed or open), the homology groups H_i(C_n(M); Q) are representation stable in the sense of [Church-Farb]. Applying this to the trivial representation, we obtain as a corollary that the un…

Cataclysm deformations study Anosov representations, leading to new formulas and non-open sets.

The effective representation of proteins is a crucial task that directly affects the performance of many bioinformatics problems. Related proteins usually bind to similar ligands. Chemical characteristics of ligands are known to capture the functional and mechanistic properties of proteins suggesting that a ligand base…

Given the fundamental group $Γ$ of a finite-volume complete hyperbolic $3$-manifold $M$, it is possible to associate to any representation $ρ:Γ\rightarrow \text{Isom}(\mathbb{H}^3)$ a numerical invariant called volume. This invariant is bounded by the hyperbolic volume of $M$ and satisfies a rigidity condition: if the …

Using the work of Bonahon-Dreyer and Fock-Goncharov, one can construct a real-analytic parameterization for the PSL(n,R) Hitchin component of a surface S, that is explicitly analogous to the Fenchel-Nielsen coordinates on the Teichmuller space of S. Given a Hitchin representation, we give a lower bound on the "length" …

Bi-directional LSTMs are a powerful tool for text representation. On the other hand, they have been shown to suffer various limitations due to their sequential nature. We investigate an alternative LSTM structure for encoding text, which consists of a parallel state for each word. Recurrent steps are used to perform lo…

The paper studies how neural networks evolve representations, finding a unique fixed point for nonlinear activations.

Programmatic Motion Concepts learn human actions from paired videos.

Hrrformer uses HRR to speed up self-attention for long sequences.

GTEA learns node representations in temporal interaction graphs.

Representation stability is a theory describing a way in which a sequence of representations of different groups is related, and essentially contains a finite amount of information. Starting with Church-Ellenberg-Farb's theory of $FI$-modules describing sequences of representations of the symmetric groups, we now have …

New algorithms extract low-dimensional representations from sequential data, revealing insights into complex processes.

Seq2Tens uses tensors to efficiently represent sequences, improving performance on time series and video tasks.

Sequential modelling with self-attention has achieved cutting edge performances in natural language processing. With advantages in model flexibility, computation complexity and interpretability, self-attention is gradually becoming a key component in event sequence models. However, like most other sequence models, self…

Representation stability is a phenomenon whereby the structure of certain sequences $X_n$ of spaces can be seen to stabilize when viewed through the lens of representation theory. In this paper I describe this phenomenon and sketch a framework, the theory of FI-modules, that explains the mechanism behind it.

Process Monitoring involves tracking a system's behaviors, evaluating the current state of the system, and discovering interesting events that require immediate actions. In this paper, we consider monitoring temporal system state sequences to help detect the changes of dynamic systems, check the divergence of the syste…

Recently, self-supervised learning has proved to be effective to learn representations of events suitable for temporal segmentation in image sequences, where events are understood as sets of temporally adjacent images that are semantically perceived as a whole. However, although this approach does not require expensive…

The Hitchin component is a connected component of the character variety of reductive group homomorphisms from the fundamental group of a closed surface S of genus greater than 1 to the Lie group PSL_m(R). The Teichmuller space of S naturally embeds into the Hitchin component. The limit points in the Thurston compactifi…

ATS2S model predicts RUL of industrial equipment using attention mechanism.