New method learns disentangled discrete representations using categorical variational autoencoders.
problem Learning disentangled representations from discrete latent spaces.
method Replaced standard Gaussian VAE with a categorical VAE to mitigate rotational invariance.
result Categorical distributions improve learning of disentangled representations.
A new method learns discrete representations for images and videos, improving upon previous models.
problem Learning discrete representations for images and videos to improve performance.
method Depthwise application of Vector Quantized Variational Autoencoders (VQVAE) to feature axis.
result 33% improvement in performance compared to previous discrete models.
Learning discrete representations of data is a central machine learning task because of the compactness of the representations and ease of interpretation. The task includes clustering and hash learning as special cases. Deep neural networks are promising to be used because they can model the non-linearity of data and s…
This paper introduces lattice representations for efficient discrete learning.
problem Efficient learning of discrete representations in Euclidean space.
method Lattice quantization and novel algorithms for efficient learning.
result New mathematical result linking training and inference expressions.
Improved training for VQ-VAE models with robust codebook learning.
problem Challenges in training discrete latent variable models, especially VQ-VAEs.
method Increased learning rate and periodic re-initialization of codebook for robust training.
result More robust training and increased usage of latent codewords, even for large codebooks.
Novel SVAE learns interpretable discrete data representations from deep learning.
problem Learning interpretable discrete data representations from deep learning.
method Structured variational autoencoder (SVAE) with novel optimization algorithms.
result First competitive comparisons with state-of-the-art time series models.
We present a framework for learning disentangled and interpretable jointly continuous and discrete representations in an unsupervised manner. By augmenting the continuous latent distribution of variational autoencoders with a relaxed discrete distribution and controlling the amount of information encoded in each latent…
A new method uncovers discrete and continuous factors in gene expression data.
problem Jointly identifying discrete and continuous factors of variability without supervision.
method cpl-mixVAE framework using multiple interacting networks.
result The method successfully uncovers discrete and continuous factors in gene expression data.
New representations for discrete surfaces derived from dual transforms.
problem Constructing discrete surfaces in differential geometry.
method Using Ω-dual transform and lightlike Gauss maps in Laguerre geometry. result All discrete linear Weingarten surfaces arise via Weierstrass-type representations.
Improves regression accuracy by using multiple discrete representations.
problem Improving regression accuracy using deep learning.
method Proposes using multiple discrete representations simultaneously for regression problems.
result Reduces prediction error compared to a baseline RvC approach.
High-dimensional time series are common in many domains. Since human cognition is not optimized to work well in high-dimensional spaces, these areas could benefit from interpretable low-dimensional representations. However, most representation learning algorithms for time series data are difficult to interpret. This is…
The paper proposes a method to learn structured representations from unlabeled data using mutual information maximization.
problem Learning structured representations from unlabeled data.
method Adversarial maximization of mutual information between a structured latent variable and a target variable.
result The proposed method outperforms current baselines in document hashing and yields highly compressed interpretable representations.
Discrete PU(1,1) representations of hyperelliptic groups are proven.
problem Characterizing PU(1,1) representations of hyperelliptic groups.
method Proving representations are basic if and only if they are discrete and faithful.
result A conjecture by S. Anan'in and E. Bento Gonçalves is partially proven.
Simple loop conjecture proven for certain discrete representations.
problem Proving the existence of simple closed curves in specific discrete representations.
method Analyzing representations of fundamental groups into PSL(2, R).
result Simple loop conjecture holds for discrete but non-faithful representations.
This paper reviews discrete curvature models for geometric data analysis.
problem Capturing intrinsic geometric structure in diverse data representations.
method Comprehensive review of discrete curvature models from Riemannian and metric geometry perspectives.
result Systematic pipeline for curvature-driven data analysis and learning.
Short introduction to discrete flat fronts in hyperbolic space with a Weierstrass representation proof.
problem Understanding discrete flat fronts in hyperbolic space.
method Proving a Weierstrass representation for discrete flat fronts.
result Any discrete flat front in the mixed area sense admits a Weierstrass representation.
The paper finds formulas for special surface shapes in 3D space.
problem Creating formulas for constant mean curvature surfaces.
method Weierstrass representations for discrete surfaces in isotropic space.
result Constructs examples of surfaces with discrete parametrizations.
The Weierstrass representation of discrete isotropic surfaces in R2,1, R3,1 and R2,2math.DG Using an integrable discrete Dirac operator, we construct a discrete version of the Weierstrass representation of time-like surfaces parametrized along isotropic directions in R2,1, R3,1 and R2,2. The corresponding discrete surfaces have isotropic edges. We show that any discrete surface satisfying a gen…
DCRL learns causal relationships from mixed-type discrete data.
problem Challenges in learning causal relationships from discrete, mixed-type data.
method Generative framework modeling directed acyclic graph and sparse bipartite graph, flexible measurement models for different types of data.
result Consistent recovery of latent causal structure from observed data distribution.
Proposes a non-parametric method for deep discrete latent variable models.
problem Learning sparse discrete latent representations in deep models.
method Iterative algorithm with Beta-Bernoulli process prior and local data scaling.
result Improves sparsity and scalability of deep discrete latent variable models.
Paper analyzes symbolic-dynamics inspired Markov modeling for time-series data.
problem Capturing temporal patterns in sequential data for statistical learning.
method Two-step process: discretization of continuous attributes and estimation of temporal memory.
result Effective Markov modeling depends on accurate discretization and memory estimation.
We present a representation formula for discrete indefinite affine spheres via loop group factorizations. This formula is derived from the Birkhoff decomposition of loop groups associated with discrete indefinite affine spheres. In particular we show that a discrete indefinite improper affine sphere can be constructed …
Proposes a VAE with a discrete bottleneck for better text generation.
problem VAEs struggle with latent variable auto-regressive decoding in text generation.
method Introduces a discretized bottleneck to enforce latent feature matching in a compact space.
result Demonstrates improved text generation capabilities across various tasks.
We prove that a free group F_2 admits a faithful discrete representation into Diff_{+}(I). We also prove that F_2 admits a faithful discrete representation into Homeo_{+}(I). Some properties of these representations have been studied. In the last section we raise several questions.
We give a Weierstrass type representation for semi-discrete minimal surfaces in Euclidean 3-space. We then give explicit parametrizations of various smooth, semi-discrete and fully-discrete catenoids, determined from either variational or integrable systems principles. Finally, we state the shared properties that those…
A formula connects discrete harmonic surfaces to holomorphic functions.
problem Creating smooth discrete harmonic surfaces from holomorphic data.
method Weierstrass representation formula for discrete harmonic surfaces.
result Smooth converging sequence of discrete harmonic surfaces converges to a minimal surface.
VQ-DRAW compresses images and generates realistic samples.
problem Learning compact discrete representations of images.
method Sequential discrete VAE with vector quantization.
result VQ-DRAW effectively compresses and generates images.
Factorizes discrete representations of finitely generated groups into PSL(2, R).
problem Understanding discrete representations of finitely generated groups into PSL(2, R).
method Factorization theorem for Fuchsian groups, Makanin-Razborov diagrams, and new class of groups called PSL(2, R)-discrete limit groups.
result Obtained useful information about PSL(2, R)-discrete limit groups.
Neural networks' feature geometry evolves like discrete Ricci flow.
problem Understanding neural feature representations and their geometric transformations.
method Approximating input manifold with geometric graphs and analyzing their evolution during training.
result Neural feature geometry evolves like discrete Ricci flow, with nonlinear activations playing a crucial role.
Proves critical exponent for Θ−positive representations in discrete subgroups.
problem Determining the critical exponent for Θ−positive representations. method Analyzes discrete subgroups Γ⊂PSL(2,R) and their geometric properties. result Equality of critical exponent holds if and only if Γ is a lattice for geometrically finite Γ. Neural nets learn robust geometric data representations.
problem Ensuring neural networks are robust to adversarial attacks.
method Topological Data Analysis via persistence diagrams, Lipschitz stability.
result Certified ε-robustness on ORBIT5K dataset. SOM-VQ tokenizes discrete models with semantic structure and navigable topology.
problem Lack of semantic structure in vector quantized representations limits interpretable human control.
method Combines vector quantization with Self-Organizing Maps to learn discrete codebooks with explicit topology.
result SOM-VQ produces more learnable token sequences and provides an explicit navigable geometry in code space.
New CR representations are found and shown to be redundant.
problem Identifying and classifying CR representations of 3-manifolds.
method Experimental computation of limit sets and exact computations of triangle groups.
result Many CR representations are redundant and conjugate.
Neural networks learn discrete tasks on continuous data via emergent geometry.
problem Understanding how neural networks perform discrete computations on continuous data.
method Analysis of Riemannian pullback metric across neural network layers.
result Neural networks learn to discretize continuous inputs and perform logical operations on these discretized variables.
Let M be a cusped hyperbolic 3-manifold, e.g. a knot complement. Thurston showed that the space of deformations of its fundamental group in PGL(2,C) (up to conjugation) is of complex dimension the number ν of cusps near the hyperbolic representation. It seems natural to ask whether some …
We study the limit set of discrete subgroups arising from Anosov representations. Specially we study the limit set of discrete groups arising from strictly convex real projective structures and Anosov representations from a finitely generated word hyperbolic group into a semisimple Lie group.
Learning compact discrete representations of data is a key task on its own or for facilitating subsequent processing of data. In this paper we present a model that produces Discrete InfoMax Codes (DIMCO); we learn a probabilistic encoder that yields k-way d-dimensional codes associated with input data. Our model's lear…
The paper explains emergent phenomena in deep learning using entropic forces.
problem Understanding the cause of emergent phenomena in deep learning and large language models.
method Proposes a rigorous entropic-force theory for neural networks trained with SGD and variants.
result Shows that representation learning is governed by emergent entropic forces that break continuous symmetries and preserve discrete ones.
Framework estimates multiple plausible solutions with uncertainty measures.
problem Machine learning models need to propose multiple plausible solutions with meaningful uncertainty.
method Discrete latent variables model one-to-many mappings, allowing effective conditional probability estimation.
result Framework outperforms state-of-the-art in uncertainty estimation and is practical.
Any discrete differential manifold M (finite set endowed with an algebraic differential calculus) can be represented by appropriate polyhedron P(M). This representation demonstrates the adequacy of the calculus of discrete differential manifolds and links this approach with that based on finitary substitutes…
This paper gives a process for finding discrete real specializations of sesquilinear representations of the braid groups using Salem numbers. This method is applied to the Jones and BMW representations, and some details on the commensurability of the target groups are given.
We describe recent links between two topics: geometric structures on manifolds in the sense of Ehresmann and Thurston, and dynamics "at infinity" for representations of discrete groups into Lie groups.
We outline a representation for discrete multivariate distributions in terms of interventional potential functions that are globally normalized. This representation can be used to model the effects of interventions, and the independence properties encoded in this model can be represented as a directed graph that allows…
Deep latent-variable models learn representations of high-dimensional data in an unsupervised manner. A number of recent efforts have focused on learning representations that disentangle statistically independent axes of variation by introducing modifications to the standard objective function. These approaches general…
We consider the task of unsupervised extraction of meaningful latent representations of speech by applying autoencoding neural networks to speech waveforms. The goal is to learn a representation able to capture high level semantic content from the signal, e.g.\ phoneme identities, while being invariant to confounding l…
Survey on discrete minimal surfaces and their properties.
problem Discretizing minimal surfaces in Euclidean space.
method Polyhedral surfaces with parallel face offsets and circle patterns.
result All simply connected discrete minimal surfaces can be constructed from circle patterns.
We establish what semi-discrete linear Weingarten surfaces with Weierstrass-type representations in 3-dimensional Riemannian and Lorentzian spaceforms are, confirming their required properties regarding curvatures and parallel surfaces, and then classify them. We then define and analyze their singularities. In partic…
Deep neural network learns discrete state abstractions for efficient planning.
problem Efficient sequential decision making in large state spaces.
method Information bottleneck method for learning approximate bisimulations using deep neural encoders and action-conditioned HMM.
result Trained method efficiently plans for unseen goals in multi-goal reinforcement learning.