dpVAEs improve VAEs by decoupling representation and generation.
problem VAEs struggle with both representation learning and sample generation.
method Introduce decoupled priors (dpVAEs) that separate representation and generation spaces.
result dpVAEs enable regularization without compromising sample generation.
This paper introduces a quantization-based regularizer for autoencoders to improve latent representations.
problem Autoencoders can overfit and collapse, leading to poor latent representations.
method The authors combine VQ-VAE and denoising methods to introduce a bottleneck Bayesian estimator that soft quantizes latent codes.
result The method results in better latent representations for supervised and clustering tasks.
Paper shows regularization improves robustness in domain generalization.
problem Improving robustness in domain generalization.
method Derives novel theoretical analysis to control representation smoothness and proposes a regularization method.
result Regularization improves robustness in domain generalization.
Study examines how statistical properties of deep learning representations can be adjusted.
problem Improving performance in deep learning models.
method Investigated eight representation regularization methods, including two new rank regularizers.
result Manipulating statistical properties of representations can indirectly improve model performance.
Regularized LAEs learn principal components efficiently.
problem Learning optimal linear representations with LAEs.
method Proper regularization schemes (non-uniform ℓ2 and nested dropout).
result Convergence to optimal representation is slow due to ill-conditioning.
New autoencoder learns structured representations without regularization.
problem Learning structured representations without relying on regularization.
method Proposes a novel autoencoder architecture that learns a hierarchy of latent variables.
result Improves results in generation, disentanglement, and extrapolation tasks.
Researchers compute determinants and torsions of Rumin complex in specific Lie group representations.
problem Computing determinants and torsions of Rumin complex in specific Lie group representations.
method Analyzing Schrodinger and generic representations of the (2,3,5) nilpotent Lie group.
result Computed the spectrum and zeta regularized determinant of Rumin differentials in Schrodinger representations and evaluated their alternating product in generic representations.
Paper generalizes regularization methods for Banach spaces.
problem Ill-posedness in machine learning and signal reconstruction.
method Generalizes regularization to Banach spaces and presents a representer theorem.
result Retrieves and extends known results in optimization and machine learning.
We prove that any minimal (maximal) strongly regular surface in the three-dimensional Minkowski space locally admits canonical principal parameters. Using this result, we find a canonical representation of minimal strongly regular time-like surfaces, which makes more precise the Weierstrass representation and shows mor…
Two new regularizers leverage class information to improve deep network performance.
problem Improving deep network performance and feature independence for classification tasks.
method Class-wise Covariance Regularizer (cw-CR) and Variance Regularizer (cw-VR) designed to manipulate statistical characteristics per class.
result Significant improvements in classification performance for 21 out of 22 tasks.
AL2 progressively penalizes network activations to prevent overfitting.
problem Avoiding overfitting in neural networks with limited training data.
method Progressive Activation Loss (PAL) method to regularize network representations.
result AL2 outperforms traditional regularization methods on benchmark datasets.
Unified theory for representation learning using learnable functions.
problem Insufficient theoretical understanding of unsupervised and self-supervised learning.
method Discriminative theoretical framework for analyzing sample complexity.
result Learnable regularization functions can reduce the amount of labeled data needed.
Model learns tensor representations from imperfect multimodal data.
problem Learning from imperfect multimodal data with noise or missing entries.
method Tensor rank minimization to regularize rank of tensor representations.
result Model effectively learns tensor representations from imperfect data.
A method identifies domain-general features using causal graph constraints and regularization.
problem Identifying domain-general features without prior knowledge of spurious features.
method Proposes a novel regularization framework based on causal graph constraints.
result Demonstrates effectiveness in both synthetic and real-world data, outperforming state-of-the-art methods.
We identify action representations from video data, proving their statistical benefits.
problem Identifying latent action policies from video data.
method Entropy-regularized LAPO objective, formalizing desiderata for action representations.
result Entropy-regularized LAPO identifies action representations satisfying desiderata under suitable conditions.
New loss function for learning sparse representations from data.
problem Emergence of sparse representations in neural networks.
method Analysis of input data distribution and regularization.
result Introduction of a new loss function for sparse regularization.
SinSim improves self-supervised learning by integrating optimal transport into contrastive learning.
problem Lack of explicit regularization in contrastive learning methods leads to suboptimal generalization.
method Integrates Sinkhorn regularization from optimal transport theory into SimCLR.
result SinSim outperforms SimCLR and other self-supervised methods on various datasets.
Group equivariant and steerable convolutional neural networks (regular and steerable G-CNNs) have recently emerged as a very effective model class for learning from signal data such as 2D and 3D images, video, and other data where symmetries are present. In geometrical terms, regular G-CNNs represent data in terms of s…
Study on how optimal representations emerge during deep learning training, focusing on the role of implicit regularization.
problem Understanding how optimal representations for tasks are learned during training.
method Investigates the role of implicit regularization in learning minimal sufficient representations, analyzing changes in representation content during training.
result Semantically meaningful but ultimately irrelevant information is encoded in early transient dynamics of training, which is later discarded.
Statistical methods protecting sensitive information or the identity of the data owner have become critical to ensure privacy of individuals as well as of organizations. This paper investigates anonymization methods based on representation learning and deep neural networks, and motivated by novel information theoretica…
We systematically investigate the problem of representing Markov chains by families of random maps, and which regularity of these maps can be achieved depending on the properties of the probability measures. Our key idea is to use techniques from optimal transport to select optimal such maps. Optimal transport theory a…
A new method uses algebraic insights to create approximately equivariant networks without complex architectures.
problem Designing equivariant neural networks with complex architectures and high computational cost.
method Imposes the group's regular representation as an inductive bias via an auxiliary loss, adding no learnable parameters.
result Matches or outperforms specialized models in several cases, even for infinite groups.
Regularized OT improves disentangled latent representations in GANs.
problem Disentangled representation learning in GANs.
method Structured regularization of optimal transport for latent space.
result Regularization leads to informative latent dimensions in GANs.
Paper introduces regularization for multi-head attention to spot keywords.
problem Redundancy in multi-head attention leads to lack of rich information.
method Regularization technique to enforce orthogonality between attention heads.
result Significant improvement in keyword spotting performance.
We provide adaptive inference methods, based on ℓ1 regularization, for regular (semi-parametric) and non-regular (nonparametric) linear functionals of the conditional expectation function. Examples of regular functionals include average treatment effects, policy effects, and derivatives. Examples of non-regular f…
The paper calculates Alexander polynomials for knots using finite group representations.
problem Calculating Alexander polynomials for knots using specific group representations.
method Defined twisted Alexander polynomials associated with regular representations of finite groups.
result Several formulas for the twisted Alexander polynomial are provided.
Regularization improves spectral embedding by focusing on the largest blocks.
problem Improving the quality of spectral embedding for graph data.
method Explained the impact of complete graph regularization on spectral embedding of a block model.
result Regularization forces spectral embedding to focus on the largest blocks, making it less sensitive to noise or outliers.
Paper proposes a graph network for EHR data that learns robust representations.
problem Learning robust representations for EHR data with implicit connections.
method Variationally regularized encoder-decoder graph network.
result Model outperforms existing methods in various EHR predictive tasks.
Large neural networks learn low-dimensional representations that balance complexity and regularity.
problem Understanding the tradeoff between low-dimensional representations and complexity in deep neural networks.
method Computed finite depth corrections to reveal a measure of regularity that bounds the pseudo-determinant of the Jacobian.
result Proved the conjectured bottleneck structure in learned features as network depth increases, showing almost all hidden representations are approximately low-dimensional and weight matrices have singular values close to 1.
We give sufficient conditions for a parametrised family of probability measures on a Riemannian manifold with boundary to be represented by random maps of class Ck. The conditions allow for the probability densities to approach zero towards the boundary of the manifold. We also formulate two obstructions to regular …
A new method for unsupervised disentanglement in GANs.
problem Learning disentangled representations in generative models.
method Regularizing GANs by aligning Jacobian vectors with coordinate axes.
result Unsupervised disentanglement achieved in GANs through spectral regularization.
Training deep neural networks is known to require a large number of training samples. However, in many applications only few training samples are available. In this work, we tackle the issue of training neural networks for classification task when few training samples are available. We attempt to solve this issue by pr…
Represents neural networks as solutions to inverse problems in Banach spaces.
problem Understanding the function learned by neural networks.
method Variational framework, representer theorem, polynomial ridge splines.
result Neural networks are solutions to inverse problems in Banach spaces.
New method for high-fidelity shape representations from raw data.
problem Creating accurate shape representations from raw data.
method A simple loss function encouraging neural network to vanish on input point cloud and have unit norm gradient.
result Our method produces high-fidelity, smooth, and natural zero level set surfaces.
New method uses random convex polytopes to measure representation quality.
problem Measuring the quality of deep learning representations.
method Random Polytope Descriptor method based on random convex polytopes.
result Regularization in autoencoders can degrade out-of-distribution detection.
Improved zero-shot learning with graph-based regularization.
problem Transfer knowledge to unknown classes in zero-shot learning.
method Isoperimetric loss for learning map between visual and semantic embeddings, exploiting graph structure.
result Regularization alone outperforms state-of-the-art methods in zero-shot learning benchmarks.
MAE tackles KL Varnishing in VAEs by controlling latent space geometry.
problem KL Varnishing in VAEs with expressive decoders.
method Mutual posterior-divergence regularization to control latent space geometry.
result MAE achieves comparable or superior density estimation and meaningful representation learning.
Sparsity inducing regularization is an important part for learning over-complete visual representations. Despite the popularity of ℓ1 regularization, in this paper, we investigate the usage of non-convex regularizations in this problem. Our contribution consists of three parts. First, we propose the leaky capped …
New technique prevents Q-learning collapse by maximizing diversity among ensembles.
problem Value function collapse in ensemble Q-learning.
method Maximizing representation diversity through regularization.
result Regularized approach significantly outperforms existing methods.
It is proved that every discrete Morse function in the sense of Forman on a finite regular CW complex can be represented by a polyhedral Morse function in the sense of Banchoff on an appropriate embedding in Euclidean space of the barycentric subdivision of the CW complex; such a representation preserves critical point…
Unified taxonomy for graph representation learning.
problem Lack of unified understanding and integration of graph representation learning methods.
method Proposes a Graph Encoder Decoder Model (GRAPHEDM) to unify graph neural networks, network embedding, and graph regularization.
result Unified taxonomy and Graph Encoder Decoder Model (GRAPHEDM) for graph representation learning.
The paper analyzes deep neural networks using rectified linear units.
problem Understanding the individual affine linear representations of deep neural networks.
method Signal processing perspective, atomic decompositions, Lipschitz regularity estimation.
result Conditions for stabilizing learning in deep neural networks without network depth constraints.
Two definitions quantify C2,α regularity of Riemannian surfaces.
problem Quantify the regularity of Riemannian surfaces.
method Intrinsic and extrinsic definitions using Hölder norms and smooth local representations.
result Intrinsic and extrinsic definitions are equivalent up to a constant.
New framework improves reliability of learned representations by modeling uncertainty and structural constraints.
problem Uncertainty in learned representations treated as deterministic, leading to unreliable models.
method Proposes a principled framework for reliable representation learning with uncertainty-aware regularization and structural constraints.
result Improves stability, calibration, and robustness of learned representations.
A VB-groupoid is a Lie groupoid equipped with a compatible linear structure. In this paper, we describe a correspondence, up to isomorphism, between VB-groupoids and 2-term representations up to homotopy of Lie groupoids. Under this correspondence, the tangent bundle of a Lie groupoid G corresponds to the "adjoint repr…
Introduces REVE, a regularization scheme that compresses class conditioned entropy.
problem Improving generalization performance of deep learning models.
method Identifies a variable responsible for final prediction, compresses class conditioned entropy, introduces a variational upper bound, and integrates a tractable loss into training.
result Demonstrates the efficiency of REVE on various neural networks and datasets.
New representation theory for closed geodesic subflows.
problem Classifying representations with good geometric properties.
method Restricting to invariant closed geodesic subflows.
result Equivalent characterizations and properties of new representations.
Given a flag in each of the vertex-transitive tessellations of the Euclidean plane by regular polygons, we determine the flag stabilizer under the action of the automorphism group of a regular cover. In so doing we give a presentation of these tilings as quotients of regular (infinite) polyhedra.