Contrastive regularization improves semi-supervised learning by better propagating confident pseudo-labels.
problem Consistency regularization's limitation in high performance and efficiency.
method Proposes contrastive regularization to update model features, pushing confident labels into unlabeled samples.
result Improves semi-supervised learning tasks with fewer training iterations and robust performance.
SinSim improves self-supervised learning by integrating optimal transport into contrastive learning.
problem Lack of explicit regularization in contrastive learning methods leads to suboptimal generalization.
method Integrates Sinkhorn regularization from optimal transport theory into SimCLR.
result SinSim outperforms SimCLR and other self-supervised methods on various datasets.
Paper proposes a new regularization method to prevent model degradation under distribution shifts.
problem Model performance degrades under distribution shifts.
method Supervised contrastive learning with heterogeneous similarity.
result The proposed method outperforms existing regularization methods on benchmark datasets.
A fast geometric regularizer improves event camera performance.
problem Event collapse in contrast maximization framework.
method Geometric regularizer to mitigate overfitting.
result State-of-the-art accuracy with reduced computational complexity.
i-Mix improves contrastive learning across domains without domain-specific augmentations.
problem Improving contrastive representation learning for unlabeled data across diverse domains.
method i-Mix treats contrastive learning as a non-parametric classifier problem, mixing data in input and virtual label spaces.
result i-Mix consistently improves representation quality across image, speech, and tabular data domains.
ReCoRe learns invariant features for world navigation using contrastive learning and regularizers.
problem Limited sample efficiency and overfitting to training scenarios in RL for visual navigation.
method Contrastive unsupervised learning and intervention-invariant regularizer.
result Significantly improves sample efficiency and generalization in out-of-distribution point navigation tasks.
Optimal CATE estimation with structured contrast functions using KRR.
problem Estimating CATEs with complex response functions in RKHS.
method Unified two-stage kernel ridge regression method for structured contrast functions.
result Minimax rates governed by contrast function complexity, enabling adaptation.
In contrast with what happens for Legendrian embeddings, there always exist positive loops of Legendrian immersions.
The paper uses neural networks to learn system dynamics from data with Lipschitz regularization.
problem Learning governing equations from time-sampled data.
method Lipschitz regularized deep neural networks for ODE system identification.
result Lipschitz regularization improves the smoothness and generalization of the learned function.
This work shows that supervised contrastive learning achieves similar results to cross-entropy but requires more iterations.
problem The question of whether there are fundamental differences in representation geometry between supervised contrastive learning and cross-entropy.
method The authors prove that both losses attain their minimum when representations of each class collapse to the vertices of a regular simplex, and they empirically validate this finding.
result Supervised contrastive learning requires more iterations to reach a close-to-optimal state compared to cross-entropy, indicating different optimization behavior.
Study of degenerate contrast functions on Lie groupoids and their geometric structures.
problem Understanding geometric structures on Lie groupoids with degenerate metrics.
method Using Lie groupoids and algebroids, analyze contrast functions and degenerate two-forms.
result Reduction of degenerate two-forms to pseudometric structures under regular conditions.
This work investigates image augmentations for GAN training, improving image quality.
problem Improving the accuracy and robustness of GAN models for image synthesis.
method Systematic study of various image augmentation techniques for GAN training.
result Vanilla GANs can achieve state-of-the-art generation quality with image augmentations.
VCL adds uncertainty to contrastive learning models.
problem Lack of uncertainty quantification in contrastive learning methods.
method VCL uses a decoder-free framework that maximizes ELBO with InfoNCE loss and KL divergence.
result VCL provides meaningful uncertainty estimates and matches deterministic baselines in accuracy.
Feature normalization prevents collapse in non-contrastive learning dynamics.
problem Non-contrastive learning can collapse into a single point due to lack of repulsive force.
method Extended previous theory based on L2 loss to cosine loss, considering feature normalization.
result Cosine loss induces stable equilibrium, preventing collapse even with insufficient repulsive force.
Method generates time-series attribution maps with identifiability guarantees.
problem Lack of identifiability guarantees in gradient-based attribution methods.
method Regularized contrastive learning algorithm trained on time-series data.
result Empirically shows robust approximation of zero vs. non-zero entries in the ground-truth attribution map.
The study examines differential smoothness in specific Artin-Schelter regular algebras of dimension 5.
problem Investigating the differential smoothness of Artin-Schelter regular algebras of dimension 5.
method Analyzing the relationship between the number of generators and Gelfand-Kirillov dimension to identify structural obstructions.
result Certain two- and four-generator AS-regular algebras of global dimension five fail to admit a differential calculus, while a five-generator graded Clifford algebra provides a positive example.
We show the local wellposedness of biharmonic wave maps with initial data of sufficiently high Sobolev regularity and a blow-up criterion in the sup-norm of the gradient of the solutions. In contrast to the wave maps equation we use a vanishing viscosity argument and an appropriate parabolic regularization in order to …
Unified view on learning unnormalized distributions using NCE.
problem Learning unnormalized distributions across different communities.
method Noise-Contrastive Estimation (NCE) for a unified perspective.
result Established finite-sample convergence rates for exponential families.
Study introduces a new method for multiple parameter regularization in polynomial functional regression.
problem Handling varying regularization parameters in polynomial functional regression.
method Developed a theoretically grounded algorithm for multiple parameter regularization and model aggregation.
result Promising results from evaluations on synthetic and real-world data.
PCA++ improves robustness to background noise in contrastive learning.
problem Recovering shared signal subspaces from positive pairs in high-dimensional data with structured background noise.
method PCA++ uses hard uniformity-constrained contrastive learning to enforce identity covariance on projected features.
result PCA++ outperforms standard PCA and alignment-only PCA+ in simulations and real-world datasets.
In this paper, we propose a new variational model for image reconstruction by minimizing the L1 norm of the \emph{Weingarten map} of image surface (x,y,f(x,y)) for a given image f:Ω→R. We analytically prove that the Weingarten map minimization model can not only keep the greyscale int…
In this paper, we discuss how a suitable family of tensor kernels can be used to efficiently solve nonparametric extensions of ℓp regularized learning methods. Our main contribution is proposing a fast dual algorithm, and showing that it allows to solve the problem efficiently. Our results contrast recent finding…
CCLF improves sample efficiency in RL by prioritizing informative samples.
problem Learning from high-dimensional observations is challenging and unstable.
method Contrastive-Curiosity-Driven Learning Framework (CCLF) that prioritizes informative samples.
result CCLF improves sample efficiency and learning performance in RL.
RényiCL uses Rényi divergence for robust contrastive learning with stronger data augmentations.
problem Learning useful representations from multiple data views with hard augmentations.
method RényiCL employs Rényi divergence for contrastive learning, using a novel variational objective to manage hard negative sampling.
result RényiCL achieves better performance with stronger augmentations compared to other methods.
GCL-LRR improves node classification in noisy graphs.
problem Noise in real-world graph data impairs GNNs' effectiveness.
method Two-stage transductive learning with low-rank regularization and attention.
result Improved node classification performance in noisy graphs.
Generalization is essential for deep learning. In contrast to previous works claiming that Deep Neural Networks (DNNs) have an implicit regularization implemented by the stochastic gradient descent, we demonstrate explicitly Bayesian regularizations in a specific category of DNNs, i.e., Convolutional Neural Networks (C…
Inserts proximal mapping into deep networks for better regularization.
problem Effective regularization of deep learning models to handle adversarial perturbations and correlations between modalities.
method Proposes a new layer that directly produces regularized hidden layer outputs using proximal mapping.
result Outperforms state-of-the-art methods in robust temporal learning and multiview modeling.
CD algorithm achieves near-optimal convergence rate for unnormalized models.
problem Training unnormalized models with high efficiency.
method Non-asymptotic analysis of contrastive divergence algorithm.
result CD can achieve O(n−1/2) convergence rate under regularity assumptions. C2VAE learns disentangled and coupled representations without prior knowledge.
problem Learning disentangled and coupled representations in latent space.
method Introduces C2VAE, a self-supervised VAE that factorizes posterior and uses Gaussian copula for dependencies. result Demonstrates strong effect in enhancing disentangled representation learning.
Improves contrastive learning invariance with novel training objectives and feature averaging.
problem Contrastive learning's implicit invariance is insufficient for robust performance.
method Introduces a novel training objective and feature averaging approach to enforce invariance.
result Improved performance and robustness to transformations on downstream tasks.
EBMs improve continual learning without external memory or regularization.
problem Improving continual learning without external memory or regularization.
method Energy-Based Models with contrastive divergence training objective.
result EBMs outperform baseline methods on various benchmarks.
Over the past few years, trace regression models have received considerable attention in the context of matrix completion, quantum state tomography, and compressed sensing. Estimation of the underlying matrix from regularization-based approaches promoting low-rankedness, notably nuclear norm regularization, have enjoye…
Studied how SGD's stability regularization affects generalization in neural networks.
problem Understanding why SGD often generalizes better than GD in neural networks.
method Analyzed stability of SGD and GD through Frobenius norm and trace of Hessian, and compared their generalization properties.
result Stable minima of SGD generalize well, while GD's stability-induced regularization is too weak.
Let Γ be the fundamental group of a manifold modeled on three dimensional Sol geometry. We prove that Γ has a finite index subgroup G which has a rational growth series with respect to a natural generating set. We do this by enumerating G by a regular language. However, in contrast to most earlier proofs of thi…
We review recent work on the local geometry and optimal regularity of Lorentzian manifolds with bounded curvature. Our main results provide an estimate of the injectivity radius of an observer, and a local canonical foliations by CMC (Constant Mean Curvature) hypersurfaces, together with spatially harmonic coordinates.…
A new method reduces speckles in high contrast imaging.
problem Over-subtraction from speckles and self-subtraction in data reduction.
method Data Imputation concept using Karhunen-Loève transform (DIKL).
result DIKL achieves high-quality results with significantly reduced computational cost.
DNN-based cross-modal retrieval has become a research hotspot, by which users can search results across various modalities like image and text. However, existing methods mainly focus on the pairwise correlation and reconstruction error of labeled data. They ignore the semantically similar and dissimilar constraints bet…
New approach uses hinge loss for iterative regularization in classification.
problem Improving classification accuracy through regularization.
method Develops an iterative regularization approach based on hinge loss.
result Proves convergence and rates of convergence for classification.
In this study, the effects of eight representation regularization methods are investigated, including two newly developed rank regularizers (RR). The investigation shows that the statistical characteristics of representations such as correlation, sparsity, and rank can be manipulated as intended, during training. Furth…
Sparse alpha-norm regularization has many data-rich applications in Marketing and Economics. Alpha-norm, in contrast to lasso and ridge regularization, jumps to a sparse solution. This feature is attractive for ultra high-dimensional problems that occur in demand estimation and forecasting. The alpha-norm objective is …
We consider the problem of learning a forest of nonlinear decision rules with general loss functions. The standard methods employ boosted decision trees such as Adaboost for exponential loss and Friedman's gradient boosting for general loss. In contrast to these traditional boosting algorithms that treat a tree learner…
New metrics prevent event collapse in contrast maximization frameworks.
problem Event collapse in contrast maximization frameworks.
method First principles of space-time deformation based on differential geometry and physics.
result Proposed metrics mitigate event collapse and do not harm well-posed warps.
Study shows torsion grows subexponentially in book of I-bundles but can grow exponentially in non-regular covers.
problem Growth rates of torsion in book of I-bundles.
method Analysis of torsion in homology of book of I-bundles using finite-sheeted covers.
result Torsion growth rates differ between regular and non-regular finite-sheeted covers.
Gradient descent implicitly favors group sparsity in neural networks.
problem Understanding implicit regularization in neural networks for structured sparsity.
method Novel neural reparameterization for diagonally grouped linear networks.
result Gradient descent without explicit regularization biases towards group sparsity.
SepVAE separates patient-specific patterns from healthy ones using contrastive VAE.
problem Separating patient-specific patterns from healthy ones in medical datasets.
method SepVAE uses a contrastive VAE with disentangling and classification losses to differentiate between common and salient features.
result SepVAE outperforms previous methods in three medical applications and a CelebA dataset.
Dropout is a simple but effective technique for learning in neural networks and other settings. A sound theoretical understanding of dropout is needed to determine when dropout should be applied and how to use it most effectively. In this paper we continue the exploration of dropout as a regularizer pioneered by Wager,…
There has been considerable advance in understanding the properties of sparse regularization procedures in high-dimensional models. In time series context, it is mostly restricted to Gaussian autoregressions or mixing sequences. We study oracle properties of LASSO estimation of weakly sparse vector-autoregressive model…
State entropy regularization improves robustness in reinforcement learning, especially under structured perturbations.
problem Structured and spatially correlated perturbations in reinforcement learning.
method State entropy regularization, compared to policy entropy.
result State entropy regularization provides better robustness to structured and spatially correlated perturbations.