Develops a framework for potentials on Lauritzen manifolds using bi-forms.
problem Developing a framework for potentials on Lauritzen manifolds using bi-forms.
method Developing a framework for potentials on Lauritzen manifolds using bi-forms.
result Constructing a canonical contrast bi-form on dually curvature-free Lauritzen manifolds.
Bi-forms extend contrast functions to handle torsion in information geometry.
problem Insufficient contrast-based approaches for geometric structures with torsion.
method Introducing contrast bi-forms, a generalization of contrast functions.
result Bi-forms provide a unified framework for statistical potentials.
CNPs improve function approximation by contrastive learning.
problem Learning from non-i.i.d function instantiations in high-dimensional, noisy spaces.
method CNPs with TCL and FCL contrastive branches for better function approximation.
result CNPs outperform other variants in function distribution reconstruction and parameter identification.
Optimal CATE estimation with structured contrast functions using KRR.
problem Estimating CATEs with complex response functions in RKHS.
method Unified two-stage kernel ridge regression method for structured contrast functions.
result Minimax rates governed by contrast function complexity, enabling adaptation.
BrainSurfCNN predicts task contrasts from resting-state fingerprints, improving accuracy over baseline.
problem Predicting task-evoked activity from resting-state functional connectivity.
method Surface-based convolutional neural network (BrainSurfCNN) with reconstructive-contrastive loss.
result Significantly improved accuracy in predicting task contrasts over baseline.
Study of degenerate contrast functions on Lie groupoids and their geometric structures.
problem Understanding geometric structures on Lie groupoids with degenerate metrics.
method Using Lie groupoids and algebroids, analyze contrast functions and degenerate two-forms.
result Reduction of degenerate two-forms to pseudometric structures under regular conditions.
The article derives some novel independence measures and contrast functions for Blind Source Separation (BSS) application. For the kth order differentiable multivariate functions with equal hyper-volumes (region bounded by hyper-surfaces) and with a constraint of bounded support for k>1, it proves that equality …
Paper introduces new loss functions for Siamese networks using FDA.
problem Training Siamese networks with improved loss functions.
method Proposes Fisher Discriminant Triplet (FDT) and Fisher Discriminant Contrastive (FDC) loss functions based on FDA.
result Shows effectiveness of FDT and FDC on MNIST and histopathology datasets.
A new mutual information optimization method using self-supervised binary contrastive learning.
problem Improving self-supervised contrastive learning for better model performance.
method Proposes a novel loss function for contrastive learning that optimizes mutual information in positive and negative pairs.
result The proposed method outperforms state-of-the-art self-supervised contrastive frameworks on various benchmark datasets.
Contrastive Code Representation Learning improves code summarization and type inference.
problem Code representations are sensitive to edits, hindering downstream semantic understanding tasks.
method ContraCode: a contrastive pre-training task that learns code functionality.
result Contrastive pre-training improves code summarization and type inference accuracy.
We present a generalization of independent component analysis (ICA), where instead of looking for a linear transform that makes the data components independent, we look for a transform that makes the data components well fit by a tree-structured graphical model. Treating the problem as a semiparametric statistical prob…
The paper analyzes generalization in deep contrastive learning.
problem Generalization analysis for unsupervised deep contrastive representation learning.
method Parameter-counting and norm-based bounds derived for neural networks of varying sizes and depths.
result Bounds are independent of network depth and size, reducing dependency on matrix norms.
CIR method preserves relation for case-control studies.
problem Learning low-dimensional structure in case-control studies.
method Contrastive inverse regression (CIR) on Stiefel manifold.
result CIR outperforms other methods for high-dimensional data.
Contrastive learning simplifies statistical inference for complex models.
problem Computational intractability of likelihood functions for certain models.
method Contrastive learning as an alternative for parameter estimation and inference.
result Contrastive learning enables practical methods for diverse statistical problems.
GAAVI offers anytime-valid tests for CMF global null and contrasts.
problem Inference on the conditional mean function for high confidence decisions.
method Asymptotic anytime-valid tests for CMF global null and contrasts.
result Achieves asymptotic type-I error guarantees, power one, and optimal sample complexity.
MCD reformulates conditional density estimation into binary classification.
problem Conditional density estimation in statistical and machine learning.
method Marginal Contrastive Discrimination, reformulating into marginal and ratio density functions for binary classification.
result Significantly outperforms existing methods on most density models and regression datasets.
We propose a novel biologically-plausible solution to the credit assignment problem motivated by observations in the ventral visual pathway and trained deep neural networks. In both, representations of objects in the same category become progressively more similar, while objects belonging to different categories become…
Contrastive learning works well with redundant data views.
problem Improving unsupervised learning with redundant data views.
method Theoretical analysis of contrastive learning in a multi-view setting.
result Linear functions of learned representations are nearly optimal on prediction tasks when views are redundant.
Unsupervised contrastive learning has gained increasing attention in the latest research and has proven to be a powerful method for learning representations from unlabeled data. However, little theoretical analysis was known for this framework. In this paper, we study the optimization of deep unsupervised contrastive l…
Paper analyzes how contrastive learning structures learned representations.
problem Understanding the structure of learned representations in contrastive learning.
method Kernel-based contrastive learning framework (KCL) and statistical dependency viewpoint.
result Theoretical upper bound and generalization error bound for KCL.
Paper develops a theory explaining contrastive pre-training for multimodal AI.
problem Limited theoretical understanding of contrastive pre-training for multi-modal AI.
method Introduces approximate sufficient statistics and Joint Generative Hierarchical Model.
result Near-minimizers of contrastive loss are approximately sufficient, enabling diverse downstream tasks.
Improved similarity search in embeddings using InfoNCE loss.
problem Improving similarity search in embedding models trained by contrastive learning.
method Introduced a new continuity bound for InfoNCE loss via Gâteaux differentiation, preserving the averaging effect of negative samples.
result Demonstrated that the averaging effect of k negative samples in InfoNCE loss carries over to stabilisation of generalisation error as k grows. Paper investigates multimodal contrastive learning and incorporates unpaired data.
problem Improving feature learning ability of multimodal models under noisy data.
method Initiates investigation of nonlinear loss functions for multimodal contrastive learning, analyzes performance, proposes new loss incorporating unpaired data.
result MMCL can outperform unimodal contrastive learning and robustly handle noisy data.
EP-learning framework improves causal contrast estimation efficiency.
problem Estimating heterogeneous causal contrasts efficiently and stably.
method EP-learning framework combining T-learning and DR-learning.
result EP-learners are oracle-efficient and outperform competitors.
Supervised contrastive learning improves image classification accuracy.
problem Improving image classification accuracy using supervised contrastive learning.
method Extending self-supervised batch contrastive approach to fully-supervised setting, leveraging label information.
result Top-1 accuracy of 81.4% on ImageNet dataset, outperforming cross-entropy.
Simplified non-contrastive learning avoids representation collapse.
problem Training failure modes in self-supervised learning.
method Hyperdimensional computing and inductive bias.
result The approach avoids representation collapses.
New method learns stochastic process representations without exact reconstruction.
problem Learning exact representations of high-dimensional noisy stochastic processes.
method CReSP framework for contrastive learning of stochastic processes.
result Effective for learning representations of various stochastic processes.
A new contrastive learning method extracts asset embeddings from financial time series.
problem Extracting meaningful latent features from noisy financial data.
method Contrastive learning framework using hypothesis testing for positive and negative samples.
result Effective asset embeddings significantly outperform existing methods on financial tasks.
A new mathematical framework for multimodal learning.
problem Linking different data modalities for better understanding.
method Interpreting contrastive learning as optimizing encoders for conditional probability distributions.
result Novel probabilistic loss functions and metrics for alignment in latent space.
A framework evaluates the impact of different modules in graph contrastive learning.
problem Insufficient module-level evaluation in existing graph contrastive learning methods.
method Proposes a framework decomposing GCL models into four modules for module-level evaluation.
result Identifies module-level guidelines and competitive performance of different modules.
Optimal sample complexity for contrastive learning of distances.
problem Minimum labeled tuples needed for high accuracy in learning distances.
method Analyzes sample complexity in various distance settings, proving tight bounds.
result Almost optimal bound on sample complexity for learning ℓp distances. New method learns robust meta-representations for fast task adaptation.
problem Transferability of meta-representations in fast adaptation.
method Decoupled encoder-decoder approach with contrastive objective.
result Meta-representations improve downstream performance and noise robustness.
The paper explores tail diversification in financial markets using entropy and mutual information.
problem Tail diversification in financial time series.
method Statistical independence through differential entropy and mutual information, using moments as contrast functions.
result Tail covariance matrix is a key driver of tail diversification.
Theoretical proof shows COMs are a type of contrastive divergence model with improved sampling.
problem Improving sampling quality in offline model-based optimization.
method Showed COMs are contrastive divergence models, proposed Langevin MCMC sampler, and decoupled model.
result Improved sampling quality achieved by decoupling model and using Langevin MCMC.
This work studies how contrastive learning extracts features from unlabeled data.
problem How neural networks trained by contrastive learning can extract features from unlabeled data.
method Formal analysis of contrastive learning's feature learning process, considering two types of features: sparse and dense.
result Contrastive learning using ReLU networks can learn sparse features if proper augmentations are adopted.
By exploiting the property that the RBM log-likelihood function is the difference of convex functions, we formulate a stochastic variant of the difference of convex functions (DC) programming to minimize the negative log-likelihood. Interestingly, the traditional contrastive divergence algorithm is a special case of th…
A new unsupervised contrastive learning framework improves time series representation learning.
problem Lack of labeled data in time series data.
method Proposes an unsupervised contrastive learning framework using a novel contrastive loss and data augmentation.
result Framework outperforms other approaches on univariate and multivariate time series, and benefits transfer learning.
A new method for online VI in SSMs using asymptotic contrast.
problem Lack of functionality for streaming data in standard VI methods for SSMs.
method Propose maximising an IWAE-type variational lower bound on the asymptotic contrast function using stochastic approximation.
result OSIWAE allows for online learning of model parameters and latent states.
We prove a new and general concentration inequality for the excess risk in least-squares regression with random design and heteroscedastic noise. No specific structure is required on the model, except the existence of a suitable function that controls the local suprema of the empirical process. So far, only the case of…
Noise-Contrastive Estimation improves efficiency for estimating log-likelihood of complex point processes.
problem Estimating log-likelihood of complex multivariate point processes is computationally expensive.
method Noise-Contrastive Estimation adapted for multivariate point processes, with provable guarantees.
result Our method achieves similar log-likelihood with fewer evaluations and less time.
There are many models, often called unnormalized models, whose normalizing constants are not calculated in closed form. Maximum likelihood estimation is not directly applicable to unnormalized models. Score matching, contrastive divergence method, pseudo-likelihood, Monte Carlo maximum likelihood, and noise contrastive…
Introduces a new geometric structure for statistical manifolds with degenerate metrics.
problem Degenerate metrics in statistical manifolds affect geometric structures and applications.
method Introduces quasi-Codazzi structure for degenerate metrics and coherent tangent bundles.
result Generalizes geometric structures and relations for statistical models with degenerate metrics.
A new topology design improves zero-shot classification performance in contrastive learning.
problem Improving zero-shot classification performance in contrastive visual-textual alignment.
method Proposed an alternative topology design using multiple class tokens and an oblique manifold with negative inner product.
result Improves zero-shot classification performance by an average of 6.1%.
New measure assesses time series pre-training data quality without labels.
problem Challenges in collecting diverse pre-training datasets for time series classification.
method Contrastive-learning-based foundation model and contrastive accuracy measure.
result Contrastive accuracy correlates with model performance on downstream tasks.
In recent years, spectral clustering has become a standard method for data analysis used in a broad range of applications. In this paper we propose a new class of algorithms for multiway spectral clustering based on optimization of a certain "contrast function" over the unit sphere. These algorithms, partly inspired by…
We consider inference about a scalar parameter under a non-parametric model based on a one-step estimator computed as a plug in estimator plus the empirical mean of an estimator of the parameter's influence function. We focus on a class of parameters that have influence function which depends on two infinite dimensiona…
CCLF improves sample efficiency in RL by prioritizing informative samples.
problem Learning from high-dimensional observations is challenging and unstable.
method Contrastive-Curiosity-Driven Learning Framework (CCLF) that prioritizes informative samples.
result CCLF improves sample efficiency and learning performance in RL.
Paper proposes a new loss function for conditional models using soft targets.
problem Improving generalization performance of deep neural networks on supervised classification tasks.
method Introduces a new loss function compatible with soft targets, based on noise contrastive estimation.
result Soft target InfoNCE loss performs on par with cross-entropy baselines and outperforms other losses.