New unsupervised method selects hard negative samples for contrastive learning.
problem How to select good negative examples for contrastive learning without using true similarity information.
method Developed a new family of unsupervised sampling methods for hard negative selection.
result Improves downstream performance across multiple modalities.
SANS uses graph structure to find meaningful negatives for entity and relation embeddings.
problem Finding hard negatives for entity and relation embeddings in knowledge graphs.
method Structure Aware Negative Sampling (SANS) that selects negatives from a node's k-hop neighborhood.
result SANS finds semantically meaningful negatives and is competitive with state-of-the-art approaches.
Paper introduces negative margin loss for better few-shot classification accuracy.
problem Improving few-shot classification accuracy with metric learning.
method Introduces negative margin loss and analyzes its impact on feature discriminability.
result Negative margin loss outperforms regular softmax loss on few-shot classification benchmarks.
New method learns from either positive or negative feedback alone.
problem Limited applicability of existing preference optimization methods in scenarios with only unpaired feedback.
method Decouples learning from positive and negative feedback, using expectation-maximization (EM) to optimize probability of positive outcomes and explicitly incorporate negative examples.
result Stable learning from negative feedback alone demonstrated.
The study investigates the impact of negative examples in contrastive learning.
problem Understanding how the number of negative examples affects downstream performance in contrastive learning.
method Theoretical analysis and empirical evaluation of NLP and vision tasks.
result The optimal number of negative examples should scale with the number of underlying concepts in the data.
New framework analyzes why more negative samples improve self-supervised learning performance.
problem Inconsistency between theoretical degradation and empirical improvement of downstream supervised tasks with more negative samples.
method Coupon collector's problem framework to analyze self-supervised representation learning with more negative samples.
result Bound can implicitly incorporate supervised loss in self-supervised loss by increasing negative samples.
Proposes a novel method for generating hard negatives near time series data boundaries.
problem Challenges in generating effective negative samples for time series anomaly detection.
method Reconstruction-driven boundary negative generation framework using reinforcement learning.
result Improves anomaly representation learning and achieves competitive detection performance.
New theory shows how learning algorithms can create a bias towards negative outcomes.
problem Negativity bias in adaptive learning algorithms.
method Generalization of the Hot Stove Effect to settings with negative estimates leading to smaller sample sizes.
result Negativity bias persists even when negative estimates do not lead to avoidance.
NCL improves interpretability of deep features by enforcing non-negativity.
problem Lack of interpretability in deep representations.
method Non-negative Contrastive Learning (NCL) using non-negativity constraints.
result NCL outperforms standard contrastive learning in feature disentanglement and selection.
Negative dependence improves machine learning performance.
problem Traditional independence limits machine learning capabilities.
method Negative dependence models, including DPPs and perturbed lattice models.
result Negative dependence often surpasses traditional methods in various machine learning tasks.
Motivated by applications in protein function prediction, we consider a challenging supervised classification setting in which positive labels are scarce and there are no explicit negative labels. The learning algorithm must thus select which unlabeled examples to use as negative training points, possibly ending up wit…
Extends conformal prediction to contrastive learning for better coverage of positive samples.
problem Lack of principled guarantees on coverage in contrastive learning.
method Introduces minimum-volume covering sets with learnable constraints.
result Improves inclusion-exclusion trade-offs in positive and negative samples.
New insights into negative sampling for graph representation learning.
problem Challenges in generating high-quality graph representations for large node sets.
method Theoretical analysis and derivation of negative sampling distribution correlation, proposing MCNS method.
result The negative sampling distribution should be positively but sub-linearly correlated to the positive sampling distribution.
Improved visual representation learning with conditional negative sampling.
problem Learning strong unsupervised visual representations using contrastive learning.
method Introduce a family of mutual information estimators that sample negatives conditionally.
result Improves accuracy by 2-5% points on four standard image datasets.
A new method improves graph node embeddings by considering both nearby and distant node similarities.
problem Improving graph node embeddings by considering both nearby and distant node similarities.
method Distance-aware Negative Sampling (DNS) which maximizes cohesion at nearby node-pairs and separation at distant node-pairs.
result DNS outperforms baseline methods in downstream node classification tasks on various datasets and GRL algorithms.
Paper proposes a new loss function for PU learning without negative examples.
problem Traditional machine learning struggles with negative examples, leading to biased predictions.
method Developed a collective loss function (cPU) for positive and unlabeled data.
result The cPU consistently outperforms existing methods in PU learning benchmarks and real-world datasets.
This paper tackles negative transfer in multi-task learning by introducing class-wise weights.
problem Negative transfer hampers function from achieving optimality in multi-task learning.
method Introduces class-wise weights to drive positive transfer and suppress negative transfer.
result Demonstrates improved performance in multi-task learning by reducing negative transfer.
In supervised machine learning for author name disambiguation, negative training data are often dominantly larger than positive training data. This paper examines how the ratios of negative to positive training data can affect the performance of machine learning algorithms to disambiguate author names in bibliographic …
Contrastive learning performance doesn't degrade with more negative samples.
problem Theoretical and empirical evidence of negative samples hurting performance in contrastive learning.
method Simple theoretical setting and empirical support on CIFAR-10 and CIFAR-100 datasets.
result Contrastive learning performance does not degrade with the number of negative samples.
Paper explains contrastive learning using cosine similarity and proposes mitigations for batch size effects.
problem Understanding and improving contrastive learning through batch size effects.
method Unified framework of cosine similarity, theoretical insights, and auxiliary loss.
result Performance improvement in small-batch settings through proposed auxiliary loss.
Negative step sizes improve second-order methods for neural networks.
problem Second-order methods discard negative curvature, limiting their effectiveness.
method Introduce negative step sizes in second-order methods combined with Wolfe line search.
result Negative step sizes lead to global convergence and improved performance.
Enhances contrastive learning for better representation learning on wild images.
problem Binary partition of views from same and different instances limits CL's performance.
method Doubly Contrastive Learning (CACR) with contrastive attraction and repulsion.
result CACR improves performance and robustness on wild image datasets.
DeepMP improves non-negative sparse recovery performance.
problem Recovering non-negative sparse signals with high coherence.
method Reformulated non-negative matching pursuit as a deep neural network.
result DeepMP yields significant improvement in exact recovery performance.
When labeled data is scarce for a specific target task, transfer learning often offers an effective solution by utilizing data from a related source task. However, when transferring knowledge from a less related source, it may inversely hurt the target performance, a phenomenon known as negative transfer. Despite its p…
This paper shows how optimizing with hard negative examples improves image retrieval.
problem Training with hard negative examples leads to poor training behavior.
method Characterize the space of triplets, derive why hard negatives fail, and offer a fix to the loss function.
result Optimizing with hard negative examples leads to more generalizable features and better image retrieval.
PCA-based MTL improves performance with reduced computation.
problem Negative transfer in multi-task learning.
method Random matrix approach to PCA-based MTL with counter-measures.
result Simple counter-measures prevent negative transfer and improve performance.
Transformer learns to estimate negative binomial parameters efficiently.
problem Parameter estimation for over-dispersed count data in large screens.
method Pre-trained transformer trained on synthetic data generation to invert parameter to count transformation.
result Method of moments provides faster, more efficient, and better-calibrated estimates.
New method corrects skewed confidence for PbN classification.
problem Weakly supervised binary classification with biased negative data.
method Corrects skewed confidence in negative data to improve classifier.
result Reduces distortion in posterior probability for PbN classification.
Non-negative L1-approximating polynomials for Gaussian distributions are proven for certain classes of sets.
problem Existence of non-negative L1-approximating polynomials for Gaussian distributions. method Proving the existence of degree-k non-negative polynomials that approximate indicator functions of sets with Gaussian surface area in L1-norm. result Proves the existence of non-negative L1-approximating polynomials for certain classes of sets with Gaussian surface area. Kernel methods identify treatment effects with unobserved confounding using negative controls.
problem Learning causal relationships with unmeasured confounding.
method Kernel ridge regression algorithms for nonparametric treatment effects.
result Uniform consistency and finite sample rates of convergence proved.
Paper tackles entity matching over multi-source data, optimizing alignment and mitigating negative transfer.
problem Learning effective entity matching models over multi-source large-scale data with relaxed assumptions.
method Proposes a Relaxed Multi-source Large-scale Entity-matching (RMLE) problem and Incentive Compatible Pareto Alignment (ICPA) method.
result Optimized cross-source alignments and mitigated negative transfer, improving entity matching accuracy.
In adversarial imitation learning, a discriminator is trained to differentiate agent episodes from expert demonstrations representing the desired behavior. However, as the trained policy learns to be more successful, the negative examples (the ones produced by the agent) become increasingly similar to expert ones. Desp…
A simple strategy prevents negative transfer in transfer learning.
problem Negative transfer in transfer learning where source representations harm target performance.
method Residual feature integration with a trainable target-side encoder.
result The method provably prevents negative transfer with theoretical guarantees.
We propose to formulate multi-label learning as a estimation of class distribution in a non-linear embedding space, where for each label, its positive data embeddings and negative data embeddings distribute compactly to form a positive component and negative component respectively, while the positive component and nega…
Negative user preference is an important context that is not sufficiently utilized by many existing recommender systems. This context is especially useful in scenarios where the cost of negative items is high for the users. In this work, we describe a new recommender algorithm that explicitly models negative user prefe…
Bayesian active learning tackles nuisance parameters, leading to bias and dilemmas.
problem Bayesian active learning with nuisance parameters leads to bias and dilemmas.
method Characterizes and mitigates negative interference by accurately estimating nuisance parameters.
result The extent of negative interference can be extremely large, and accurate estimation of nuisance parameters is critical.
We introduce Negative Sampling in Semi-Supervised Learning (NS3L), a simple, fast, easy to tune algorithm for semi-supervised learning (SSL). NS3L is motivated by the success of negative sampling/contrastive estimation. We demonstrate that adding the NS3L loss to state-of-the-art SSL algorithms, such as the Virtual Adv…
Survey on negative transfer in machine learning.
problem Negative transfer in transfer learning reduces target domain performance.
method Systematic review of 50+ approaches to mitigate negative transfer.
result Lack of systematic survey on negative transfer.
New DKPP family controls positive and negative dependence in random subsets.
problem Challenges in seamlessly bridging probabilistic models for positive and negative dependence.
method Introduced DKPP family and developed computational methods for probabilistic operations and inference.
result Controllability of positive and negative dependence demonstrated through numerical experiments.
Paper tackles continuous transfer learning with evolving target domains.
problem Challenges of negative transfer in evolving target domains.
method Proposes label-informed C-divergence for measuring distribution shift and negative transfer.
result Demonstrates effectiveness of TransLATE framework in minimizing classification error and C-divergence.
New mutual information framework improves contrastive learning for vision tasks.
problem Maximizing mutual information for better unsupervised learning representations.
method Reformulated mutual information as a lower bound, introducing new negative sampling strategies.
result Improved representations outperform previous methods in various vision tasks.
Negative momentum accelerates convergence in minimax games but at a suboptimal rate.
problem The convergence rate of negative momentum in minimax games is suboptimal.
method Extending variational inequality formulation, connecting momentum method with Chebyshev polynomials.
result Negative momentum accelerates convergence locally but at a suboptimal rate.
The study addresses negative transfer in multi-output Gaussian processes by proposing latent structures.
problem Negative transfer in multi-output Gaussian processes leading to decreased performance.
method Defining negative transfer, deriving conditions for avoiding it, proposing latent structures.
result Latent structures can avoid negative transfer and scale to large datasets.
Bayesian method mitigates negative transfer in unknown source data.
problem Negative transfer in transfer learning where target performance worsens after source data consideration.
method Proxy-informed robust method for probabilistic transfer learning (PROMPT).
result Negative transfer can be mitigated without prior knowledge of source data.
Data analysis in high energy physics often deals with data samples consisting of a mixture of signal and background events. The sPlot technique is a common method to subtract the contribution of the background by assigning weights to events. Part of the weights are by design negative. Negative weights lead to the diver…
New approach uses negative controls to estimate causal parameters without completeness conditions.
problem Estimating causal parameters when not all confounders are observed.
method Identification strategy based on minimax learning formulations for general function classes.
result Avoids completeness conditions and uniqueness assumptions on bridge functions.
Generative replay improves continual learning by using generated data as negative examples.
problem Catastrophic forgetting in continual learning.
method Using generative models to provide negative examples for new classes.
result Generative replay can improve learning new classes even when existing approaches fail.
We propose weighted inner product similarity (WIPS) for neural network-based graph embedding. In addition to the parameters of neural networks, we optimize the weights of the inner product by allowing positive and negative values. Despite its simplicity, WIPS can approximate arbitrary general similarities in…