Contrastive learning harms minority group representations, affecting downstream tasks.
problem Representation harm in contrastive learning, especially affecting minority groups.
method Causal mediation analysis and stochastic block model explanation.
result Representation harm in contrastive learning is partly responsible for allocation harm in downstream tasks.
While harms of allocation have been increasingly studied as part of the subfield of algorithmic fairness, harms of representation have received considerably less attention. In this paper, we formalize two notions of stereotyping and show how they manifest in later allocative harms within the machine learning pipeline. …
Introduces MPR to measure and optimize representation across intersectional groups in retrieval.
problem Harmful stereotypes, cultural erasure, and social disparities in image search and retrieval.
method Develops MPR metric, practical estimation methods, theoretical guarantees, and optimization algorithms.
result Optimizing MPR yields more proportional representation across multiple intersectional groups, often with minimal retrieval accuracy compromise.
Algorithm removes spurious concepts from neural network representations without harming task performance.
problem Spurious correlations hinder neural network out-of-distribution generalization.
method Iterative algorithm that identifies two orthogonal subspaces in neural network representation.
result Algorithm outperforms existing methods on computer vision and natural language processing benchmarks.
COCA refactors training data to identify and erase unsafe concepts in LLMs.
problem Identifying and erasing unsafe concepts in Large Language Models (LLMs) for safety alignment.
method Concept Concentration (COCA) refactors training data with an explicit reasoning process to identify and erase unsafe concepts.
result COCA significantly reduces both in-distribution and out-of-distribution jailbreak success rates while maintaining strong performance on regular tasks.
Proposes a new method to measure and avoid harm in machine learning decisions.
problem Measuring and avoiding harm in machine learning algorithms.
method Formal definition of harm and benefit using causal models, counterfactual objective functions.
result Demonstrates that standard machine learning methods can lead to harmful policies under distributional shifts.
Aims to create safe reinforcement learning policies by considering individual harm.
problem Optimal policies for a population may harm certain individuals.
method Formalizes individual harm, proposes a two-stage procedure, and establishes finite-sample properties.
result Learned policies maximize expected return while minimizing harm.
Researchers classify harmful structures on unimodular Lie groups.
problem Characterizing harmful structures on Lie groups.
method Analyzing left-invariant structures and using Clifford multiplication.
result Classification of harmful structures on unimodular Lie groups of dimension ≤ 4.
LLM safety alignment explained as divergence estimation.
problem Aligning large language models to avoid harmful outputs.
method Presented a theoretical framework showing alignment methods as divergence estimators.
result KLDO method improves safety alignment using compliance-refusal datasets.
Prediction models can harm patients even when accurate, leading to self-fulfilling prophecies.
problem Prediction models can lead to harmful decisions that worsen patient outcomes.
method Formal characterization of harmful prediction models and analysis of their impact.
result Well-calibrated models are ineffective for decision-making as they do not change the data distribution.
New method stops experiments early for harm in diverse groups.
problem Early stopping of experiments for harmful treatment effects in diverse populations.
method Causal machine learning approach (CLASH) for early stopping.
result CLASH effectively stops experiments early for harmful treatment effects in diverse groups.
Paper proposes HGAT for detecting fake news on HIN.
problem Viral spread of fake news causing social harm.
method Hierarchical Graph Attention Network (HGAT) for node representation learning in HIN.
result HGAT outperforms text-based and network-based models.
Cross-domain sentiment analysis is currently a hot topic in the research and engineering areas. One of the most popular frameworks in this field is the domain-invariant representation learning (DIRL) paradigm, which aims to learn a distribution-invariant feature representation across domains. However, in this work, we …
Theorem. Let M be a compact, connected, oriented smooth Riemannian n-manifold with non-empty boundary. Then the cohomology of the complex (Harm*(M),d) of harmonic forms on M is given by the direct sum H^p(Harm*(M),d) = H^p(M;R) + H^(p-1)(M;R) for p=0,1,...,n. When M is a closed manifold, a form is harmonic if and only …
Learning knowledge representation is an increasingly important technology applicable in many domain-specific machine learning problems. We discuss the effectiveness of traditional Link Prediction or Knowledge Graph Completion evaluation protocol when embedding knowledge representation for categorised multi-relational d…
We present a large-scale study of gender bias in occupation classification, a task where the use of machine learning may lead to negative outcomes on peoples' lives. We analyze the potential allocation harms that can result from semantic representation bias. To do so, we study the impact on occupation classification of…
New method estimates harmful instances in GANs for better model performance.
problem Estimating harmful instances in GANs for improved model performance.
method Uses Jacobian of generator's loss to trace harmfulness of instances.
result Correctly inferred changes in GAN evaluation metrics and improved model performance.
The paper analyzes how deep neural networks handle noisy labels and finds disparate impacts.
problem Disparate impacts of noisy labels on instances with different representation frequencies.
method Quantifying harms, analyzing solutions, and comparing their impacts on different frequency instances.
result Existing solutions lead to disparate treatments, benefiting higher-frequency instances more.
Study shows harmful overfitting in Sobolev spaces even as training data grows.
problem Harmful overfitting in Sobolev spaces under noisy conditions.
method Geometric argument using Sobolev inequalities.
result Approximately norm-minimizing interpolators exhibit harmful overfitting.
New framework shows algorithmic recourse can be harmful.
problem Risks of providing algorithmic recourse in binary classification.
method Abstract learning-theoretic framework comparing risks with and without recourse.
result Providing recourse can be harmful, increasing class uncertainty and mistakes.
As machine learning (ML) increasingly affects people and society, awareness of its potential unwanted consequences has also grown. To anticipate, prevent, and mitigate undesirable downstream consequences, it is critical that we understand when and how harm might be introduced throughout the ML life cycle. In this paper…
Common fairness definitions in machine learning focus on balancing notions of disparity and utility. In this work, we study fairness in the context of risk disparity among sub-populations. We are interested in learning models that minimize performance discrepancies across sensitive groups without causing unnecessary ha…
Improves GAN performance by identifying and removing harmful training instances.
problem Challenges in applying previous instance removal methods to GANs.
method Proposes influence estimation methods using Jacobian of generator's gradient and instance evaluation scheme based on GAN evaluation metrics.
result Removing harmful instances significantly improves GAN performance.
Contrastive learning benefits from generated data but can be harmed by it too.
problem Contrastive learning's reliance on data augmentation and the impact of generated data.
method Investigates the role of generated data in contrastive learning and proposes Adaptive Inflation (AdaInf).
result Generated data can sometimes harm contrastive learning, and AdaInf improves performance.
Bayesian approach quantifies uncertainty in LLM evaluations.
problem Statistical uncertainty in evaluating LLM behavior.
method Bayesian evaluation of LLM behavior using probabilistic text generation strategies.
result Bayesian approach provides useful uncertainty quantification about LLM behavior.
The choice of making an intervention depends on its potential benefit or harm in comparison to alternatives. Estimating the likely outcome of alternatives from observational data is a challenging problem as all outcomes are never observed, and selection bias precludes the direct comparison of differently intervened gro…
REN addresses uncertainty in user feedbacks for better recommendation systems.
problem Recurrent neural networks focus solely on item relevance, neglecting diverse item exploration.
method Proposes REN, a new type of recurrent neural network that balances relevance and exploration while accounting for representation uncertainty.
result REN achieves satisfactory long-term rewards on synthetic and real-world recommendation datasets, outperforming state-of-the-art models.
The study finds that memorization is necessary or harmful depending on the prior distribution and noise level.
problem The impact of memorization on generalization in overparameterized models.
method An overparameterized linear model with general priors in a Bayesian setup.
result Explicit conditions for optimal generalization based on the prior distribution and noise level.
Contrastive learning outperforms autoencoders and GANs in feature recovery and downstream tasks.
problem Theoretical understanding of contrastive learning's superiority in feature learning.
method Theoretical analysis of contrastive learning in linear representation settings.
result Contrastive learning outperforms autoencoders and GANs for feature recovery and in-domain downstream tasks.
New framework assesses LLM security risks in BFSI.
problem Lack of domain-specific security evaluation for LLMs in BFSI.
method Risk-aware evaluation framework combining taxonomy, automated red-teaming, and ensemble judging.
result Higher decoding stochasticity and adaptive interaction lead to more severe disclosures.
Study characterizes harmful low-fidelity data sources for surrogate models.
problem Identifying which low-fidelity data sources to use in constructing surrogate models.
method Employed benchmark filtering techniques to assess harmful sources using limited data.
result Provided guidelines for using low-fidelity sources in an industrial setting.
Neural Networks are being integrated into safety critical systems, e.g., perception systems for autonomous vehicles, which require trained networks to perform safely in novel scenarios. It is challenging to verify neural networks because their decisions are not explainable, they cannot be exhaustively tested, and finit…
Machine learning helps predict smoke types for safer forest burns.
problem Determining which fuels to burn safely to reduce wildfire risk and minimize smoke.
method Machine learning techniques like spectral clustering and manifold learning.
result Interpretable representations and tools for differentiating smoke types.
Diffusion LLMs can efficiently generate harmful prompts for adversarial testing.
problem Generating harmful prompts for adversarial testing is resource-intensive and costly.
method Transformed adversarial prompt optimization into an efficient inference task using pretrained Diffusion LLMs.
result Only a few conditional samples are required to generate harmful prompts with high reward.
Detects harmful distribution shifts in deployed models without false alarms.
problem Detecting harmful distribution shifts in deployed models without false alarms.
method Sequential tools for testing if the difference between source and target distributions leads to a significant increase in a risk function.
result Demonstrated the efficacy of the proposed framework through extensive empirical studies.
A new method learns manifold-valued latents without an encoder.
problem Distorting data with intrinsic non-Euclidean structure.
method Riemannian generative decoder that learns latents directly.
result Learned representations respect the prescribed geometry and capture intrinsic non-Euclidean structure.
Combining ensembles and data augmentation harms model calibration.
problem Improving model calibration and robustness with ensembles and data augmentation leads to a trade-off.
method Combining ensemble averaging and data augmentation techniques.
result Combining ensembles and data augmentation can harm model calibration.
Selective planning with imperfect models reduces harmful effects of model inadequacy.
problem Harmful effects of using an imperfect model in reinforcement learning.
method Selective planning with heteroscedastic regression to estimate predictive uncertainty from model inadequacy.
result Effective selective planning requires considering both parameter uncertainty and model inadequacy.
New framework removes harmful momentum effect for long-tailed classification.
problem Challenges in maintaining balanced datasets with long-tailed data.
method Causal inference framework to disentangle and remove harmful effects of momentum.
result Achieves state-of-the-art performance on long-tailed visual recognition benchmarks.
RESTA defends LLMs against jailbreaking attacks by adding random noise to embeddings.
problem Vulnerability of LLMs to jailbreaking attacks that generate harmful outputs.
method Adds random noise to embedding vectors and aggregates during token generation.
result RESTA achieves superior robustness versus utility tradeoffs compared to baseline defenses.
Data whitening and second order optimization harm generalization by reducing access to dataset information.
problem Harmful effects of data whitening and second order optimization on generalization in machine learning.
method Analysis of fully connected models and experimental verification.
result Data whitening and second order optimization reduce or prevent generalization by limiting access to dataset information.
Two-layer CNNs can overfit without harm under certain conditions.
problem Understanding when and how overfitting occurs in neural networks.
method Theoretical analysis of a two-layer CNN trained by gradient descent.
result A sharp phase transition between benign and harmful overfitting based on signal-to-noise ratio.
Prediction markets can be manipulated by traders who can move contract settlements, harming price discovery.
problem Manipulation of settlement times in prediction markets leads to unfair wealth transfer and harms price discovery.
method Developed a model showing how settlement manipulation transfers wealth and harms price discovery, and observed real-world effects on Polymarket's Bitcoin contract.
result Manipulators capture significant profits from retail traders, especially when settlement times are short.
Detects harmful shifts without labels for model performance.
problem Detecting distribution shifts without access to labels.
method Uses a proxy derived from predictions of an error estimator.
result High power and false alarm control under various shifts.
Lower class selectivity makes networks more robust to natural perturbations but more vulnerable to adversarial attacks.
problem Understanding how class selectivity affects robustness to different types of perturbations in neural networks.
method Investigated the relationship between class selectivity and robustness to natural and adversarial perturbations in neural networks.
result Lower class selectivity increases robustness to natural perturbations but decreases robustness to adversarial attacks.
"Overlearning" means that a model trained for a seemingly simple objective implicitly learns to recognize attributes and concepts that are (1) not part of the learning objective, and (2) sensitive from a privacy or bias perspective. For example, a binary gender classifier of facial images also learns to recognize races…
Ethereum block builders can earn up to $14M/month by reordering transactions, harming users.
problem Block builders can exploit transaction reordering to earn significant profits, harming users.
method Estimation of MEV payments and analysis of reordering effects.
result Block builders can earn up to $14M/month by reordering transactions, skewing the distribution.
Adversarial training overfits, harming robustness; early stopping fixes this.
problem Adversarial robustness in deep learning overfits to training data.
method Empirical study of adversarially trained deep networks.
result Overfitting to training data harms robust performance in adversarial training.