Bayesian approach quantifies uncertainty in LLM evaluations.
problem Statistical uncertainty in evaluating LLM behavior.
method Bayesian evaluation of LLM behavior using probabilistic text generation strategies.
result Bayesian approach provides useful uncertainty quantification about LLM behavior.
RESTA defends LLMs against jailbreaking attacks by adding random noise to embeddings.
problem Vulnerability of LLMs to jailbreaking attacks that generate harmful outputs.
method Adds random noise to embedding vectors and aggregates during token generation.
result RESTA achieves superior robustness versus utility tradeoffs compared to baseline defenses.
The study categorizes reward errors in reinforcement learning, finding some can be beneficial.
problem Training language models with imperfect proxy rewards.
method Theoretical analysis of policy gradient optimization and categorization of reward errors.
result Reward errors can be benign or even beneficial, preventing policy from stalling.
Proposes a new method to measure and avoid harm in machine learning decisions.
problem Measuring and avoiding harm in machine learning algorithms.
method Formal definition of harm and benefit using causal models, counterfactual objective functions.
result Demonstrates that standard machine learning methods can lead to harmful policies under distributional shifts.
Contrastive learning harms minority group representations, affecting downstream tasks.
problem Representation harm in contrastive learning, especially affecting minority groups.
method Causal mediation analysis and stochastic block model explanation.
result Representation harm in contrastive learning is partly responsible for allocation harm in downstream tasks.
While harms of allocation have been increasingly studied as part of the subfield of algorithmic fairness, harms of representation have received considerably less attention. In this paper, we formalize two notions of stereotyping and show how they manifest in later allocative harms within the machine learning pipeline. …
We present FlipTest, a black-box technique for uncovering discrimination in classifiers. FlipTest is motivated by the intuitive question: had an individual been of a different protected status, would the model have treated them differently? Rather than relying on causal information to answer this question, FlipTest lev…
The issue of fairness in machine learning models has recently attracted a lot of attention as ensuring it will ensure continued confidence of the general public in the deployment of machine learning systems. We focus on mitigating the harm incurred by a biased machine learning system that offers better outputs (e.g. lo…
The paper proves neural networks are almost always surjective, impacting model safety.
problem Ensuring neural networks can generate any output, including harmful content.
method Analyzing fundamental neural architectures and generative models.
result Many neural architectures are almost always surjective, allowing for arbitrary outputs.
Aims to create safe reinforcement learning policies by considering individual harm.
problem Optimal policies for a population may harm certain individuals.
method Formalizes individual harm, proposes a two-stage procedure, and establishes finite-sample properties.
result Learned policies maximize expected return while minimizing harm.
Researchers classify harmful structures on unimodular Lie groups.
problem Characterizing harmful structures on Lie groups.
method Analyzing left-invariant structures and using Clifford multiplication.
result Classification of harmful structures on unimodular Lie groups of dimension ≤ 4.
Exploration-exploitation of functions, that is learning and optimizing a mapping between inputs and expected outputs, is ubiquitous to many real world situations. These situations sometimes require us to avoid certain outcomes at all cost, for example because they are poisonous, harmful, or otherwise dangerous. We test…
Prediction models can harm patients even when accurate, leading to self-fulfilling prophecies.
problem Prediction models can lead to harmful decisions that worsen patient outcomes.
method Formal characterization of harmful prediction models and analysis of their impact.
result Well-calibrated models are ineffective for decision-making as they do not change the data distribution.
New method stops experiments early for harm in diverse groups.
problem Early stopping of experiments for harmful treatment effects in diverse populations.
method Causal machine learning approach (CLASH) for early stopping.
result CLASH effectively stops experiments early for harmful treatment effects in diverse groups.
Theorem. Let M be a compact, connected, oriented smooth Riemannian n-manifold with non-empty boundary. Then the cohomology of the complex (Harm*(M),d) of harmonic forms on M is given by the direct sum H^p(Harm*(M),d) = H^p(M;R) + H^(p-1)(M;R) for p=0,1,...,n. When M is a closed manifold, a form is harmonic if and only …
New method estimates harmful instances in GANs for better model performance.
problem Estimating harmful instances in GANs for improved model performance.
method Uses Jacobian of generator's loss to trace harmfulness of instances.
result Correctly inferred changes in GAN evaluation metrics and improved model performance.
Recent advances in machine learning and artificial intelligence are now being considered in safety-critical autonomous systems where software defects may cause severe harm to humans and the environment. Design organizations in these domains are currently unable to provide convincing arguments that their systems are saf…
Study examines how decoding algorithms affect fairness in language generation models.
problem Impact of decoding algorithms on fairness in open-ended language generation.
method Systematic analysis of top-p, top-k, and temperature decoding algorithms. result Decoding algorithms significantly impact fairness across demographic groups.
Study shows harmful overfitting in Sobolev spaces even as training data grows.
problem Harmful overfitting in Sobolev spaces under noisy conditions.
method Geometric argument using Sobolev inequalities.
result Approximately norm-minimizing interpolators exhibit harmful overfitting.
New framework shows algorithmic recourse can be harmful.
problem Risks of providing algorithmic recourse in binary classification.
method Abstract learning-theoretic framework comparing risks with and without recourse.
result Providing recourse can be harmful, increasing class uncertainty and mistakes.
As machine learning (ML) increasingly affects people and society, awareness of its potential unwanted consequences has also grown. To anticipate, prevent, and mitigate undesirable downstream consequences, it is critical that we understand when and how harm might be introduced throughout the ML life cycle. In this paper…
Common fairness definitions in machine learning focus on balancing notions of disparity and utility. In this work, we study fairness in the context of risk disparity among sub-populations. We are interested in learning models that minimize performance discrepancies across sensitive groups without causing unnecessary ha…
Improves GAN performance by identifying and removing harmful training instances.
problem Challenges in applying previous instance removal methods to GANs.
method Proposes influence estimation methods using Jacobian of generator's gradient and instance evaluation scheme based on GAN evaluation metrics.
result Removing harmful instances significantly improves GAN performance.
The study finds that memorization is necessary or harmful depending on the prior distribution and noise level.
problem The impact of memorization on generalization in overparameterized models.
method An overparameterized linear model with general priors in a Bayesian setup.
result Explicit conditions for optimal generalization based on the prior distribution and noise level.
Paper tackles underranking in group-fair ranking systems, proving a trade-off and presenting an algorithm.
problem Underranking in group-fair ranking systems can worsen social and economic inequalities.
method Formulated underranking as a new problem, proved a lower bound, and presented a fair ranking algorithm.
result Algorithm achieves best of underranking and group fairness, confirming theoretical trade-off.
New framework assesses LLM security risks in BFSI.
problem Lack of domain-specific security evaluation for LLMs in BFSI.
method Risk-aware evaluation framework combining taxonomy, automated red-teaming, and ensemble judging.
result Higher decoding stochasticity and adaptive interaction lead to more severe disclosures.
Study characterizes harmful low-fidelity data sources for surrogate models.
problem Identifying which low-fidelity data sources to use in constructing surrogate models.
method Employed benchmark filtering techniques to assess harmful sources using limited data.
result Provided guidelines for using low-fidelity sources in an industrial setting.
Neural Networks are being integrated into safety critical systems, e.g., perception systems for autonomous vehicles, which require trained networks to perform safely in novel scenarios. It is challenging to verify neural networks because their decisions are not explainable, they cannot be exhaustively tested, and finit…
Introduces MPR to measure and optimize representation across intersectional groups in retrieval.
problem Harmful stereotypes, cultural erasure, and social disparities in image search and retrieval.
method Develops MPR metric, practical estimation methods, theoretical guarantees, and optimization algorithms.
result Optimizing MPR yields more proportional representation across multiple intersectional groups, often with minimal retrieval accuracy compromise.
Despite the vast success of Deep Neural Networks in numerous application domains, it has been shown that such models are not robust i.e., they are vulnerable to small adversarial perturbations of the input. While extensive work has been done on why such perturbations occur or how to successfully defend against them, we…
TENP prunes experts and neurons in Mixture-of-Experts models for efficient deployment.
problem Efficient deployment of large language models constrained by static parameter footprint.
method Structured Trapezoidal ExpertNeuron Pruning (TENP) identifies and retains important experts and neurons.
result DeepSeek model achieves 10% better performance on code generation tasks with 40% expert sparsity.
Develops a support-aware framework for reserve-policy selection in advertising markets.
problem Log-based reserve-price evaluation risks weak support and subgroup harm.
method Support-aware offline decision framework converting logged evidence into certified policies.
result Preserves the best gate-passing policy while eliminating only policies with certified regret.
Diffusion LLMs can efficiently generate harmful prompts for adversarial testing.
problem Generating harmful prompts for adversarial testing is resource-intensive and costly.
method Transformed adversarial prompt optimization into an efficient inference task using pretrained Diffusion LLMs.
result Only a few conditional samples are required to generate harmful prompts with high reward.
Detects harmful distribution shifts in deployed models without false alarms.
problem Detecting harmful distribution shifts in deployed models without false alarms.
method Sequential tools for testing if the difference between source and target distributions leads to a significant increase in a risk function.
result Demonstrated the efficacy of the proposed framework through extensive empirical studies.
Combining ensembles and data augmentation harms model calibration.
problem Improving model calibration and robustness with ensembles and data augmentation leads to a trade-off.
method Combining ensemble averaging and data augmentation techniques.
result Combining ensembles and data augmentation can harm model calibration.
Selective planning with imperfect models reduces harmful effects of model inadequacy.
problem Harmful effects of using an imperfect model in reinforcement learning.
method Selective planning with heteroscedastic regression to estimate predictive uncertainty from model inadequacy.
result Effective selective planning requires considering both parameter uncertainty and model inadequacy.
Data whitening and second order optimization harm generalization by reducing access to dataset information.
problem Harmful effects of data whitening and second order optimization on generalization in machine learning.
method Analysis of fully connected models and experimental verification.
result Data whitening and second order optimization reduce or prevent generalization by limiting access to dataset information.
Two-layer CNNs can overfit without harm under certain conditions.
problem Understanding when and how overfitting occurs in neural networks.
method Theoretical analysis of a two-layer CNN trained by gradient descent.
result A sharp phase transition between benign and harmful overfitting based on signal-to-noise ratio.
Prediction markets can be manipulated by traders who can move contract settlements, harming price discovery.
problem Manipulation of settlement times in prediction markets leads to unfair wealth transfer and harms price discovery.
method Developed a model showing how settlement manipulation transfers wealth and harms price discovery, and observed real-world effects on Polymarket's Bitcoin contract.
result Manipulators capture significant profits from retail traders, especially when settlement times are short.
About the economic growth the Keynesian theorists defend circular and cumulative processes, benefiting the rich localities and harming the poorest, without external interventions. In these processes the Verdoorn law has an important role. For Verdoorn (1949) the productivity growth rate is endogenous and depends of the…
Detects harmful shifts without labels for model performance.
problem Detecting distribution shifts without access to labels.
method Uses a proxy derived from predictions of an error estimator.
result High power and false alarm control under various shifts.
COCA refactors training data to identify and erase unsafe concepts in LLMs.
problem Identifying and erasing unsafe concepts in Large Language Models (LLMs) for safety alignment.
method Concept Concentration (COCA) refactors training data with an explicit reasoning process to identify and erase unsafe concepts.
result COCA significantly reduces both in-distribution and out-of-distribution jailbreak success rates while maintaining strong performance on regular tasks.
Algorithm removes spurious concepts from neural network representations without harming task performance.
problem Spurious correlations hinder neural network out-of-distribution generalization.
method Iterative algorithm that identifies two orthogonal subspaces in neural network representation.
result Algorithm outperforms existing methods on computer vision and natural language processing benchmarks.
Ethereum block builders can earn up to $14M/month by reordering transactions, harming users.
problem Block builders can exploit transaction reordering to earn significant profits, harming users.
method Estimation of MEV payments and analysis of reordering effects.
result Block builders can earn up to $14M/month by reordering transactions, skewing the distribution.
Adversarial training overfits, harming robustness; early stopping fixes this.
problem Adversarial robustness in deep learning overfits to training data.
method Empirical study of adversarially trained deep networks.
result Overfitting to training data harms robust performance in adversarial training.
Improving cancer treatment decisions requires considering causal effects, not just model accuracy.
problem Cancer outcome prediction models may cause harm when used for treatment decisions.
method Explains the importance of considering causal effects in model validation and provides guidelines.
result Building and validating models that are useful for decision making requires considering causal effects.
LLM safety alignment explained as divergence estimation.
problem Aligning large language models to avoid harmful outputs.
method Presented a theoretical framework showing alignment methods as divergence estimators.
result KLDO method improves safety alignment using compliance-refusal datasets.
New algorithm ensures fairness without sacrificing accuracy.
problem Ensuring fairness in machine learning without harming accuracy.
method Demographic-Agnostic Fairness without Harm (DAFH) algorithm.
result DAFH algorithm achieves higher accuracy than existing methods.