Researchers classify harmful structures on unimodular Lie groups.
problem Characterizing harmful structures on Lie groups.
method Analyzing left-invariant structures and using Clifford multiplication.
result Classification of harmful structures on unimodular Lie groups of dimension ≤ 4.
Proposes a new method to measure and avoid harm in machine learning decisions.
problem Measuring and avoiding harm in machine learning algorithms.
method Formal definition of harm and benefit using causal models, counterfactual objective functions.
result Demonstrates that standard machine learning methods can lead to harmful policies under distributional shifts.
Contrastive learning harms minority group representations, affecting downstream tasks.
problem Representation harm in contrastive learning, especially affecting minority groups.
method Causal mediation analysis and stochastic block model explanation.
result Representation harm in contrastive learning is partly responsible for allocation harm in downstream tasks.
While harms of allocation have been increasingly studied as part of the subfield of algorithmic fairness, harms of representation have received considerably less attention. In this paper, we formalize two notions of stereotyping and show how they manifest in later allocative harms within the machine learning pipeline. …
Aims to create safe reinforcement learning policies by considering individual harm.
problem Optimal policies for a population may harm certain individuals.
method Formalizes individual harm, proposes a two-stage procedure, and establishes finite-sample properties.
result Learned policies maximize expected return while minimizing harm.
Prediction models can harm patients even when accurate, leading to self-fulfilling prophecies.
problem Prediction models can lead to harmful decisions that worsen patient outcomes.
method Formal characterization of harmful prediction models and analysis of their impact.
result Well-calibrated models are ineffective for decision-making as they do not change the data distribution.
New method stops experiments early for harm in diverse groups.
problem Early stopping of experiments for harmful treatment effects in diverse populations.
method Causal machine learning approach (CLASH) for early stopping.
result CLASH effectively stops experiments early for harmful treatment effects in diverse groups.
Theorem. Let M be a compact, connected, oriented smooth Riemannian n-manifold with non-empty boundary. Then the cohomology of the complex (Harm*(M),d) of harmonic forms on M is given by the direct sum H^p(Harm*(M),d) = H^p(M;R) + H^(p-1)(M;R) for p=0,1,...,n. When M is a closed manifold, a form is harmonic if and only …
Multi-scanner Antivirus systems provide insightful information on the nature of a suspect application; however there is often a lack of consensus and consistency between different Anti-Virus engines. In this article, we analyze more than 250 thousand malware signatures generated by 61 different Anti-Virus engines after…
New method estimates harmful instances in GANs for better model performance.
problem Estimating harmful instances in GANs for improved model performance.
method Uses Jacobian of generator's loss to trace harmfulness of instances.
result Correctly inferred changes in GAN evaluation metrics and improved model performance.
Study shows harmful overfitting in Sobolev spaces even as training data grows.
problem Harmful overfitting in Sobolev spaces under noisy conditions.
method Geometric argument using Sobolev inequalities.
result Approximately norm-minimizing interpolators exhibit harmful overfitting.
New framework shows algorithmic recourse can be harmful.
problem Risks of providing algorithmic recourse in binary classification.
method Abstract learning-theoretic framework comparing risks with and without recourse.
result Providing recourse can be harmful, increasing class uncertainty and mistakes.
As machine learning (ML) increasingly affects people and society, awareness of its potential unwanted consequences has also grown. To anticipate, prevent, and mitigate undesirable downstream consequences, it is critical that we understand when and how harm might be introduced throughout the ML life cycle. In this paper…
Common fairness definitions in machine learning focus on balancing notions of disparity and utility. In this work, we study fairness in the context of risk disparity among sub-populations. We are interested in learning models that minimize performance discrepancies across sensitive groups without causing unnecessary ha…
Improves GAN performance by identifying and removing harmful training instances.
problem Challenges in applying previous instance removal methods to GANs.
method Proposes influence estimation methods using Jacobian of generator's gradient and instance evaluation scheme based on GAN evaluation metrics.
result Removing harmful instances significantly improves GAN performance.
Bayesian approach quantifies uncertainty in LLM evaluations.
problem Statistical uncertainty in evaluating LLM behavior.
method Bayesian evaluation of LLM behavior using probabilistic text generation strategies.
result Bayesian approach provides useful uncertainty quantification about LLM behavior.
The study finds that memorization is necessary or harmful depending on the prior distribution and noise level.
problem The impact of memorization on generalization in overparameterized models.
method An overparameterized linear model with general priors in a Bayesian setup.
result Explicit conditions for optimal generalization based on the prior distribution and noise level.
New framework assesses LLM security risks in BFSI.
problem Lack of domain-specific security evaluation for LLMs in BFSI.
method Risk-aware evaluation framework combining taxonomy, automated red-teaming, and ensemble judging.
result Higher decoding stochasticity and adaptive interaction lead to more severe disclosures.
Study characterizes harmful low-fidelity data sources for surrogate models.
problem Identifying which low-fidelity data sources to use in constructing surrogate models.
method Employed benchmark filtering techniques to assess harmful sources using limited data.
result Provided guidelines for using low-fidelity sources in an industrial setting.
We present a non-naive version of the Precautionary (PP) that allows us to avoid paranoia and paralysis by confining precaution to specific domains and problems. PP is intended to deal with uncertainty and risk in cases where the absence of evidence and the incompleteness of scientific knowledge carries profound implic…
Introduces MPR to measure and optimize representation across intersectional groups in retrieval.
problem Harmful stereotypes, cultural erasure, and social disparities in image search and retrieval.
method Develops MPR metric, practical estimation methods, theoretical guarantees, and optimization algorithms.
result Optimizing MPR yields more proportional representation across multiple intersectional groups, often with minimal retrieval accuracy compromise.
New metric predicts neural network reliability under novel conditions.
problem Verifying neural networks' safety in novel scenarios.
method ML Dependability metric, Task Undependability, Harmful Undependability.
result Accurately predicts reliability under novel conditions.
Diffusion LLMs can efficiently generate harmful prompts for adversarial testing.
problem Generating harmful prompts for adversarial testing is resource-intensive and costly.
method Transformed adversarial prompt optimization into an efficient inference task using pretrained Diffusion LLMs.
result Only a few conditional samples are required to generate harmful prompts with high reward.
Detects harmful distribution shifts in deployed models without false alarms.
problem Detecting harmful distribution shifts in deployed models without false alarms.
method Sequential tools for testing if the difference between source and target distributions leads to a significant increase in a risk function.
result Demonstrated the efficacy of the proposed framework through extensive empirical studies.
Combining ensembles and data augmentation harms model calibration.
problem Improving model calibration and robustness with ensembles and data augmentation leads to a trade-off.
method Combining ensemble averaging and data augmentation techniques.
result Combining ensembles and data augmentation can harm model calibration.
Selective planning with imperfect models reduces harmful effects of model inadequacy.
problem Harmful effects of using an imperfect model in reinforcement learning.
method Selective planning with heteroscedastic regression to estimate predictive uncertainty from model inadequacy.
result Effective selective planning requires considering both parameter uncertainty and model inadequacy.
RESTA defends LLMs against jailbreaking attacks by adding random noise to embeddings.
problem Vulnerability of LLMs to jailbreaking attacks that generate harmful outputs.
method Adds random noise to embedding vectors and aggregates during token generation.
result RESTA achieves superior robustness versus utility tradeoffs compared to baseline defenses.
Data whitening and second order optimization harm generalization by reducing access to dataset information.
problem Harmful effects of data whitening and second order optimization on generalization in machine learning.
method Analysis of fully connected models and experimental verification.
result Data whitening and second order optimization reduce or prevent generalization by limiting access to dataset information.
Two-layer CNNs can overfit without harm under certain conditions.
problem Understanding when and how overfitting occurs in neural networks.
method Theoretical analysis of a two-layer CNN trained by gradient descent.
result A sharp phase transition between benign and harmful overfitting based on signal-to-noise ratio.
Prediction markets can be manipulated by traders who can move contract settlements, harming price discovery.
problem Manipulation of settlement times in prediction markets leads to unfair wealth transfer and harms price discovery.
method Developed a model showing how settlement manipulation transfers wealth and harms price discovery, and observed real-world effects on Polymarket's Bitcoin contract.
result Manipulators capture significant profits from retail traders, especially when settlement times are short.
Detects harmful shifts without labels for model performance.
problem Detecting distribution shifts without access to labels.
method Uses a proxy derived from predictions of an error estimator.
result High power and false alarm control under various shifts.
COCA refactors training data to identify and erase unsafe concepts in LLMs.
problem Identifying and erasing unsafe concepts in Large Language Models (LLMs) for safety alignment.
method Concept Concentration (COCA) refactors training data with an explicit reasoning process to identify and erase unsafe concepts.
result COCA significantly reduces both in-distribution and out-of-distribution jailbreak success rates while maintaining strong performance on regular tasks.
Algorithm removes spurious concepts from neural network representations without harming task performance.
problem Spurious correlations hinder neural network out-of-distribution generalization.
method Iterative algorithm that identifies two orthogonal subspaces in neural network representation.
result Algorithm outperforms existing methods on computer vision and natural language processing benchmarks.
Ethereum block builders can earn up to $14M/month by reordering transactions, harming users.
problem Block builders can exploit transaction reordering to earn significant profits, harming users.
method Estimation of MEV payments and analysis of reordering effects.
result Block builders can earn up to $14M/month by reordering transactions, skewing the distribution.
Adversarial training overfits, harming robustness; early stopping fixes this.
problem Adversarial robustness in deep learning overfits to training data.
method Empirical study of adversarially trained deep networks.
result Overfitting to training data harms robust performance in adversarial training.
Improving cancer treatment decisions requires considering causal effects, not just model accuracy.
problem Cancer outcome prediction models may cause harm when used for treatment decisions.
method Explains the importance of considering causal effects in model validation and provides guidelines.
result Building and validating models that are useful for decision making requires considering causal effects.
The study categorizes reward errors in reinforcement learning, finding some can be beneficial.
problem Training language models with imperfect proxy rewards.
method Theoretical analysis of policy gradient optimization and categorization of reward errors.
result Reward errors can be benign or even beneficial, preventing policy from stalling.
LLM safety alignment explained as divergence estimation.
problem Aligning large language models to avoid harmful outputs.
method Presented a theoretical framework showing alignment methods as divergence estimators.
result KLDO method improves safety alignment using compliance-refusal datasets.
New algorithm ensures fairness without sacrificing accuracy.
problem Ensuring fairness in machine learning without harming accuracy.
method Demographic-Agnostic Fairness without Harm (DAFH) algorithm.
result DAFH algorithm achieves higher accuracy than existing methods.
Cold posteriors in BNNs harm performance, likely due to incorrect likelihood.
problem Cold posteriors in Bayesian neural networks degrade performance.
method Developed a generative model explaining cold posteriors and matched it to the tempered likelihoods.
result Cold posteriors are a result of using the wrong likelihood for image classification datasets.
LLM sandbox and persona dynamics create unethical reality gaps that shift risk to users.
problem Ethical issues arise from LLMs generating reality gaps that shift risk to uninformed users.
method Analyzes the ethical implications of LLM sandbox and persona dynamics, comparing them to financial regulation and compliance.
result Active generation of reality gaps is unethical as it shifts epistemic risk to users.
Automated scoring engines are increasingly being used to score the free-form text responses that students give to questions. Such engines are not designed to appropriately deal with responses that a human reader would find alarming such as those that indicate an intention to self-harm or harm others, responses that all…
Study shows market volatility affects optimal communication design for trading strategies.
problem Investigating how communication impacts trading strategy performance in multi-agent systems.
method 5-agent LLM-based trading systems across 450 experiments spanning 21 months, comparing 5 organizational structures.
result Communication improves performance but depends on market characteristics, with competitive conversation excelling in volatile tech stocks.
Campfire simplifies sparse training for CNNs, achieving high accuracy with minimal pruning.
problem Training CNNs with sparse weights to maintain high accuracy and robustness.
method Gradual pruning technique with simplified sparsity structure, exploring various tradeoffs.
result Sparse ResNet-50 and ResNet-50 v1.5 achieve 75% top-1 accuracy with 70% sparsity, <1% accuracy loss.
Personalized models using group attributes reduce performance, study finds.
problem Reducing performance of models using group attributes like race or gender.
method Formal conditions and collective preference guarantees to ensure fair use.
result Models personalized with group attributes reduce performance at a group level.
We present FlipTest, a black-box technique for uncovering discrimination in classifiers. FlipTest is motivated by the intuitive question: had an individual been of a different protected status, would the model have treated them differently? Rather than relying on causal information to answer this question, FlipTest lev…
Study examines how bank holding structures affect financial stress spread.
problem Financial stress spread in a network of bank holdings and subsidiaries.
method Investigates the spread of contagion in a multilayered banking network with different holding support rules.
result Holding structures can either amplify or mitigate financial stress, depending on network capitalization.
The paper analyzes how deep neural networks handle noisy labels and finds disparate impacts.
problem Disparate impacts of noisy labels on instances with different representation frequencies.
method Quantifying harms, analyzing solutions, and comparing their impacts on different frequency instances.
result Existing solutions lead to disparate treatments, benefiting higher-frequency instances more.