Unintended effects from scaling neural network outputs with adaptive learning rates.
problem Adaptive learning rate optimization's behavior is altered by output scaling, leading to misinterpretation.
method Presented a modified optimization algorithm to mitigate unintended effects.
result Adaptive learning rate's effectiveness is significantly impacted by output scaling, especially for small scaling factors.
Study shows FL reduces unintended memorization by clustering data and using strong user-level privacy.
problem Unintended memorization in federated learning.
method Examined the effect of clustering data and using strong user-level differential privacy in FL.
result Clustering data and strong user-level differential privacy reduce unintended memorization.
Unintended bias in Machine Learning can manifest as systemic differences in performance for different demographic groups, potentially compounding existing challenges to fairness in society at large. In this paper, we introduce a suite of threshold-agnostic metrics that provide a nuanced view of this unintended bias, by…
This report examines the Pinned AUC metric introduced and highlights some of its limitations. Pinned AUC provides a threshold-agnostic measure of unintended bias in a classification model, inspired by the ROC-AUC metric. However, as we highlight in this report, there are ways that the metric can obscure different kinds…
Mitigates bias in text classification by weighting instances.
problem Unintended biases in text classification datasets based on demographic terms.
method Instance weighting to recover non-discrimination distribution.
result Effective mitigation of unintended biases without sacrificing generalization.
Proposes a test to ensure predictive algorithms predict intended outcomes better than unintended ones.
problem Unintended model behavior leading to prediction of unintended outcomes.
method Falsification framework using nonparametric hypothesis testing to compare prediction losses across outcomes.
result Establishes discriminant validity with respect to gender but not race in an admissions setting.
Facial analysis models are increasingly used in applications that have serious impacts on people's lives, ranging from authentication to surveillance tracking. It is therefore critical to develop techniques that can reveal unintended biases in facial classifiers to help guide the ethical use of facial analysis technolo…
This work identifies and mitigates reasoning shortcuts in Neuro-Symbolic models.
problem Neuro-Symbolic models can achieve high accuracy by using unintended concepts.
method Characterized reasoning shortcuts as unintended optima of the learning objective and identified four key conditions.
result Reasoning shortcuts are difficult to mitigate, casting doubt on NeSy solutions' trustworthiness and interpretability.
Arguments in favor of injecting symbolic knowledge into neural architectures abound. When done right, constraining a sub-symbolic model can substantially improve its performance and sample complexity and prevent it from predicting invalid configurations. Focusing on deep probabilistic (logical) graphical models -- i.e.…
We look at the effect of the tick size changes on the TOPIX 100 index names made by the Tokyo Stock Exchange on Jan-14-2014 and Jul-22-2104. The intended consequence of the change is price improvement and shorter time to execution. We look at security level metrics that include the spread, trading volume, number of tra…
New findings reveal discount regularization can be seen as a strong prior, leading to poor performance in unevenly sampled data.
problem Discount regularization leads to poor performance in unevenly sampled data.
method Equivalence theorem showing discount regularization as a strong prior, setting regularization parameters locally for individual state-action pairs.
result Discount regularization can be seen as a strong prior, leading to poor performance in unevenly sampled data.
New method stops experiments early for harm in diverse groups.
problem Early stopping of experiments for harmful treatment effects in diverse populations.
method Causal machine learning approach (CLASH) for early stopping.
result CLASH effectively stops experiments early for harmful treatment effects in diverse groups.
SPAT improves adversarial robustness by preserving semantics in adversarial training.
problem Adversarial examples often have different semantics than original data, introducing unintended biases.
method Semantics-preserving adversarial training (SPAT) that encourages pixel perturbation shared among all classes.
result SPAT improves adversarial robustness and achieves state-of-the-art results in CIFAR-10 and CIFAR-100.
A number of machine learning (ML) methods have been proposed recently to maximize model predictive accuracy while enforcing notions of group parity or fairness across sub-populations. We propose a desirable property for these procedures, slack-consistency: For any individual, the predictions of the model should be mono…
Machine unlearning can compromise privacy, study shows.
problem Machine unlearning may leave data imprints in ML models, risking privacy.
method Proposed a membership inference attack to detect leakage.
result Machine unlearning can lead to unintended privacy risks.
Decisions taken in our everyday lives are based on a wide variety of information so it is generally very difficult to assess what are the strategies that guide us. Stock market therefore provides a rich environment to study how people take decision since responding to market uncertainty needs a constant update of these…
SHIFT framework identifies subgroups with large ML model performance decay.
problem Large model performance decay in subgroups when deployed.
method Subgroup-scanning Hierarchical Inference Framework (SHIFT) for performance drift.
result SHIFT identifies interpretable subgroups with large performance decay and suggests targeted actions to mitigate it.
Study shows online learning algorithms incentivize low-quality content, proposing new algorithms to improve quality.
problem Online learning algorithms in content recommender systems incentivize producers to create low-quality content.
method Analyzed the game between producers and content quality, designed new learning algorithms to incentivize high effort and quality.
result New algorithms incentivize producers to invest high effort and achieve high user welfare, improving content quality.
Proposes a falsification framework to test algorithmic discriminant validity.
problem Unintended model behavior in predictive algorithms.
method Falsification framework based on statistical tests comparing prediction losses across outcomes.
result Establishes discriminant validity for some outcomes but not others.
The paper tests semantic importance in opaque models using betting.
problem Precise statistical guarantees for semantic concepts in black-box models.
method Formalizes global and local statistical importance via conditional independence and SKIT.
result Shows effectiveness and flexibility of the framework on various models.
Machine Learning (ML) algorithms are used to train computers to perform a variety of complex tasks and improve with experience. Computers learn how to recognize patterns, make unintended decisions, or react to a dynamic environment. Certain trained machines may be more effective than others because they are based on mo…
We investigated publicly reported security breaches of internal controls in corporate systems to determine whether SOX assessments are information bearing with respect to breaches which can lead to materially significant losses and misstatements. SOX Section 404 adverse decisions on effectiveness of controls occurred i…
Enhances machine learning models by preserving data structure, addressing statistical distortions.
problem Statistical distortions in synthetic data generated by Mixup.
method Proposes a generalized mixup method with a flexible weighting scheme to preserve data structure.
result Preserves statistical properties of original data while maintaining model performance.
Combining Bayesian deep learning and split conformal prediction affects out-of-distribution coverage.
problem Improving out-of-distribution coverage in multiclass image classification.
method Combining Bayesian deep learning with split conformal prediction methods.
result Combining methods can reduce out-of-distribution coverage in some cases.
We discuss the objectives of automation equipped with non-trivial decision making, or creating artificial intelligence, in the financial markets and provide a possible alternative. Intelligence might be an unintended consequence of curiosity left to roam free, best exemplified by a frolicking infant. For this unintenti…
Transfer learning --- transferring learned knowledge --- has brought a paradigm shift in the way models are trained. The lucrative benefits of improved accuracy and reduced training time have shown promise in training models with constrained computational resources and fewer training samples. Specifically, publicly ava…
As recent literature has demonstrated how classifiers often carry unintended biases toward some subgroups, deploying machine learned models to users demands careful consideration of the social consequences. How should we address this problem in a real-world system? How should we balance core performance and fairness me…
Enhances reward specification in RL with a novel language-based approach.
problem Reward specification in RL can lead to unintended, potentially harmful behaviours.
method Developed a novel class of language-based Reward Machines using RML's built-in memory.
result Can specify non-regular, non-Markovian reward functions for complex tasks.
BC-Aligner maintains backward compatibility of embeddings after frequent updates.
problem Updating embeddings without requiring consumer teams to retrain their models.
method Learning backward compatible embeddings through BC-Aligner.
result BC-Aligner maintains backward compatibility with existing unintended tasks after multiple model version updates.
New research suggests privileged information doesn't improve model performance.
problem Challenges in transferring knowledge using privileged information in machine learning.
method Critical examination of existing theoretical and empirical analyses of LUPI methods.
result LUPI methods often fail to effectively transfer knowledge from privileged information.
We look at a collection of conjectures with the unifying message that smaller social systems, tend to be less complex and can be aligned better, towards fulfilling their intended objectives. We touch upon a framework, referred to as the four pronged approach that can aid the analysis of social systems. The four prongs …
The study quantifies the impact of fund miscategorization using machine learning.
problem The impact of fund miscategorization on investment decisions.
method Formulated as a distance-based outlier detection problem, used Random Forest based distance metric learning.
result Identified funds with strong relationship to future returns as outliers.
Framework for safely updating machine learning models.
problem Continuous updates to machine learning models can lead to unintended consequences.
method Formalizes the problem as computing the largest locally invariant domain (LID), uses tractable primal-dual formulation.
result Matches or exceeds heuristic baselines for avoiding forgetting while providing formal safety guarantees.
Paper bridges AI/ML and causal modeling to reduce bias.
problem Difficulty in combining methods from different assumptions.
method Integrates system dynamics and structural equation modeling.
result Unified mathematical framework for AI/ML and causal modeling.
Recommender systems often rely on models which are trained to maximize accuracy in predicting user preferences. When the systems are deployed, these models determine the availability of content and information to different users. The gap between these objectives gives rise to a potential for unintended consequences, co…
Recent research has helped to cultivate growing awareness that machine learning systems fueled by big data can create or exacerbate troubling disparities in society. Much of this research comes from outside of the practicing data science community, leaving its members with little concrete guidance to proactively addres…
Previous studies have found that an adversary attacker can often infer unintended input information from intermediate-layer features. We study the possibility of preventing such adversarial inference, yet without too much accuracy degradation. We propose a generic method to revise the neural network to boost the challe…
The wide and rapid adoption of deep learning by practitioners brought unintended consequences in many situations such as in the infamous case of Google Photos' racist image recognition algorithm; thus, necessitated the utilization of the quantified uncertainty for each prediction. There have been recent efforts towards…
Fairness is a critical trait in decision making. As machine-learning models are increasingly being used in sensitive application domains (e.g. education and employment) for decision making, it is crucial that the decisions computed by such models are free of unintended bias. But how can we automatically validate the fa…
New framework quantifies and reduces concept-based models' leakage.
problem Information leakage in concept-based models reduces interpretability.
method Information-theoretic framework with CTL and ICL measures.
result Measures predict model behaviour and identify leakage causes.
Learning reward functions from data is a promising path towards achieving scalable Reinforcement Learning (RL) for robotics. However, a major challenge in training agents from learned reward models is that the agent can learn to exploit errors in the reward model to achieve high reward behaviors that do not correspond …
New method uses coupled SDEs to edit images with high fidelity and consistency.
problem Challenges in editing image content with text-to-image models.
method Using coupled stochastic differential equations to guide generative model sampling.
result Achieves high prompt fidelity and near-pixel-level consistency.
Paper improves DOA estimation in sparse arrays using Siamese neural networks.
problem Challenges in DOA estimation with limited snapshots in sparse linear arrays.
method Introduces a Siamese neural network with a sparse augmentation layer for enhanced signal feature embedding.
result Demonstrates improved DOA estimation accuracy in sparse arrays.
Learning robot objective functions from human input has become increasingly important, but state-of-the-art techniques assume that the human's desired objective lies within the robot's hypothesis space. When this is not true, even methods that keep track of uncertainty over the objective fail because they reason about …
Introduces FairCOCCO for fair learning with multitype, multivariate sensitive attributes.
problem Fairness in machine learning with multiple, complex sensitive attributes.
method FairCOCCO measure based on cross-covariance operators, incorporating a regularisation term.
result Consistent improvements in balancing fairness and predictive power on real-world datasets.
Export bans during pandemic worsen medical supply shortages globally.
problem Impact of export restrictions on medical goods during pandemic.
method Model of shock diffusion through international trade network.
result Export bans are counterproductive, making most countries worse off.
Reinforcement learning (RL) for robotics is challenging due to the difficulty in hand-engineering a dense cost function, which can lead to unintended behavior, and dynamical uncertainty, which makes exploration and constraint satisfaction challenging. We address these issues with a new model-based reinforcement learnin…
Image recognition systems have demonstrated tremendous progress over the past few decades thanks, in part, to our ability of learning compact and robust representations of images. As we witness the wide spread adoption of these systems, it is imperative to consider the problem of unintended leakage of information from …