Strong inductive biases prevent harmless interpolation in overparameterized models.
problem Understanding the conditions under which overparameterized models can interpolate noise without overfitting.
method Theoretical analysis of high-dimensional kernel regression and deep neural networks, focusing on the role of inductive biases.
result The strength of an estimator's inductive bias determines whether interpolation is harmless or requires fitting noise for good generalization.
DatedGPT prevents lookahead bias in financial forecasting models.
problem Lookahead bias in large language models trained on internet-scale data.
method Time-aware pretraining with annual data cutoffs and instruction fine-tuning.
result Models' knowledge is effectively bounded by their data cutoff year, improving forecasting validity.
Framework audits synthetic datasets for trustworthiness across various use cases.
problem Assessing the trustworthiness of synthetic datasets and models.
method Holistic auditing framework focusing on bias, fidelity, utility, robustness, and privacy.
result Introduces a trustworthiness index and model selection process for controllable trade-offs.
New technique prevents Q-learning collapse by maximizing diversity among ensembles.
problem Value function collapse in ensemble Q-learning.
method Maximizing representation diversity through regularization.
result Regularized approach significantly outperforms existing methods.
Gradient flow in ReLU networks biases towards generalization but makes them vulnerable to adversarial attacks.
problem Generalization vs. Adversarial Robustness in ReLU Networks
method Analysis of gradient flow in two-layer ReLU networks with clustered data.
result Gradient flow biases towards generalization but also makes networks vulnerable to adversarial attacks.
Recurrent models can produce infinite sequences, causing bias; new methods prevent this.
problem Inconsistency in decoding infinite-length sequences from recurrent language models.
method Defined and proved inconsistency of common decoding algorithms; proposed remedies.
result Proposed methods prevent inconsistency in practice.
New method detects and prevents unfairness in few-shot regression models.
problem Fairness issues in supervised few-shot meta-learning models.
method Causal Bayesian knowledge graph for dependency visualization, risk difference quantification, and fast-adapted bias-control approach.
result Efficiently detects and mitigates unfairness in model predictions.
Improves decision-making by correcting feature selection bias.
problem Cofounding bias in feature selection affects machine learning predictions.
method Proposes a meta-algorithm using a novel adjustment criterion based on causal sufficiency.
result Corrects cofounding bias to improve prediction performance.
Graph data sets often contain isomorphism bias, artificially inflating model performance.
problem Isomorphism bias in graph data sets causing inflated model performance.
method Analysis of 54 graph data sets, recommendations for model setup, open sourcing new data sets.
result Graph data sets commonly contain isomorphism bias, artificially inflating model performance.
Proposes fair mapping to prevent bias in model predictions without distorting data.
problem Reduces bias in model predictions without altering the data distribution.
method Uses Wasserstein GAN and AttGAN frameworks to transform data distributions while preserving privacy and interpretability.
result Preserves data interpretability and fairness in subsequent analysis tasks.
A RL framework selects features to balance bias and accuracy dynamically.
problem Bias in automated feature selection when predictors are correlated.
method Multi-component reward function with policy gradient for dynamic regularization and bias mitigation.
result Model balances fairness and accuracy during training.
Gradient descent stagnates in low-precision, but unbiased rounding schemes improve convergence.
problem Stagnation of gradient descent in low-precision computation.
method Proposed unbiased stochastic rounding schemes that trade zero bias for larger probability of preserving small gradients.
result Unbiased rounding methods typically improve convergence rate of gradient descent for convex problems.
Gradient clipping helps private SGD converge despite potential bias.
problem Gradient clipping in private SGD can bias convergence.
method Theoretical analysis and empirical evaluation of gradient clipping effects.
result Gradient clipping can prevent convergence to stationary points and introduces bias.
Theoretical study shows AI models can recover from contaminated training data.
problem Data contamination in AI training can degrade model performance.
method Theoretical analysis and experiments on various data types.
result Models converge to true distribution under mild conditions, with rate dependent on real data fraction.
Proposes volumization for neural networks to control bias-variance tradeoff.
problem Improving generalization and preventing memorization in neural networks.
method Defines a physical volume for weights, interpolating between L2 and L∞ regularization.
result Volumization interpolates between weight decay and clipping, improving generalization.
Securely trains fair models using homomorphic encryption.
problem Protecting sensitive features while testing model fairness.
method Fully homomorphic encryption for training and testing.
result Practical application to adult income data set.
AMAGOLD improves stochastic gradient MCMC by infrequent Metropolis-Hastings corrections.
problem Bias in stochastic gradient Hamiltonian Monte Carlo (SGHMC).
method AMAGOLD infrequently uses Metropolis-Hastings corrections to remove bias, with a fixed step size schedule.
result AMAGOLD converges to the target distribution with a fixed, rather than a diminishing, step size, and at most a constant factor slower convergence rate.
Paper introduces SLS to improve label smoothing regularization.
problem Improve generalization of neural networks by softening labels.
method Structural Label Smoothing (SLS) to mitigate bias in Bayes error rate.
result Empirical validation shows SLS outperforms traditional label smoothing.
The objective of this paper is to measure the degree of home bias (HB) within holdings portfolio and to identify their determining factors. By following literature and an international capital asset pricing model, we have chosen quite a number of susceptible factors that impact HB. This model is, hence, estimated for 2…
What makes a paper independently reproducible? Debates on reproducibility center around intuition or assumptions but lack empirical results. Our field focuses on releasing code, which is important, but is not sufficient for determining reproducibility. We take the first step toward a quantifiable answer by manually att…
Non-discrimination is a recognized objective in algorithmic decision making. In this paper, we introduce a novel probabilistic formulation of data pre-processing for reducing discrimination. We propose a convex optimization for learning a data transformation with three goals: controlling discrimination, limiting distor…
This research secures deployed sentiment analysis models by identifying and defending against attack vectors.
problem Securing deployed machine learning models, particularly sentiment analysis systems, from adversarial attacks.
method BAD (Build, Attack, Defend) Architecture, evaluating two implementations.
result Demonstrated a viable methodology for securing machine learning models in production.
Improves group fairness in tensor completion by augmenting tensors with balanced entities.
problem Preventing discrimination in tensor decomposition based on social grounds.
method STAFF (Sparse Tensor Augmentation For Fairness) which augments tensors with balanced entities to mitigate imbalance and bias.
result Consistently shows the best trade-off between completion error and group fairness, reducing errors by 36% and 59% respectively.
Simplifies RL training with fewer techniques, reducing bias and instability.
problem Training instabilities and high sample complexity in RL.
method Introduced a simple deterministic policy gradient, used propensity estimation, and delayed policy updates.
result Improved performance and reduced sample complexity through these techniques.
We introduce several techniques for sampling and visualizing the latent spaces of generative models. Replacing linear interpolation with spherical linear interpolation prevents diverging from a model's prior distribution and produces sharper samples. J-Diagrams and MINE grids are introduced as visualizations of manifol…
New method prevents mode collapse in deep SVDD for anomaly detection.
problem Mode collapse in deep SVDD due to architectural constraints.
method Two regularizers: noise injection and minibatch variance penalization.
result Regularized deep SVDD outperforms state-of-the-art methods.
GRASP removes spurious correlations in fine-tuned models, improving task performance and reducing bias.
problem Fine-tuned models can latch onto spurious correlations, leading to bias and reduced generalization.
method GRASP identifies and removes spurious correlations from model weights without removing latent factors.
result GRASP significantly reduces bias and improves task performance in various fine-tuning tasks.
The paper investigates model collapse in language models from a probabilistic perspective.
problem Understanding and preventing model collapse in language model training.
method Investigates recursive parametric model training from a probabilistic standpoint, characterizing conditions for model collapse and proposing mitigation strategies.
result Progressively increasing sample size is necessary to prevent model collapse, with a superlinear growth rate required in the asymptotic regime.
SGD in linear regression overfits but performs well due to bias-variance trade-off.
problem Understanding overfitting in SGD for linear regression.
method Constant-stepsize SGD with iterate averaging or tail averaging, analyzing full eigenspectrum of data covariance matrix.
result Sharp excess risk bounds revealing bias-variance decomposition for SGD in linear regression.
The paper analyzes the implicit bias of SGD near loss manifold and provides new insights.
problem Understanding the implicit bias of SGD near loss manifolds in overparametrized models.
method Adapting ideas from Katzenberger (1991) to analyze SGD dynamics using a stochastic differential equation (SDE).
result SGD with label noise locally decreases the sharpness of loss, leading to a global analysis of implicit bias.
A2A metric evaluates bias correction methods, reducing ATE estimation errors.
problem Selection biases in non-randomized studies of medical treatments.
method Propensity score matching (PSM) with novel metric A2A.
result Reduces ATE estimation errors by up to 90% across synthetic and real-world datasets.
Metric evaluates symmetry-breaking in datasets, revealing severe biases.
problem Symmetry-breaking in datasets can hinder the performance of symmetry-aware methods.
method Developed a metric to quantify symmetry-breaking using a two-sample classifier test.
result Symmetry-breaking can prevent optimal performance of invariant methods, even when labels are invariant.
Automated decision making systems are increasingly being used in real-world applications. In these systems for the most part, the decision rules are derived by minimizing the training error on the available historical data. Therefore, if there is a bias related to a sensitive attribute such as gender, race, religion, e…
Study finds racial bias in pulse oximeter readings has minimal impact on ICU ventilation rates.
problem Racial disparities in pulse oximeter readings affect clinical decisions in ICU settings.
method Causal inference using path-specific effects and doubly robust estimator.
result Minimal impact of racial discrepancies on invasive ventilation rates, but more pronounced on ventilation duration.
Evaluating prediction models under covariate shift and selective labels
problem Model performance evaluation under distribution shift and selection bias
method Double machine learning
result Accurate estimation of target risk
Noise in imputed values corrects biases in machine learning models.
problem Systematic biases in imputed values affect downstream analyses.
method Introducing noise to imputed values to correct biases.
result Noise-corrected imputation methods produce unbiased estimates.
The paper analyzes how low-rank layers in neural networks improve generalization.
problem Understanding how low-rank layers affect generalization in neural networks.
method Applying Maurer's chain rule for Gaussian complexity to analyze rank and spectral norm constraints.
result Deep networks with low-rank layers achieve better generalization than those with full-rank layers.
"Overlearning" means that a model trained for a seemingly simple objective implicitly learns to recognize attributes and concepts that are (1) not part of the learning objective, and (2) sensitive from a privacy or bias perspective. For example, a binary gender classifier of facial images also learns to recognize races…
Kernel-smoothed scores improve diffusion models by reducing memorization.
problem Diffusion models can memorize training data, leading to biased samples.
method Interpret empirical score as noisy version of true score, kernel-smoothed.
result Kernel-smoothing reduces variance and improves generalization.
SCQRNN prevents quantile crossing and improves computational efficiency.
problem Quantile crossing issue in regression models.
method Integrates ad hoc sorting in training to prevent quantile crossing and enhance computational efficiency.
result SCQRNN achieves faster convergence and non-intersecting quantiles.
New method preserves privacy while detecting communities in distributed networks.
problem Privacy-preserving community detection in locally distributed multi-layer networks.
method Privacy-preserving Distributed Spectral Clustering (ppDSC) using randomized response mechanism.
result Developed a novel algorithm that maintains community structure while protecting privacy.
Mathematical model audits social media algorithms to prevent bias.
problem Algorithmic filtering can bias users' decisions and societal norms.
method Formalized mathematical framework for auditing social media algorithms.
result Data-driven statistical auditing procedure to regulate algorithmic bias.
New research challenges the independence assumption in neurosymbolic learning, leading to overconfident predictions and unrepresentable uncertainty.
problem The independence assumption in neurosymbolic learning systems can lead to overconfident predictions and hinder uncertainty quantification.
method The study proves the limitations of the independence assumption and introduces new loss functions that are non-convex and difficult to optimise.
result Neurosymbolic learning systems using the independence assumption are prone to overconfidence and cannot represent uncertainty over multiple valid options.
This report was commissioned by the Commission of Inquiry Respecting the Muskrat Falls Project to provide the national and international context in which the Muskrat Falls Project took place. The Commission asked for the report to cover three specific topics of questions: (1) What is the national and international cont…
High-dimensional kernel regression struggles due to rotational invariance.
problem Kernel ridge regression struggles in high dimensions due to rotational invariance.
method Analysis of kernel properties and their impact on high-dimensional data.
result Lower bound on generalization error for high-dimensional kernel regression.
How can we control for latent discrimination in predictive models? How can we provably remove it? Such questions are at the heart of algorithmic fairness and its impacts on society. In this paper, we define a new operational fairness criteria, inspired by the well-understood notion of omitted variable-bias in statistic…
Many immunization strategies have been proposed to prevent infectious viruses from spreading through a network. In this study, we propose efficient immunization strategies to prevent a default contagion that might occur in a financial network. An essential difference from the previous studies on immunization strategy i…
Study on parameter dynamics in exponential families under closed-loop learning.
problem Understanding and preventing convergence to biased absorbing states in model parameter estimation.
method Derived equations of motion for parameter dynamics in exponential families. Showed convergence to biased states under maximum likelihood estimation and proposed solutions.
result Closed-loop learning can lead to biased parameter estimates, but can be mitigated by using maximum a posteriori estimation or regularisation.