The paper explores how regularization can improve multi-objective learning with high-dimensional data.
problem Improving multi-objective learning with high-dimensional and costly data.
method A two-stage MOL framework that leverages low-dimensional structure.
result Vanilla regularization approaches often fail in multi-objective learning, and a two-stage framework can successfully exploit low-dimensional structure.
Framework for ensuring fairness in machine learning models across multiple groups.
problem Ensuring fairness in machine learning models across multiple groups.
method Introduces (s,G,α)−GMC for multi-dimensional mappings and constraint sets, proposing algorithms to achieve multicalibration. result Demonstrates the effectiveness of the framework on various scenarios, including image segmentation, hierarchical classification, and text generation.
A framework for anonymized risk sharing without revealing identities or preferences.
problem Risk sharing without revealing individual identities or preferences.
method Axiomatic framework with four key axioms: actuarial fairness, risk fairness, risk anonymity, and operational anonymity.
result The conditional mean risk sharing rule is uniquely characterized by these axioms.
The paper develops fair machine learning models using causal path-specific effects.
problem Fairness in machine learning models under causal constraints.
method Lagrange multiplier approach for infinite-dimensional functional estimation, closed-form solutions for constrained optimization.
result Theoretical and flexible semiparametric estimation strategies for fair predictions.
CCI combines Bayesian and gradient boosting to create fair, reliable credit risk scores.
problem Tackles high-stakes lending decisions with changing data distributions and fairness constraints.
method Combines Bayesian neural risk scorer and fairness-constrained gradient boosting with shift-aware fusion.
result CCI achieves best trade-off between discrimination, calibration, stability, and fairness.
The paper optimizes risk-sharing in decentralized networks.
problem Optimizing risk-sharing among networked agents.
method Analyzes actuarially fair risk-sharing rules among friends in a network.
result Characterizes the optimal signed linear risk-sharing rule.
Paper finds a method to compute fair risk-sharing rules.
problem Finding a fair and understandable risk-sharing rule.
method Established a one-to-one correspondence with a fixed point approach.
result Fast numerical method for computing AFPO risk-sharing rules.
Recommender systems are one of the most pervasive applications of machine learning in industry, with many services using them to match users to products or information. As such it is important to ask: what are the possible fairness risks, how can we quantify them, and how should we address them? In this paper we offer …
Ensuring that classifiers are non-discriminatory or fair with respect to a sensitive feature (e.g., race or gender) is a topical problem. Progress in this task requires fixing a definition of fairness, and there have been several proposals in this regard over the past few years. Several of these, however, assume either…
Unified framework for fair regression under demographic parity.
problem Ensuring fairness in regression tasks subject to demographic parity constraints.
method Proposes a unified framework applicable to various regression tasks with a broad spectrum of loss functions, derived a novel characterization of the fair risk minimizer, and established theoretical consistency and convergence rates.
result Effective minimization of risk while satisfying fairness constraints across various regression settings.
In this paper we develop a novel methodology for estimation of risk capital allocation. The methodology is rooted in the theory of risk measures. We work within a general, but tractable class of law-invariant coherent risk measures, with a particular focus on expected shortfall. We introduce the concept of fair capital…
The study quantifies and compares aleatoric and epistemic discrimination in ML models.
problem Sources of discrimination in ML models and their impact on performance.
method Quantifying aleatoric and epistemic discrimination using statistical experiments and model accuracy.
result State-of-the-art fairness interventions are effective at removing epistemic discrimination but not aleatoric discrimination in datasets with missing values.
The paper assesses fairness in risk score models, focusing on epistemic value.
problem Fairness of risk score models in communicating uncertainty.
method Identified key fairness desiderata, developed metrics for quantitative assessment, and applied methodology in two case studies.
result Introduced a novel calibration error metric for meaningful comparisons between groups of different sizes.
Paper uses deep learning for systemic risk measures.
problem Computing optimal capital allocations for systemic risk.
method Deep learning algorithms to solve primal and dual problems.
result Deep learning provides fair risk allocations.
We provide a general framework for characterizing the trade-off between accuracy and robustness in supervised learning. We propose a method and define quantities to characterize the trade-off between accuracy and robustness for a given architecture, and provide theoretical insight into the trade-off. Specifically we in…
A new framework enables real-time task trade-off control.
problem Conflict between multiple related tasks in a fixed model capacity.
method Formulates MTL as a preference-conditioned multiobjective optimization problem; uses a hypernetwork-based neural network.
result A single model can handle different trade-off preferences among multiple tasks.
New trade-off found between accuracy and adversarial robustness in regression.
problem Finding a balance between accuracy and robustness in regression models.
method Deriving a fundamental trade-off between standard and adversarial risk in regression with polynomial ridge functions.
result A necessary condition for achieving adversarial robustness without significant accuracy loss.
Paper explores robust regression methods and their bias-variance trade-off.
problem Understanding the trade-off between robust estimation and optimization methods.
method Examines traditional outlier-resistant robust estimation and robust optimization.
result Both methods follow converse strategies due to a bias-variance trade-off.
This paper studies trade-offs in private prediction methods.
problem Leakage of training data information in machine learning predictions.
method Private training and private prediction methods with trade-offs.
result Private training methods outperform private prediction methods in various settings.
Efficiently approximates fairness-accuracy trade-offs for diverse datasets.
problem Inherent trade-off between fairness and accuracy in machine learning models.
method You-Only-Train-Once (YOTO) framework for computationally efficient approximation.
result Robust methodology for auditing model fairness with statistical guarantees.
New insights into convergence and accuracy trade-offs in federated and meta-learning.
problem Understanding the trade-offs between convergence and accuracy in federated and meta-learning.
method Generalized local update methods, proving equivalence to first-order optimization on a surrogate loss.
result Novel convergence rates and insights into the importance of algorithmic choices in communication-limited settings.
We empirically investigate the best trade-off between sparse and uniformly-weighted multiple kernel learning (MKL) using the elastic-net regularization on real and simulated datasets. We find that the best trade-off parameter depends not only on the sparsity of the true kernel-weight spectrum but also on the linear dep…
Proposes FACT, a diagnostic for understanding group fairness trade-offs.
problem Group fairness notions often conflict with each other, requiring a cost in model performance.
method Characterizes trade-offs via the fairness-confusion tensor and optimizes accuracy and fairness objectives.
result Demonstrates the use of FACT on synthetic and real datasets to understand accuracy-fairness trade-offs.
Pareto MTL finds optimal solutions for multiple tasks with different trade-offs.
problem Finding a single optimal solution for multiple conflicting tasks.
method Formulate multi-task learning as multiobjective optimization, decompose into subproblems, solve in parallel.
result Generates well-representative Pareto optimal solutions for different trade-offs.
Study shows exponential error reduction in multiclass classification without bias-variance trade-off.
problem Multiclass classification with margin conditions.
method Analysis of classification error under hard-margin conditions.
result Exponential decrease in classification error without bias-variance trade-off.
A trade-off between accuracy and fairness is almost taken as a given in the existing literature on fairness in machine learning. Yet, it is not preordained that accuracy should decrease with increased fairness. Novel to this work, we examine fair classification through the lens of mismatched hypothesis testing: trying …
Vecchia approximations provide the best accuracy-runtime trade-off for Gaussian process approximations.
problem High computational cost of Gaussian processes for large data sets.
method Systematic comparison of different Gaussian process approximations.
result Vecchia approximations consistently provide the best accuracy-runtime trade-off.
The paper explores the trade-off between bias and variance in high-dimensional models.
problem Understanding the unavoidable trade-off between bias and variance in high-dimensional statistical models.
method Proposes a general strategy to obtain lower bounds on the variance of estimators with a specified bias, and applies it to various statistical models.
result Shows the extent to which the bias-variance trade-off is unavoidable and quantifies the performance loss for methods that do not balance it.
New fairness criterion for risk-sensitive decisions in regulated industries.
problem Ensuring equitable outcomes in risk-sensitive decision-making.
method Marginal fairness for generalized distortion risk measures, two-step decision-making process.
result Ensures fairness in decision-making under risk measures, regardless of protected attributes.
Fine-tuning LLMs improves capability but harms safety, study finds.
problem Balancing capability and safety in LLM fine-tuning.
method Theoretical framework and numerical experiments for two safety-aware fine-tuning strategies.
result Characterization of fundamental limits of safety-capability trade-off in LLM fine-tuning.
This paper analyzes the trade-off between accuracy and communication in personalized federated learning.
problem The accuracy-communication trade-off in personalized federated learning.
method The paper provides a quantitative characterization of the personalization degree on the trade-off, establishing minimax optimality.
result The paper offers theoretical insights for choosing the personalization degree and validates the results on synthetic and real-world datasets.
SCORE resolves the robustness vs accuracy trade-off by redefining robust error.
problem The inherent trade-off between robustness and accuracy in adversarial training.
method SCORE defines local equivariance as the ideal robust behavior, leading to a new robust error metric.
result SCORE reconciles robustness and accuracy, improving model performance on RobustBench.
Two Fisher information matrix estimators are analyzed for neural networks, focusing on their variances and trade-offs.
problem Estimating the Fisher information matrix in neural networks due to its high computational cost.
method Examined two popular diagonal Fisher information matrix estimators and their variances in neural networks for regression and classification.
result The variances of the estimators depend on the non-linearity with respect to different parameter groups and should not be neglected.
Method orders Pareto solutions using transformed objective scores.
problem Lack of orderability in multi-objective optimization solutions.
method Probability integral transform to map objectives to scores.
result Pareto efficient solutions can be ordered in score space.
Differential privacy is a mathematical framework for privacy-preserving data analysis. Changing the hyperparameters of a differentially private algorithm allows one to trade off privacy and utility in a principled way. Quantifying this trade-off in advance is essential to decision-makers tasked with deciding how much p…
SharpBalance improves deep ensemble performance by balancing sharpness and diversity.
problem Improving deep ensemble performance in both in-distribution and out-of-distribution scenarios.
method Introducing SharpBalance, a novel training approach that balances sharpness and diversity within ensembles.
result SharpBalance effectively improves the sharpness-diversity trade-off and ensemble performance in ID and OOD scenarios.
Research explores the trade-off between multi-task learning and multitasking in deep neural networks.
problem Trade-off between multi-task learning and multitasking in deep neural networks.
method Meta-learning algorithm to manage the trade-off between shared and separated representations.
result Agent successfully optimizes training strategy based on environment.
VFDS selects dynamic features for efficient HAR tasks, optimizing performance-cost trade-offs.
problem Optimizing feature selection for varying costs and dynamic contexts in machine learning tasks.
method Bayesian learning framework with variational dynamic selection policy.
result VFDS selects different features under changing contexts, saving sensory costs while maintaining HAR accuracy.
New method optimizes privacy and compute trade-offs for deep learning.
problem Privacy and compute trade-offs in deep learning training.
method Decoupling privacy analysis and experimental behavior, using TAN and scaling laws for DP-SGD.
result Stronger privacy guarantees with significant reduction in computational budget.
We propose a new PAC-Bayesian bound and a way of constructing a hypothesis space, so that the bound is convex in the posterior distribution and also convex in a trade-off parameter between empirical performance of the posterior distribution and its complexity. The complexity is measured by the Kullback-Leibler divergen…
Paper develops a dynamic Bayesian approach for active learning that optimizes exploration-exploitation balance.
problem Balancing exploration and exploitation in active learning for unknown functions.
method Develops BHEEM, a Bayesian hierarchical approach with approximate Bayesian computation for sampling trade-off parameters.
result BHEEM achieves at least 21% and 11% improvement over pure exploration and exploitation strategies respectively.
Study tests whether trade-off functions are above or below benchmarks using finite samples.
problem Testing trade-off functions between unknown distributions.
method Identifies a condition for nontrivial testing, constructs a test with error guarantees, and inverts the test for confidence bands.
result Finite-sample testing is possible under specific structural assumptions about rejection regions.
Study on trade-offs between accuracy and interpretability in machine learning.
problem Lack of formal study on statistical cost of interpretability.
method Modeling interpretability as a constraint in empirical risk minimization for binary classification.
result Explains conditions under which accuracy trade-off occurs with interpretability constraints.
New insights into neural network forgetting reveal a trade-off between node activation and re-use.
problem Challenges in maintaining performance on old tasks while learning new ones.
method Theoretical analysis of synthetic and real data setups, focusing on node activation vs re-use.
result Worst forgetting occurs in an intermediate similarity regime between learned tasks.
Bayesian optimization offers the possibility of optimizing black-box operations not accessible through traditional techniques. The success of Bayesian optimization methods such as Expected Improvement (EI) are significantly affected by the degree of trade-off between exploration and exploitation. Too much exploration c…
Learning algorithms need bias to generalize and perform better than random guessing. We examine the flexibility (expressivity) of biased algorithms. An expressive algorithm can adapt to changing training data, altering its outcome based on changes in its input. We measure expressivity by using an information-theoretic …
Improved Bayesian optimisation method using randomised Gaussian process UCB.
problem Improving performance in Bayesian optimisation.
method Developed a modified Gaussian process upper confidence bound (GP-UCB) acquisition function.
result The method achieves better performance than GP-UCB in various problems.
Meta-learning balances task-specific modeling and optimization complexity.
problem Balancing accurate task-specific modeling with ease of optimization in meta-learning.
method Theoretical and empirical analysis of trade-off between modeling and optimization in meta-learning.
result Explicit bounds on modeling and optimization errors for non-convex and linear regression problems.