Unified approach improves accuracy in private estimation.
problem Protecting sensitive information in statistical estimation.
method Unified Bayes Private Point (UBaPP) approach to Bayes point estimation under differential privacy constraints.
result Achieves better accuracy-privacy trade-off than traditional methods.
DP-BNNs improve accuracy, privacy, and reliability in neural networks.
problem Balancing privacy and accuracy in neural networks.
method Proposed three DP-BNNs: DP-SGLD, DP-BBP, and DP-MC Dropout.
result DP-SGLD achieves high accuracy under strong privacy guarantees.
New algorithm protects privacy in IVaR regression while maintaining accuracy.
problem Privacy leakage in classical IVaR methods.
method Noisy two-stage gradient descent with differential privacy guarantees.
result Achieves statistical efficiency and privacy in IVaR regression.
Semi-supervised learning has proven to be a powerful paradigm for leveraging unlabeled data to mitigate the reliance on large labeled datasets. In this work, we unify the current dominant approaches for semi-supervised learning to produce a new algorithm, MixMatch, that works by guessing low-entropy labels for data-aug…
Study on private and robust multi-armed bandits with contaminated heavy-tailed rewards.
problem Private and robust multi-armed bandits with contaminated heavy-tailed rewards.
method Proposed a meta-algorithm with a private and robust mean estimation sub-routine exttt{PRM}.
result Achieved nearly-optimal regret for two heavy-tailed settings.
Private statistics estimation faces a bias, accuracy, and privacy trilemma.
problem Balancing privacy, accuracy, and bias in statistical estimation.
method Use differential privacy (DP) for private statistics, but clip samples to control sensitivity and add noise for privacy, introducing bias.
result No algorithm can simultaneously have low bias, low error, and low privacy loss for arbitrary distributions.
Paper presents privacy-preserving techniques for HD computing.
problem Privacy loss in HD computing due to reversible computation.
method Quantization and pruning of hypervectors for differential privacy.
result Differentially private HD model for cloud inference.
Improved federated learning methods for privacy and efficiency.
problem Privacy and efficiency trade-offs in federated learning.
method FedHybrid and FedNewton: improved federated learning algorithms.
result Achieved comparable accuracy to FedSGD with fewer communication rounds.
First DP algorithm for Wasserstein barycenters on private data.
problem Computing Wasserstein barycenters on private datasets.
method Differentially private algorithms for Wasserstein barycenters.
result High-quality private barycenters with strong accuracy-privacy tradeoffs.
We present and evaluate Deep Private-Feature Extractor (DPFE), a deep model which is trained and evaluated based on information theoretic constraints. Using the selective exchange of information between a user's device and a service provider, DPFE enables the user to prevent certain sensitive information from being sha…
New algorithms improve privacy in statistical estimation by making them robust.
problem Improving privacy in statistical estimation methods.
method Black-box reduction from privacy to robustness, using Sum-of-Squares method.
result Design of polynomial-time private estimators with optimal tradeoffs among sample complexity, accuracy, and privacy.
Paper protects privacy and fairness in deep learning models.
problem Ensuring fairness in deep learning models while protecting sensitive data.
method Uses differential privacy and Lagrangian duality to design fair predictors.
result Demonstrates improved model performance on prediction tasks.
The paper investigates how data imbalance affects fairness and accuracy in differentially private deep learning.
problem Impact of data imbalance on fairness and accuracy in differentially private deep learning.
method Study the effects of different levels of imbalance in the data on the accuracy and fairness of decisions made by a model trained with differential privacy.
result Small imbalances and loose privacy guarantees can cause disparate impacts on model accuracy and fairness.
Study compares federated learning and coreset approaches for privacy in distributed machine learning.
problem Measuring privacy in distributed machine learning approaches.
method Comparison of federated learning and coreset approaches using membership inference attack.
result Federated learning offers better privacy than coreset, but with higher communication cost.
We provide a general framework for characterizing the trade-off between accuracy and robustness in supervised learning. We propose a method and define quantities to characterize the trade-off between accuracy and robustness for a given architecture, and provide theoretical insight into the trade-off. Specifically we in…
This paper improves deep learning models' accuracy with differential privacy using gradient encoding and denoising.
problem Deep learning models leak sensitive information about their training datasets.
method Gradient encoding to map gradients to a smaller vector space, and denoising for post-processing.
result Our technique achieves better model accuracy with differential privacy guarantees compared to state-of-the-art methods.
LDP-Fed protects privacy in federated learning with neural networks.
problem Privacy protection for high-dimensional, continuous model parameters in federated learning.
method Local Differential Privacy (LDP) for repeated collection of model training parameters, selection and filtering of parameter updates.
result LDP-Fed achieves model accuracy comparable to non-private methods while preserving privacy.
A new framework enables real-time task trade-off control.
problem Conflict between multiple related tasks in a fixed model capacity.
method Formulates MTL as a preference-conditioned multiobjective optimization problem; uses a hypernetwork-based neural network.
result A single model can handle different trade-off preferences among multiple tasks.
New trade-off found between accuracy and adversarial robustness in regression.
problem Finding a balance between accuracy and robustness in regression models.
method Deriving a fundamental trade-off between standard and adversarial risk in regression with polynomial ridge functions.
result A necessary condition for achieving adversarial robustness without significant accuracy loss.
Paper explores robust regression methods and their bias-variance trade-off.
problem Understanding the trade-off between robust estimation and optimization methods.
method Examines traditional outlier-resistant robust estimation and robust optimization.
result Both methods follow converse strategies due to a bias-variance trade-off.
This paper studies trade-offs in private prediction methods.
problem Leakage of training data information in machine learning predictions.
method Private training and private prediction methods with trade-offs.
result Private training methods outperform private prediction methods in various settings.
Efficiently approximates fairness-accuracy trade-offs for diverse datasets.
problem Inherent trade-off between fairness and accuracy in machine learning models.
method You-Only-Train-Once (YOTO) framework for computationally efficient approximation.
result Robust methodology for auditing model fairness with statistical guarantees.
New insights into convergence and accuracy trade-offs in federated and meta-learning.
problem Understanding the trade-offs between convergence and accuracy in federated and meta-learning.
method Generalized local update methods, proving equivalence to first-order optimization on a surrogate loss.
result Novel convergence rates and insights into the importance of algorithmic choices in communication-limited settings.
We empirically investigate the best trade-off between sparse and uniformly-weighted multiple kernel learning (MKL) using the elastic-net regularization on real and simulated datasets. We find that the best trade-off parameter depends not only on the sparsity of the true kernel-weight spectrum but also on the linear dep…
Proposes FACT, a diagnostic for understanding group fairness trade-offs.
problem Group fairness notions often conflict with each other, requiring a cost in model performance.
method Characterizes trade-offs via the fairness-confusion tensor and optimizes accuracy and fairness objectives.
result Demonstrates the use of FACT on synthetic and real datasets to understand accuracy-fairness trade-offs.
Pareto MTL finds optimal solutions for multiple tasks with different trade-offs.
problem Finding a single optimal solution for multiple conflicting tasks.
method Formulate multi-task learning as multiobjective optimization, decompose into subproblems, solve in parallel.
result Generates well-representative Pareto optimal solutions for different trade-offs.
Study shows exponential error reduction in multiclass classification without bias-variance trade-off.
problem Multiclass classification with margin conditions.
method Analysis of classification error under hard-margin conditions.
result Exponential decrease in classification error without bias-variance trade-off.
A trade-off between accuracy and fairness is almost taken as a given in the existing literature on fairness in machine learning. Yet, it is not preordained that accuracy should decrease with increased fairness. Novel to this work, we examine fair classification through the lens of mismatched hypothesis testing: trying …
Vecchia approximations provide the best accuracy-runtime trade-off for Gaussian process approximations.
problem High computational cost of Gaussian processes for large data sets.
method Systematic comparison of different Gaussian process approximations.
result Vecchia approximations consistently provide the best accuracy-runtime trade-off.
The paper explores the trade-off between bias and variance in high-dimensional models.
problem Understanding the unavoidable trade-off between bias and variance in high-dimensional statistical models.
method Proposes a general strategy to obtain lower bounds on the variance of estimators with a specified bias, and applies it to various statistical models.
result Shows the extent to which the bias-variance trade-off is unavoidable and quantifies the performance loss for methods that do not balance it.
Fine-tuning LLMs improves capability but harms safety, study finds.
problem Balancing capability and safety in LLM fine-tuning.
method Theoretical framework and numerical experiments for two safety-aware fine-tuning strategies.
result Characterization of fundamental limits of safety-capability trade-off in LLM fine-tuning.
This paper analyzes the trade-off between accuracy and communication in personalized federated learning.
problem The accuracy-communication trade-off in personalized federated learning.
method The paper provides a quantitative characterization of the personalization degree on the trade-off, establishing minimax optimality.
result The paper offers theoretical insights for choosing the personalization degree and validates the results on synthetic and real-world datasets.
SCORE resolves the robustness vs accuracy trade-off by redefining robust error.
problem The inherent trade-off between robustness and accuracy in adversarial training.
method SCORE defines local equivariance as the ideal robust behavior, leading to a new robust error metric.
result SCORE reconciles robustness and accuracy, improving model performance on RobustBench.
Two Fisher information matrix estimators are analyzed for neural networks, focusing on their variances and trade-offs.
problem Estimating the Fisher information matrix in neural networks due to its high computational cost.
method Examined two popular diagonal Fisher information matrix estimators and their variances in neural networks for regression and classification.
result The variances of the estimators depend on the non-linearity with respect to different parameter groups and should not be neglected.
Method orders Pareto solutions using transformed objective scores.
problem Lack of orderability in multi-objective optimization solutions.
method Probability integral transform to map objectives to scores.
result Pareto efficient solutions can be ordered in score space.
Differential privacy is a mathematical framework for privacy-preserving data analysis. Changing the hyperparameters of a differentially private algorithm allows one to trade off privacy and utility in a principled way. Quantifying this trade-off in advance is essential to decision-makers tasked with deciding how much p…
SharpBalance improves deep ensemble performance by balancing sharpness and diversity.
problem Improving deep ensemble performance in both in-distribution and out-of-distribution scenarios.
method Introducing SharpBalance, a novel training approach that balances sharpness and diversity within ensembles.
result SharpBalance effectively improves the sharpness-diversity trade-off and ensemble performance in ID and OOD scenarios.
Research explores the trade-off between multi-task learning and multitasking in deep neural networks.
problem Trade-off between multi-task learning and multitasking in deep neural networks.
method Meta-learning algorithm to manage the trade-off between shared and separated representations.
result Agent successfully optimizes training strategy based on environment.
VFDS selects dynamic features for efficient HAR tasks, optimizing performance-cost trade-offs.
problem Optimizing feature selection for varying costs and dynamic contexts in machine learning tasks.
method Bayesian learning framework with variational dynamic selection policy.
result VFDS selects different features under changing contexts, saving sensory costs while maintaining HAR accuracy.
New method optimizes privacy and compute trade-offs for deep learning.
problem Privacy and compute trade-offs in deep learning training.
method Decoupling privacy analysis and experimental behavior, using TAN and scaling laws for DP-SGD.
result Stronger privacy guarantees with significant reduction in computational budget.
We propose a new PAC-Bayesian bound and a way of constructing a hypothesis space, so that the bound is convex in the posterior distribution and also convex in a trade-off parameter between empirical performance of the posterior distribution and its complexity. The complexity is measured by the Kullback-Leibler divergen…
Paper develops a dynamic Bayesian approach for active learning that optimizes exploration-exploitation balance.
problem Balancing exploration and exploitation in active learning for unknown functions.
method Develops BHEEM, a Bayesian hierarchical approach with approximate Bayesian computation for sampling trade-off parameters.
result BHEEM achieves at least 21% and 11% improvement over pure exploration and exploitation strategies respectively.
Study tests whether trade-off functions are above or below benchmarks using finite samples.
problem Testing trade-off functions between unknown distributions.
method Identifies a condition for nontrivial testing, constructs a test with error guarantees, and inverts the test for confidence bands.
result Finite-sample testing is possible under specific structural assumptions about rejection regions.
Study on trade-offs between accuracy and interpretability in machine learning.
problem Lack of formal study on statistical cost of interpretability.
method Modeling interpretability as a constraint in empirical risk minimization for binary classification.
result Explains conditions under which accuracy trade-off occurs with interpretability constraints.
New insights into neural network forgetting reveal a trade-off between node activation and re-use.
problem Challenges in maintaining performance on old tasks while learning new ones.
method Theoretical analysis of synthetic and real data setups, focusing on node activation vs re-use.
result Worst forgetting occurs in an intermediate similarity regime between learned tasks.
Bayesian optimization offers the possibility of optimizing black-box operations not accessible through traditional techniques. The success of Bayesian optimization methods such as Expected Improvement (EI) are significantly affected by the degree of trade-off between exploration and exploitation. Too much exploration c…
Learning algorithms need bias to generalize and perform better than random guessing. We examine the flexibility (expressivity) of biased algorithms. An expressive algorithm can adapt to changing training data, altering its outcome based on changes in its input. We measure expressivity by using an information-theoretic …
Meta-learning balances task-specific modeling and optimization complexity.
problem Balancing accurate task-specific modeling with ease of optimization in meta-learning.
method Theoretical and empirical analysis of trade-off between modeling and optimization in meta-learning.
result Explicit bounds on modeling and optimization errors for non-convex and linear regression problems.