Efficiently tunes hyperparameters with dynamic accuracy method.
problem Optimizing machine learning hyperparameters with inexact evaluations.
method Dynamic accuracy derivative-free optimization for hyperparameter tuning.
result Demonstrates robust and efficient hyperparameter tuning compared to fixed accuracy methods.
This paper improves deep learning models' accuracy with differential privacy using gradient encoding and denoising.
problem Deep learning models leak sensitive information about their training datasets.
method Gradient encoding to map gradients to a smaller vector space, and denoising for post-processing.
result Our technique achieves better model accuracy with differential privacy guarantees compared to state-of-the-art methods.
Framework for fair classification with noisy protected attributes and provable guarantees.
problem Fair classification with noisy protected attributes.
method Optimization framework for linear and linear-fractional fairness constraints, handling multiple non-binary attributes.
result Provably fair classifier with minimal accuracy loss, even with large noise.
Paper presents robust boosting methods for label noise.
problem Boosting methods degrade in noisy environments.
method Robust Minimax Boosting (RMBoost) with theoretical guarantees.
result RMBoost provides strong classification accuracy and robustness.
New approach uses Gaussian processes to learn and track complex systems with guaranteed accuracy.
problem Inaccurate first principle models for complex systems due to data complexity.
method Bayesian prediction error bound for Gaussian process regression, derived from kernel-based data density.
result Achieves vanishing tracking error with increasing data density, providing time-varying accuracy guarantees.
For massive and heterogeneous modern datasets, it is of fundamental interest to provide guarantees on the accuracy of estimation when computational resources are limited. In the application of learning to rank, we provide a hierarchy of rank-breaking mechanisms ordered by the complexity in thus generated sketch of the …
Algorithm speeds up search for stationary targets with guaranteed accuracy.
problem Minimize search time while ensuring high detection accuracy of stationary targets.
method Multi-fidelity Gaussian process model and EMTS algorithm.
result Guaranteed performance in target detection accuracy and search time.
New framework for fair classification in adversarial settings with provable guarantees.
problem Fairness in classification with adversarial perturbations of protected attributes.
method Optimization framework for learning fair classifiers with provable guarantees.
result Near-tightness of accuracy and fairness guarantees for multiple protected attributes and various hypothesis classes.
Crowdsourced data used in machine learning services might carry sensitive information about attributes that users do not want to share. Various methods have been proposed to minimize the potential information leakage of sensitive attributes while maximizing the task accuracy. However, little is known about the theory b…
New study shows Gaussian samplers struggle with heavy-tailed targets, while stable samplers excel.
problem The difficulty of sampling from heavy-tailed distributions using Gaussian versus stable oracles.
method Comparison of Gaussian and stable oracles for proximal samplers.
result Gaussian samplers have a fundamental barrier for high-accuracy guarantees in heavy-tailed sampling, while stable samplers excel.
Traditional differential privacy is independent of the data distribution. However, this is not well-matched with the modern machine learning context, where models are trained on specific data. As a result, achieving meaningful privacy guarantees in ML often excessively reduces accuracy. We propose Bayesian differential…
Efficiently approximates fairness-accuracy trade-offs for diverse datasets.
problem Inherent trade-off between fairness and accuracy in machine learning models.
method You-Only-Train-Once (YOTO) framework for computationally efficient approximation.
result Robust methodology for auditing model fairness with statistical guarantees.
FaiREE provides fair classification with guarantees for small datasets.
problem Fairness in classification often requires large sample sizes and distributional assumptions.
method FaiREE offers finite-sample and distribution-free fairness guarantees.
result FaiREE achieves optimal accuracy and satisfies various fairness notions.
In this paper we study the performance of the Projected Gradient Descent(PGD) algorithm for ℓp-constrained least squares problems that arise in the framework of Compressed Sensing. Relying on the Restricted Isometry Property, we provide convergence guarantees for this algorithm for the entire range of $0\leq p\…
DP-SGD provides privacy guarantees for all data points, but we propose output-specific DP to better account for individual examples.
problem Accounting for individual privacy guarantees in DP-SGD.
method Output-specific (ε,δ)-DP and an efficient algorithm to investigate individual privacy across datasets. result Most examples enjoy stronger privacy guarantees than the worst-case bound, and there is a correlation between training loss and privacy parameter.
New bounds for LDP with heterogeneous privacy levels guaranteeing high probability of accuracy.
problem Statistical estimation under LDP with users having varying privacy levels.
method Developed finite sample upper bounds in ℓ_2-norm with high probability, complemented by lower bounds.
result Optimal guarantees for heterogeneous LDP in terms of probability and constants.
Data-independent pruning method reduces neural network size with accuracy guarantees.
problem Limited computational and memory resources for neural networks.
method Structured pruning using coresets.
result First efficient algorithm with worst-case guarantees on compression and accuracy.
Learning ReLU networks to high uniform accuracy requires exponentially many samples.
problem Achieving high uniform accuracy on ReLU networks for security-critical applications.
method Quantified the number of training samples needed for any algorithm to guarantee uniform accuracy.
result The minimal number of training samples scales exponentially with network depth and input dimension.
This paper improves indoor positioning accuracy by deploying reference nodes to ensure Line-of-Sight.
problem Systematic bias errors in indoor positioning due to non-LoS propagation.
method Model indoor service area as a graph, partition into cliques for reference nodes, set minimum distance and angle parameters.
result Guaranteed LoS to reference nodes improves indoor positioning accuracy and precision.
FADE framework improves fairness and accuracy in ensemble learning.
problem Improving fairness in existing models without sacrificing accuracy.
method Flexible fair ensemble learning framework targeting multiple fairness criteria.
result Multiple unfairness measures can be minimized simultaneously with little impact on accuracy.
Enhances deep learning models to resist adversarial attacks.
problem Protecting deep learning models from adversarial examples.
method Combines two mechanisms: increased robustness at the cost of accuracy and improved accuracy without robustness guarantee.
result Combining mechanisms provides robustness against adversarial examples while maintaining accuracy.
Fair classification has been a topic of intense study in machine learning, and several algorithms have been proposed towards this important task. However, in a recent study, Friedler et al. observed that fair classification algorithms may not be stable with respect to variations in the training dataset -- a crucial con…
Proves accuracy guarantees for self-supervised learning with correlated positive pairs.
problem Lack of theoretical guarantees for self-supervised learning with correlated positive pairs.
method Novel augmentation graph concept and spectral decomposition loss.
result Provably accurate features under linear probe evaluation.
This paper speeds up simulations of hypersonic reentry by combining traditional and neural methods.
problem Accurately simulating hypersonic reentry with chemical reactions is computationally expensive.
method A hybrid simulation code combining a traditional fluid dynamics solver with a neural network approximating chemical reactions.
result Achieved significant acceleration factors (imes10 to imes18.6) while maintaining accuracy. This paper analyzes the conflict between Hamming loss and subset accuracy in multi-label classification.
problem The conflict between Hamming loss and subset accuracy in multi-label classification.
method The paper analyzes the learning guarantees of algorithms optimizing Hamming loss and subset accuracy, providing theoretical bounds and experimental support.
result Optimizing Hamming loss with its surrogate loss can lead to good performance on subset accuracy in small label spaces, contrary to theoretical expectations.
New online GP algorithm offers performance guarantees for streaming data.
problem Training and inference of GPs require all historic data, limiting online decision-making.
method Developed a new theoretical framework based on PAC-Bayes theory, optimizing empirical risk and parameter divergence.
result Offers both a guarantee of generalized performance and good accuracy.
Communication and privacy are two critical concerns in distributed learning. Many existing works treat these concerns separately. In this work, we argue that a natural connection exists between methods for communication reduction and privacy preservation in the context of distributed machine learning. In particular, we…
We introduce a new spatial data structure for high dimensional data called the \emph{approximate principal direction tree} (APD tree) that adapts to the intrinsic dimension of the data. Our algorithm ensures vector-quantization accuracy similar to that of computationally-expensive PCA trees with similar time-complexity…
SingleProp speeds up robust neural network training with minimal certification.
problem Efficiently defending neural networks against adversarial attacks with certified guarantees.
method SingleProp regularizer that requires only one forward pass per training iteration.
result Comparable certified accuracy to state-of-the-art defenses, but significantly faster training.
The paper investigates how data imbalance affects fairness and accuracy in differentially private deep learning.
problem Impact of data imbalance on fairness and accuracy in differentially private deep learning.
method Study the effects of different levels of imbalance in the data on the accuracy and fairness of decisions made by a model trained with differential privacy.
result Small imbalances and loose privacy guarantees can cause disparate impacts on model accuracy and fairness.
Simultaneous clustering and optimization (SCO) has recently drawn much attention due to its wide range of practical applications. Many methods have been previously proposed to solve this problem and obtain the optimal model. However, when a dataset evolves over time, those existing methods have to update the model freq…
Paper develops privacy-preserving federated learning for nonsmooth objectives.
problem Solving nonsmooth objective functions in a privacy-preserving manner.
method Zero-concentrated differential privacy (zCDP) with Gaussian noise, distributed ADMM, and approximation of augmented Lagrangian.
result The algorithm achieves a competitive privacy-accuracy trade-off and converges to the exact solution.
Paper introduces scalable clustering for large datasets with outliers.
problem Lack of scalable algorithms for large datasets with outliers.
method Provable robust clustering algorithm based on loss minimization for Gaussian mixture models.
result Algorithm provides high accuracy with theoretical guarantees and outperforms existing methods.
Paper explores how to design federated learning protocols that benefit all participants while maintaining privacy.
problem Privacy concerns undermine the accuracy benefits of federated learning in privacy-sensitive domains.
method The paper provides conditions for mutually beneficial federated learning protocols and designs protocols that maximize total utility and accuracy.
result The paper demonstrates that federated learning can be designed to be mutually beneficial, striking a balance between privacy and model accuracy.
Quantum machine learning can't achieve polylogarithmic runtimes, even with quantum data access.
problem Bounding the minimum number of samples required for supervised quantum learning.
method Statistical learning theory and quantum machine learning algorithms.
result Quantum machine learning algorithms for supervised learning have at most polynomial speedups over classical algorithms.
Hidden cost: Smoothing shrinks decision boundaries, affecting class-wise accuracy.
problem The fragility of machine learning models and the need for robustness verification.
method Randomized smoothing approach to achieve statistical robustness.
result Smoothed classifiers' decision boundaries shrink, leading to class-wise accuracy disparity.
DP-BNNs improve accuracy, privacy, and reliability in neural networks.
problem Balancing privacy and accuracy in neural networks.
method Proposed three DP-BNNs: DP-SGLD, DP-BBP, and DP-MC Dropout.
result DP-SGLD achieves high accuracy under strong privacy guarantees.
The paper introduces a privacy-preserving method for estimating treatment effects that maintains accuracy.
problem Estimating heterogeneous treatment effects in sensitive data while protecting privacy.
method A general meta-algorithm for CATE estimation with differential privacy guarantees, using sample splitting and parallel composition.
result The meta-algorithm maintains accuracy even with differential privacy, showing that most accuracy loss is due to variance increase.
New bounds show multicalibration error is close to prediction error.
problem Addressing fairness in machine learning systems.
method Sample complexity bounds for uniform convergence of multicalibration error.
result Uniform convergence guarantees for multicalibration error, independent of prediction error.
The paper solves robust learning of Gaussian mixtures with nearly optimal guarantees.
problem Learning a high-dimensional Gaussian mixture model with corrupted samples.
method Introduces a new framework called strong observability to circumvent the challenge of learning individual components.
result Achieves optimal robustness guarantees of ε in total variation distance for any constant number of components. New models ensure monotonicity in preference learning, improving accuracy especially with limited data.
problem Failure of widely used preference learning models to maintain monotonicity.
method Proposed Linear Generalized Bradley-Terry models with Diffusion Priors.
result New models improve accuracy, especially with limited data.
ConquerNet smooths quantile regression for deep learning with minimax guarantees.
problem Optimization challenges in quantile regression for deep models.
method ConquerNet uses convolution-smoothed quantile ReLU neural networks.
result ConquerNet provides minimax guarantees and outperforms standard quantile neural networks.
This paper improves convergence guarantees for SGD algorithms in non-convex smooth functions.
problem Theoretical convergence properties of SGD algorithms for non-convex smooth functions.
method Analysis of SGD algorithms with arbitrary data ordering for non-convex smooth functions.
result Enhanced convergence guarantees for incremental gradient and single shuffle SGD, improving the optimization term of convergence guarantee.
Study improves sampling from non-log-concave distributions using Fisher information.
problem Sampling from non-log-concave distributions with high Fisher information guarantees.
method Proximal sampler with RGO implementation, leveraging log-concave sampling results.
result Improved complexity guarantee in relative Fisher information for non-log-concave sampling.
A conformal procedure improves CoT reasoning by aggregating reasoning paths and calibrating abstention rules.
problem Aggregation uncertainty in chain-of-thought reasoning makes correct answers less reliable.
method Introduces a conformal procedure for CoT reasoning that uses weighted score aggregation and abstention rules.
result Achieves higher selective accuracy with abstention, reducing confident-error rate.
New algorithms achieve uniform-PAC guarantees for RL with bounded eluder dimension.
problem Achieving strong performance guarantees in reinforcement learning.
method Proposes algorithms for nonlinear bandits and model-based episodic RL with a bounded eluder dimension.
result Achieves uniform-PAC sample complexity that matches state-of-the-art regret bounds or sample complexity guarantees.
Algorithm estimates human decision-making in high-dimensional states with finite-time guarantees.
problem Estimating optimal policies and measures of fit in dynamic decision models with high-dimensional state spaces.
method Single-loop estimation algorithm with stochastic gradient steps for reward maximization.
result Algorithm converges to a stationary solution with finite-time guarantees and approximates maximum likelihood sublinearly.
A machine learning configuration refers to a combination of preprocessor, learner, and hyperparameters. Given a set of configurations and a large dataset randomly split into training and testing set, we study how to efficiently select the best configuration with approximately the highest testing accuracy when trained f…