arXiv research

Study shows a tradeoff between sample complexity and computational efficiency for learning halfspaces with random noise.

Paper solves NGCA for discrete distributions using LLL method.

Privacy improves robustness in statistical estimation.

Sum-of-Squares lower bound shows NGCA requires more samples than known algorithms.

New algorithm learns halfspaces with near-optimal sample complexity in noisy conditions.

Study shows efficient algorithms for noiseless linear regression require quadratic sample complexity in contamination rate.

A statistical framework for removing unwanted data domains in machine learning.

New insights into statistical and computational limits for mixed sparse linear regression.

Study on estimating Gaussian mean with missing data in high dimensions.

New research shows some distributions hard to sample via diffusions.

The main goal of this thesis is to point out that the bias-variance tradeoff is not always true (e.g. in neural networks). We advocate for this lack of universality to be acknowledged in textbooks and taught in introductory courses that cover the tradeoff. We first review the history of the bias-variance tradeoff, its …

A new tradeoff between regularization and sharpness improves model performance in overparameterized settings.

Study the tradeoff between signal distortion and human perception over finite channels.

We perform the first study of the tradeoff space of access methods and replication to support statistical analytics using first-order methods executed in the main memory of a Non-Uniform Memory Access (NUMA) machine. Statistical analytics systems differ from conventional SQL-analytics in the amount and types of memory …

Haircutting non-cash collateral has become a key element of the post-crisis reform of the shadow banking system and OTC derivatives markets. This article develops a parametric haircut model by expanding haircut definitions beyond the traditional value-at-risk measure and employing a double-exponential jump-diffusion mo…

Paper explores tradeoff between standard and robust accuracy for latent models.

Proposes a new adversarial model to avoid accuracy vs. adversarial accuracy tradeoff.

We propose a data-driven framework for optimizing privacy-preserving data release mechanisms to attain the information-theoretically optimal tradeoff between minimizing distortion of useful data and concealing specific sensitive information. Our approach employs adversarially-trained neural networks to implement random…

Researchers study fairness-accuracy tradeoffs in predictive models for multiple groups.

Despite breakthrough performance, modern learning models are known to be highly vulnerable to small adversarial perturbations in their inputs. While a wide variety of recent \emph{adversarial training} methods have been effective at improving robustness to perturbed inputs (robust accuracy), often this benefit is accom…

The paper explores robustness in linear regression models under adversarial attacks.

The paper analyzes the bias-variance tradeoff for Bregman divergences.

CDC-FM improves generative model quality-generalization tradeoff by regularizing with geometry-aware noise.

This paper explores tradeoffs between standard and adversarial risks in distributionally adversarial training.

This paper considers portfolio construction in a dynamic setting. We specify a loss function comprised of utility and complexity components with an unknown tradeoff parameter. We develop a novel regret-based criterion for selecting the tradeoff parameter to construct optimal sparse portfolios over time.

In this paper, we exhibit the tradeoffs between the (training) sample, computation and storage complexity for the problem of supervised classification using signal subspace estimation. Our main tool is the use of tensor subspaces, i.e. subspaces with a Kronecker structure, for embedding the data into lower dimensions. …

New OLO algorithms use Stein's method for better performance tradeoffs.

The paper explores the tradeoffs between fairness measures in machine learning.

While adversarial training can improve robust accuracy (against an adversary), it sometimes hurts standard accuracy (when there is no adversary). Previous work has studied this tradeoff between standard and robust accuracy, but only in the setting where no predictor performs well on both objectives in the infinite data…

Adversarial training, in which a network is trained on adversarial examples, is one of the few defenses against adversarial attacks that withstands strong attacks. Unfortunately, the high cost of generating strong adversarial examples makes standard adversarial training impractical on large-scale problems like ImageNet…

The paper analyzes the tradeoffs between accuracy and invariance in learning representations.

Paper improves privacy-accuracy balance in federated learning.

A new algorithm improves efficiency in selecting examples for deep learning.

Improves early stopping in deep networks by adjusting stepsizes.

Statistical query algorithms and low-degree tests are nearly equivalent in high-dimensional hypothesis testing.

Algorithm balances online and offline data for linear bandits.

This work explores feature learning tradeoffs in neural networks.

Faced with massive data, is it possible to trade off (statistical) risk, and (computational) space and time? This challenge lies at the heart of large-scale machine learning. Using k-means clustering as a prototypical unsupervised learning problem, we show how we can strategically summarize the data (control space) in …

Deep reinforcement learning (DRL) on Markov decision processes (MDPs) with continuous action spaces is often approached by directly training parametric policies along the direction of estimated policy gradients (PGs). Previous research revealed that the performance of these PG algorithms depends heavily on the bias-var…

New insights into bias-variance tradeoff for data-driven optimization under local misspecification.

New method quantifies redundant information using information bottleneck.

The paper analyzes Adam and SGD in nonstationary optimization, revealing tradeoffs between noise and drift.

Study near-optimal bounds for learning Gaussian halfspaces with random noise.

New estimator reduces bias and variance in tensor and matrix denoising.

The bias-variance tradeoff tells us that as model complexity increases, bias falls and variances increases, leading to a U-shaped test error curve. However, recent empirical results with over-parameterized neural networks are marked by a striking absence of the classic U-shaped test error curve: test error keeps decrea…

We introduce "Search with Amortized Value Estimates" (SAVE), an approach for combining model-free Q-learning with model-based Monte-Carlo Tree Search (MCTS). In SAVE, a learned prior over state-action values is used to guide MCTS, which estimates an improved set of state-action values. The new Q-estimates are then used…

This paper presents a bias-variance tradeoff of graph Laplacian regularizer, which is widely used in graph signal processing and semi-supervised learning tasks. The scaling law of the optimal regularization parameter is specified in terms of the spectral graph properties and a novel signal-to-noise ratio parameter, whi…

Paper characterizes fairness vs. accuracy tradeoff in classification.