Characterizes sample complexity for outcome indistinguishability in machine learning.
problem Outcome indistinguishability in machine learning, focusing on distinguishers and predictors.
method Sample complexity characterized by metric entropy of predictor and distinguisher classes, using dual Minkowski norms.
result Equivalence and tightness of sample complexity characterizations in distribution-specific and distribution-free settings.
We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L_2 regularization: We introduce the γ-adapted-dimension, which is a simple function of the spectrum of a distribution's covariance matrix, and show distribution-specific upper and lower bounds on the s…
Hardness proof for agnostically learning halfspaces from worst-case lattice problems.
problem Agnostically learning halfspaces in the presence of noise.
method Reduction to worst-case lattice problems (GapSVP, SIVP).
result No efficient algorithm can achieve misclassification error better than 1/2 - γ under given hardness assumptions.
We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L2 regularization: We introduce the margin-adapted dimension, which is a simple function of the second order statistics of the data distribution, and show distribution-specific upper and lower bounds on…
Paper proves hardness of learning various complex models under local pseudorandom generators.
problem Hardness of learning various complex models.
method Existence of local pseudorandom generators.
result Proves hardness of learning shallow ReLU neural networks and other models.
Efficient algorithm for halfspaces with specific noise conditions.
problem Learning halfspaces with Massart and Tsybakov noise.
method PAC active learning algorithm for d-dimensional halfspaces.
result Near-optimal label complexity under Massart noise and lower than passive learning under Tsybakov noise.
Non-convex SGD learns halfspaces with adversarial label noise efficiently.
problem Agnostically learning halfspaces in adversarial label noise settings.
method Non-convex SGD optimization for halfspace learning.
result Non-convex SGD achieves misclassification error close to optimal with adversarial noise.
We formulate a principle for classification with the knowledge of the marginal distribution over the data points (unlabeled data). The principle is cast in terms of Tikhonov style regularization where the regularization penalty articulates the way in which the marginal density should constrain otherwise unrestricted co…
We present Vector-Space Markov Random Fields (VS-MRFs), a novel class of undirected graphical models where each variable can belong to an arbitrary vector space. VS-MRFs generalize a recent line of work on scalar-valued, uni-parameter exponential family and mixed graphical models, thereby greatly broadening the class o…
New method estimates mean from noisy data with few outliers.
problem Estimate mean from noisy data with few outliers.
method Iterative multi-filtering approach.
result Achieves near-optimal error with practical efficiency.
Develops a method to estimate rare-event probabilities under distributional uncertainty.
problem Distributional uncertainty limits the effectiveness of rare-event simulation techniques.
method Wasserstein distributionally robust rare-event simulation (DRIS) framework.
result DRIS achieves vanishing relative error in estimating rare-event probabilities.
Boosting with unlabeled data achieves optimal sample complexity in agnostic settings.
problem Boosting's sample inefficiency in agnostic learning.
method Designing an agnostic boosting algorithm with unlabeled data to match ERM's sample complexity.
result The total sample complexity is optimal, with a vanishing fraction needing to be labeled.
Measuring divergence between two distributions is essential in machine learning and statistics and has various applications including binary classification, change point detection, and two-sample test. Furthermore, in the era of big data, designing divergence measure that is interpretable and can handle high-dimensiona…
FDR-SVM improves classification robustness in federated learning with uncertain data.
problem Federated learning with uncertain and private client data.
method Develops FDR-SVM, a robust SVM approach using a mixture of Wasserstein balls ambiguity set.
result Establishes theoretical guarantees and derives algorithms with performance bounds.
An intervention may have an effect on units other than those to which it was administered. This phenomenon is called interference and it usually goes unmodeled. In this paper, we propose to combine Lauritzen-Wermuth-Frydenberg and Andersson-Madigan-Perlman chain graphs to create a new class of causal models that can re…
Solves learning halfspaces with Massart noise for log-concave distributions.
problem Learning halfspaces with Massart noise in distribution-specific PAC model.
method Identifies a smooth non-convex surrogate loss and uses SGD to solve the learning problem.
result First computationally efficient algorithm for learning halfspaces with Massart noise for a broad family of distributions.
Generative model improves safety in self-driving simulators and human motion generation.
problem Improving generative models for constrained domains like safety-critical applications.
method Developed Gen-neG, a denoising diffusion model that uses oracle-assisted guidance.
result Empirically validated Gen-neG for collision avoidance and safety-guarded human motion generation.
In this work we study loss functions for learning and evaluating probability distributions over large discrete domains. Unlike classification or regression where a wide variety of loss functions are used, in the distribution learning and density estimation literature, very few losses outside the dominant log loss ar…
In this paper, we prove that some Gaussian structural equation models with dependent errors having equal variances are identifiable from their corresponding Gaussian distributions. Specifically, we prove identifiability for the Gaussian structural equation models that can be represented as Andersson-Madigan-Perlman cha…
A parametrization of hypergraphs based on the geometry of points in Rd is developed. Informative prior distributions on hypergraphs are induced through this parametrization by priors on point configurations via spatial processes. This prior specification is used to infer conditional independence models or M…
Robustifies elicitable functionals to handle small distribution misspecifications.
problem Determining uniquely optimal forecasts under distributional misspecification.
method Integrates statistical robustness into elicitable functionals using Kullback-Leibler divergence.
result Robust elicitable functionals admit unique solutions at the boundary of uncertainty regions.
Models predict probabilities of causation from limited data.
problem Estimating probabilities of causation requires unreliable or impractical experimental and observational data.
method Proposed Exact-MLP and Mask-MLP models trained on reliable subpopulations.
result Models achieve average MAEs of roughly 0.03, reducing MAE by 80%.
Study efficient iterative method for distribution matching using sliced optimal transport.
problem Efficiently match distributions using sliced optimal transport.
method Slice-matching scheme based on sliced optimal transport, with quantitative non-asymptotic rates derived.
result Derive quantitative non-asymptotic rates for convergence to target distribution.
HyperLISTA simplifies LISTA training with adaptive hyperparameters.
problem Sparse recovery with LISTA networks.
method Adaptive hyperparameter tuning based on previous layers.
result HyperLISTA achieves similar performance on seen data and better on unseen data.
Study selective classification with halfspaces, achieving error bounds under Gaussian distributions.
problem Modeling relationships in subsets of data defined by selection rules.
method Sparse linear classifiers for subsets defined by halfspaces, focusing on Gaussian feature distributions.
result First PAC-learning algorithm for homogeneous halfspace selectors with error guarantee $\bigO*{\sqrt{\mathrm{opt}}}$.
New method samples from multi-modal distributions on Riemannian manifolds without training.
problem Sampling from multi-modal distributions on Riemannian manifolds is challenging.
method Simulation of a non-equilibrium deterministic dynamics to transport noise toward target distributions.
result Method is entirely training-free and effective on various multi-modal problems.
MT-SGD samples from multiple target distributions using gradient descent.
problem Sampling from multiple unnormalized target distributions.
method Proposes MT-SGD, a flow of intermediate distributions to sample from multiple target distributions.
result Asymptotic analysis shows MT-SGD reduces to multiple-gradient descent for multi-objective optimization.
This paper broadens contrastive learning for disentangled representations without strict data distribution assumptions.
problem Learning disentangled representations from data with specific assumptions.
method Extends theoretical guarantees for disentanglement to a broader family of contrastive methods, relaxing data distribution assumptions.
result Identifiability of true latents for four contrastive losses proved without common independence assumptions.
Efficient bandit exploration for various distributions without distribution-specific tuning.
problem Optimizing exploration in multi-armed bandit models for different distributions.
method Sub-sampling Duelling Algorithms (SDA) with Random Block sampling for efficient exploration.
result Achieves asymptotically optimal regret for Bernoulli, Gaussian, and Poisson distributions.
Reverse sensitivity analysis for risk models under various stresses.
problem Understanding model changes under output stress.
method Deriving the closest stressed distribution and model parameters.
result Numerically efficient method for calculating stressed model.
Study minimax rates for density estimation under Huber contamination and Besov IPM losses.
problem Minimax convergence rates of nonparametric density estimation under Huber contamination model with outliers.
method Re-scaled thresholding wavelet series estimator and GAN architectures.
result Achieves minimax optimal convergence rates under Besov IPM losses.
Study proposes active learning method for estimating robust regions in uncertain function evaluations.
problem Estimating robust regions for uncertain function evaluations with unknown distributions.
method Distributionally robust level-set estimation (DRPTR) with active learning.
result The proposed method efficiently identifies reliable regions with theoretical guarantees.
Optimal learning for parametric prophet inequalities with exponential-type distributions
problem Learning in prophet inequalities with unknown parameters
method Confidence-based dynamic-programming policy
result Achieves optimal asymptotic competitive ratio using online observations
Efficient algorithm for learning halfspaces in a new model with polynomial time complexity.
problem Learning halfspaces in the testable learning model with distributional constraints.
method Developed new tests using labels and combined with moment-matching approach.
result Achieved near optimal error rates for Gaussian and strongly log-concave distributions.
The paper identifies generators of linear SDEs with noise types.
problem Identifying the generator of linear SDEs from their solution distribution.
method Deriving sufficient and necessary conditions for additive noise, and sufficient conditions for multiplicative noise.
result Generic conditions for identifying the generator of linear SDEs with both types of noise.
A basic, and still largely unanswered, question in the context of Generative Adversarial Networks (GANs) is whether they are truly able to capture all the fundamental characteristics of the distributions they are trained on. In particular, evaluating the diversity of GAN distributions is challenging and existing method…
NUTS mixing time scales as d^(1/4) for Gaussian distributions.
problem Improving the efficiency of the No-U-Turn Sampler (NUTS) for Gaussian distributions.
method Coupling argument leveraging geometric structure of Gaussian concentration, uniformity analysis of NUTS transitions.
result The mixing time of NUTS scales as d^(1/4) for Gaussian distributions, up to logarithmic factors.
New policy optimizes risk and optimality in stochastic bandits.
problem Optimizing risk in stochastic bandits with heavy-tailed risk.
method Designing policies with worst-case optimality for expected regret and light-tailed risk distribution.
result Achieves worst-case optimality for expected regret and light-tailed risk distribution.
Paper simplifies data carving inference with a parametric distribution.
problem Valid inference after selection with data carving.
method Developed a parametric distribution for data carving inference.
result Exact inference for data carving can be computed trivially.
A new framework tightens risk measure confidence bounds.
problem Improving confidence bounds for various risk measures.
method Distribution optimization framework with two estimation schemes based on concentration bounds.
result Consistently tighter confidence bounds compared to previous methods.
Adversarial attacks on deep learning models have compromised their performance considerably. As remedies, a lot of defense methods were proposed, which however, have been circumvented by newer attacking strategies. In the midst of this ensuing arms race, the problem of robustness against adversarial attacks still remai…
WEGL embeds graphs in a vector space for faster machine learning.
problem Efficiently embedding graphs for machine learning tasks.
method Wasserstein distance for node embedding similarity, Monge maps for graph representation.
result State-of-the-art classification performance with superior computational efficiency.
DALMC provides non-asymptotic error bounds for generative models.
problem Efficiently generating samples from complex data distributions.
method Analysis of diffusion paths and Langevin Monte Carlo.
result Theoretical guarantees for a class of generative models.
Proposes robust model through Wasserstein geodesic interpolation of training data.
problem Improving model robustness through data augmentation.
method Augment data by finding worst-case Wasserstein barycenter on geodesic path.
result Improves robustness on CIFAR-10 up to 7.7% and on CIFAR-100 up to 16.8%.
Although neural networks are routinely and successfully trained in practice using simple gradient-based methods, most existing theoretical results are negative, showing that learning such networks is difficult, in a worst-case sense over all data distributions. In this paper, we take a more nuanced view, and consider w…
The ACCRU framework improves probabilistic forecasts by capturing input-dependent uncertainty.
problem Uncertainty in deterministic predictions, especially for skewed and non-Gaussian errors.
method Neural network trained with a loss function balancing accuracy and reliability to learn input-dependent, non-Gaussian uncertainty distributions.
result Improves probabilistic forecasts relative to existing methods, capturing skewed and non-Gaussian errors.
The paper introduces a DRM for causal inference, offering a flexible method to analyze counterfactual distributions.
problem Estimating mean causal effects is limited; a distributional perspective is needed for a more thorough understanding.
method The paper employs a semiparametric density ratio model (DRM) with an empirical likelihood (EL) approach to estimate counterfactual distribution functions.
result The DRM framework enables direct and transparent causal inference from a distributional perspective, validated by numerical studies.
The study examines property testing and estimation under non-identically distributed samples, finding necessary and sufficient sample complexities.
problem Property testing and estimation under non-identically distributed samples.
method Analysis of distributional property testing and estimation in settings with heterogeneous entities.
result Necessary and sufficient sample complexities for property testing and estimation under non-identically distributed samples.