New method for private density estimation of high-dimensional Gaussian mixtures.
problem Private density estimation for mixtures of unrestricted high-dimensional Gaussians.
method Exploits list global stability to prove upper bound on sample complexity.
result First upper bound on sample complexity for agnostic private density estimation.
Gradient EM converges exponentially to optimal solution in agnostic mixtures.
problem Fitting k parametric functions to given data points without a generative model. method Gradient EM algorithm for agnostic mixtures of arbitrary parametric functions.
result Gradient EM converges exponentially to population loss minimizers with high probability.
We consider the problem of estimating the mean and covariance of a distribution from iid samples in Rn, in the presence of an η fraction of malicious noise; this is in contrast to much recent work where the noise itself is assumed to be from a distribution of known type. The agnostic problem includes many…
We study high-dimensional distribution learning in an agnostic setting where an adversary is allowed to arbitrarily corrupt an ε-fraction of the samples. Such questions have a rich history spanning statistics, machine learning and theoretical computer science. Even in the most basic settings, the only known…
Transformer models show robustness across domains with domain adversarial training.
problem Domain adaptation from multiple sources with no labeled data.
method Domain adversarial training and mixture of experts.
result Domain adversarial training improves representation but not performance.
This paper tackles online reinforcement learning for unseen tasks with unknown boundaries.
problem Real-world tasks violate assumptions of task distributions, independence, and clear task delineations.
method A mixture of Gaussian Processes models different dynamics, and a transition prior handles temporal dependencies.
result The approach reliably handles task distribution shifts and outperforms alternatives in non-stationary tasks.
New method for summarizing Bayesian mixture models using sliced Wasserstein distances.
problem Estimating the mixing measure in nonparametric Bayesian mixture models.
method Decision-theoretic approach using sliced Wasserstein distances for Gaussian mixtures.
result Effective estimation of the mixing measure and mixture density.
AMF-VI uses adaptive mixtures of flows for robust VI across diverse distributions.
problem Inconsistent behavior of single-flow models across different distributions.
method Sequential expert training of individual flows and adaptive global weight estimation via likelihood-driven updates.
result AMF-VI achieves lower negative log-likelihood and stable gains in transport metrics across various posterior families.
MAC combines models without locking them, improving ensemble performance.
problem Improving ensemble learning performance with flexibility.
method Model agnostic combination technique that dynamically combines models.
result MAC outperforms classical methods and competitive to boosting.
Hardness proof for agnostically learning halfspaces from worst-case lattice problems.
problem Agnostically learning halfspaces in the presence of noise.
method Reduction to worst-case lattice problems (GapSVP, SIVP).
result No efficient algorithm can achieve misclassification error better than 1/2 - γ under given hardness assumptions.
In this paper, we focus on weakly supervised learning with noisy training data for both classification and regression problems.We assume that the training outputs are collected from a mixture of a target and correlated noise distributions.Our proposed method simultaneously estimates the target distribution and the qual…
A key learning scenario in large-scale applications is that of federated learning, where a centralized model is trained based on data originating from a large number of clients. We argue that, with the existing training and inference, federated models can be biased towards different clients. Instead, we propose a new f…
In many applications, data is collected in batches, some of which are corrupt or even adversarial. Recent work derived optimal robust algorithms for estimating discrete distributions in this setting. We consider a general framework of robust learning from batches, and determine the limits of both classification and dis…
New framework shifts bandit algorithms from expected reward to preference metrics, optimizing mixtures of arms.
problem Traditional bandit algorithms focus on expected rewards, ignoring variability and risk.
method Introduces preference metrics (PMs) and designs algorithms to optimize mixtures of arms.
result Optimal policy selects mixtures of arms based on specific weights, not a single best arm.
A new method normalizes flow mixtures for better inference across different data types.
problem Inference failure across diverse posterior geometries in normalizing flows.
method Introduces a two-stage framework with a stable global weighting mechanism based on sEMA.
result Achieves consistent NLL improvements and stable weight trajectories over baselines.
We give a highly efficient "semi-agnostic" algorithm for learning univariate probability distributions that are well approximated by piecewise polynomial density functions. Let p be an arbitrary distribution over an interval I which is τ-close (in total variation distance) to an unknown probability distribution $…
Classical clients can verify quantum learning tasks efficiently.
problem Making quantum learning accessible to classical clients.
method Developed a framework for classical verification of quantum learning.
result Quantum learning tasks can be efficiently verified by classical verifiers.
New method prevents forgetting in learning new tasks.
problem Poor ability of models to solve new problems without forgetting.
method Task-agnostic hierarchical information-theoretic optimality principle with Mixture-of-Variational-Experts layer.
result Demonstrated competitive performance in continual supervised and reinforcement learning.
A new framework enhances binaural audio for moving talkers.
problem Real-time binaural audio enhancement for moving sound sources.
method Mixture-of-experts framework combining multiple binaural filters.
result Dynamic spatial audio rendering adapts to moving talkers.
New distributions allow greedy arm selection in sparse bandit problems.
problem Sparse contextual bandit problem with sparse parameters and feature distributions.
method Introduced new distribution classes and demonstrated that mixtures of these distributions are also greedy-applicable.
result Greedy algorithm applicable to a wider range of arm feature distributions, including those with origin-asymmetric support.
We consider the problem of estimating the common mean of independently sampled data, where samples are drawn in a possibly non-identical manner from symmetric, unimodal distributions with a common mean. This generalizes the setting of Gaussian mixture modeling, since the number of distinct mixture components may diverg…
We consider a scenario where an agent has multiple available strategies to explore an unknown environment. For each new interaction with the environment, the agent must select which exploration strategy to use. We provide a new strategy-agnostic method that treat the situation as a Multi-Armed Bandits problem where the…
The paper shows how training with synthetic data can lead to model improvement, not degradation, under certain conditions.
problem Model collapse in iterative training on contaminated sources.
method Statistical analysis of iterative training on a mixture of true and synthetic data.
result Training with synthetic data can lead to model improvement, not degradation, under specific conditions.
The paper cleans label noise in supervised classification using Bernoulli sampling.
problem Label noise degrades supervised classifier performance.
method Proposes a label noise cleaning method based on Bernoulli random sampling.
result The method separates clean and noisy observations without prior label information.
New IDS algorithm refines parameter norm bounds for better bandit performance.
problem Frequentist IDS requires tight norm bounds, which are often unavailable in practice.
method Iteratively refines a high-probability upper bound on true parameter norm using data.
result Regret bounds independent of assumed parameter norm, outperforming state-of-the-art algorithms.
We introduce a new and improved characterization of the label complexity of disagreement-based active learning, in which the leading quantity is the version space compression set size. This quantity is defined as the size of the smallest subset of the training data that induces the same version space. We show various a…
New method prevents forgetting in learning new tasks.
problem Learning new tasks without forgetting old knowledge.
method Hierarchical information-theoretic optimality principle and Mixture-of-Variational-Experts layer.
result Our method can operate task-agnostically and improve performance in various learning problems.
New privacy-preserving learning model for mixtures of private and public data.
problem Learning from datasets with both private and public data, where privacy concerns differ.
method Designing a differential privacy-preserving learning algorithm for a mixture of private and public sub-populations.
result Linear classifiers can be learned with sample complexity comparable to non-private PAC-learning, even when privacy status correlates with labels.
We consider off-policy policy evaluation when the trajectory data are generated by multiple behavior policies. Recent work has shown the key role played by the state or state-action stationary distribution corrections in the infinite horizon context for off-policy policy evaluation. We propose estimated mixture policy …
First proper learning algorithm for Gaussian halfspaces with matching sample and computational complexity.
problem Agnostically learning halfspaces under Gaussian distribution.
method First proper learning algorithm with matching sample and computational complexity.
result First proper learning algorithm for agnostically learning halfspaces under Gaussian distribution with matching sample and computational complexity.
In this paper, we study the problem of learning a mixture of Gaussians with streaming data: given a stream of N points in d dimensions generated by an unknown mixture of k spherical Gaussians, the goal is to estimate the model parameters using a single pass over the data stream. We analyze a streaming version of …
Extends boosting to multiclass online agnostic classification.
problem Online multiclass classification with weak learners.
method Reduces multiclass online agnostic boosting to online convex optimization.
result First boosting algorithm for online agnostic multiclass classification.
We establish a tight characterization of the worst-case rates for the excess risk of agnostic learning with sample compression schemes and for uniform convergence for agnostic sample compression schemes. In particular, we find that the optimal rates of convergence for size-k agnostic sample compression schemes are of…
Enhances financial optimization under model uncertainty using subsampling.
problem Model uncertainty in financial decision-making from limited data.
method Superimposes uncertainty measure on model space, uses subsampling for model distribution approximation, adapts SGD for efficiency.
result Uncertainty measures outperform traditional methods and achieve comparable performance to Bayesian methods.
New algorithm learns disjunctions faster than previous methods.
problem Learning Boolean disjunctions in the agnostic PAC model.
method Developed an agnostic learner with complexity 2ildeO(n1/3). result First separation between SQ and CSQ models in distribution-free agnostic learning.
The study optimizes polynomial regression for learning under Gaussian distributions.
problem Agnostic learning of Boolean and real-valued functions under Gaussian distributions.
method LP duality and polynomial degree analysis for L1-regression. result Optimal SQ lower bounds for various function classes.
New algorithms improve agnostic learning for triangles and polygons, reducing time complexity.
problem Efficient agnostic learning for geometric concept classes.
method Data structures and algorithms from computational geometry, probabilistic combinatorics.
result Optimal time complexity improvements for agnostic learning of triangles and polygons.
New algorithms save computation in agnostic learning with membership queries.
problem Efficiently learning touchstone classes with membership queries.
method Designing agnostic learning algorithms for circuits with sublinear gates.
result Agnostic learning algorithms for circuits with sublinear gates achieve significant computational savings.
Private distribution learning with public data, leveraging sample compression schemes.
problem Private distribution learning with public and private samples under differential privacy constraints.
method Connection to sample compression schemes and list learning.
result At least d public samples are necessary for private learnability of Gaussians in R^d.
New bounds for agnostic learning with average smoothness.
problem Distribution-free nonparametric regression with average smoothness.
method Distribution-free uniform convergence bounds and agnostic learning algorithm.
result Distribution-free uniform convergence bounds for average-smoothness classes in the agnostic setting.
Query access significantly speeds up learning Multi-Index Models under Gaussian distribution.
problem Agnostically learning Multi-Index Models (MIMs) under Gaussian distribution.
method Query access for MIMs with complexity O(k)poly(1/ε)poly(d) under standard regularity assumptions. result Query access gives significant runtime improvements over random examples for agnostically learning MIMs.
Boosting with unlabeled data achieves optimal sample complexity in agnostic settings.
problem Boosting's sample inefficiency in agnostic learning.
method Designing an agnostic boosting algorithm with unlabeled data to match ERM's sample complexity.
result The total sample complexity is optimal, with a vanishing fraction needing to be labeled.
Proposes class-agnostic object detection to handle all objects without class labels.
problem Difficulty and cost in creating annotated datasets limit conventional object detection models to specific object types.
method Proposes class-agnostic object detection as a new problem and proposes training and evaluation protocols. Uses adversarial learning to exclude class-specific information.
result Adversarial learning improves class-agnostic detection efficacy.
Study shows transductive learning is equivalent to PAC learning for most natural loss functions.
problem Understanding the relationship between transductive and PAC learning models.
method Extending existing results and developing new techniques to analyze the equivalence of the two models.
result Transductive learning is essentially equivalent to PAC learning for realizable learning with most natural loss functions.
Even in the absence of any explicit semantic annotation, vast collections of audio recordings provide valuable information for learning the categorical structure of sounds. We consider several class-agnostic semantic constraints that apply to unlabeled nonspeech audio: (i) noise and translations in time do not change t…
Model agnostic feature importance for DNNs in NLP.
problem Characterizing DNNs as black boxes and explaining their decision-making process.
method Phrase-wise feature importance calculation for model agnostic DNNs.
result Robust and generalizable approach to feature importance calculation.
Improved private agnostic learning with near-optimal sample complexity.
problem Private agnostic learning with arbitrary privacy parameters.
method Near-optimal sample complexity construction.
result Near-optimal extra sample complexity of \(\widetilde{O}(\mathrm{VC}(\mathcal{C})/α^2)\) for any \(\varepsilon \leq 1\).
AI methods broaden signal discovery in scientific data.
problem Limited coverage of possible signals in model-dependent searches.
method Model-agnostic AI strategies for broad exploration.
result Enhanced discovery potential in experimental science.