We study the problem of {\em distribution-independent} PAC learning of halfspaces in the presence of Massart noise. Specifically, we are given a set of labeled examples drawn from a distribution on such that the marginal distribution on the unlabeled points $\mathbf{x}…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
39 results for “distribution-independent”
In simulations of some economic gas-like models, the asymptotic regime shows an exponential wealth distribution, independently of the initial wealth distribution given to the system. The appearance of this statistical equilibrium for this type of gas-like models is explained in a rigorous analytical way.
Boosting algorithm reduces error in noisy data.
problem Improving weak learners in the presence of Massart noise.
method First computationally efficient boosting algorithm for Massart noise.
result Achieves misclassification error arbitrarily close to Massart noise threshold.
A Universal Trade-off Between the Model Size, Test Loss, and Training Loss of Linear Predictorsstat.ML
The study finds a trade-off between model size, test loss, and training loss for linear predictors.
problem Finding the optimal balance between model size, test loss, and training loss for linear predictors.
method Established an algorithm and distribution-independent trade-off using non-asymptotic analysis.
result Models with low test loss are either classical (close to noise level training loss) or modern (large number of parameters).
Study efficient learning of robust halfspaces with noise.
problem Learning robust halfspaces in the presence of adversarial perturbations and random label noise.
method Provides conditions for robust learnability and a simple algorithm for any ℓ_p perturbation.
result Simple computationally efficient algorithm for robust learning with random label noise.
New method denoises images without clean reference using Tweedie distributions.
problem Image denoising without clean reference images.
method Combining Tweedie distributions, Noise2Score, and saddle point approximation.
result General closed-form denoising formula for various noise distributions.
Study on learning halfspaces under adversarial perturbations, finding computational hardness.
problem Learning halfspaces in the presence of adversarial noise.
method Introduced an efficient learning algorithm and proved a nearly matching computational hardness result.
result The perturbations case is provably computationally harder than .
Study of estimation errors in surrogate loss minimizers, providing stronger guarantees than existing methods.
problem Estimation errors in surrogate loss minimizers for various hypothesis sets.
method Detailed study of -consistency estimation error bounds, proving general theorems for distribution-dependent and independent settings.
result Explicit bounds for zero-one and adversarial losses, showing enhancements under distributional assumptions.
We analyze the question whether sliding window time averages applied to stationary increment processes converge to a limit in probability. The question centers on averages, correlations, and densities constructed via time averages of the increment x(t,T)=x(t+T)-x(t)and the assumption is that the increment is distribute…
We develop and apply an approach for analyzing multi-curve data where each curve is driven by a latent state process. The state at any particular point determines a smooth function, forcing the individual curve to switch from one function to another. Thus each curve follows what we call a switching nonparametric regres…
New algorithms for GLMs with oblivious noise, identifying solutions even when half the data is corrupted.
problem Regression for GLMs with additive oblivious noise.
method Distribution-independent algorithms that return accurate estimates or candidate solutions.
result First algorithmic result for GLM regression with oblivious noise, handling more than half corrupted data.
New algorithm learns halfspaces with noise using Forster decomposition.
problem Learning halfspaces in noisy data.
method Forster decomposition and efficient mixture of distributions.
result First polynomial-time algorithm with strongly polynomial sample complexity.
Proposes a framework for modeling RTB auctions using point processes.
problem Modeling and optimizing repeated auctions in the RTB ecosystem.
method Develops a stochastic framework using point processes to model and optimize RTB auctions.
result The proposed framework can be approximated to a Poisson point process, enabling the use of established properties.
Estimating properties of discrete distributions is a fundamental problem in statistical learning. We design the first unified, linear-time, competitive, property estimator that for a wide class of properties and for all underlying distributions uses just samples to achieve the performance attained by the empirical…
Distributional Random Forests: Heterogeneity Adjustment and Multivariate Distributional Regressionstat.ML
A new random forest method for multivariate distributions.
problem Estimating complex multivariate distributions with heterogeneity.
method A novel splitting criterion based on MMD for multivariate responses.
result Estimates full conditional distribution for arbitrary targets.
In this paper we study the setting where features are added or change interpretation over time, which has applications in multiple domains such as retail, manufacturing, finance. In particular, we propose an approach to provably determine the time instant from which the new/changed features start becoming relevant with…
New framework improves learning across multiple distributions.
problem Modeling uncertainty in sensitive machine learning applications.
method Inspired by multi-armed bandits, provides distribution-dependent guarantees.
result Enhanced dependence on suboptimality gaps and sample size.
The paper improves density estimation in high dimensions using tensor decompositions.
problem Density estimation struggles in high-dimensional data due to the curse of dimensionality.
method The paper uses nonnegative tensor decompositions to simplify dependence assumptions and estimate marginal distributions.
result Theoretical results show that restricting estimation to low-rank nonnegative PARAFAC or Tucker decompositions removes the dimensionality exponent on bin width rates.
We develop a computationally efficient method to estimate Ollivier-Ricci curvature.
problem Computational infeasibility of evaluating Ollivier-Ricci curvature on large graphs.
method Derive explicit transfer moduli between OR and BF curvatures, construct lazy transport envelopes, and use cross-edge matching.
result Deterministic bounds for OR curvature parameterized by local graph combinatorics, reducing complexity to worst-case O(max_v deg(v)^1.5).
In this paper, we study the stochastic version of the one-sided full information bandit problem, where we have arms , and playing arm would gain reward from an unknown distribution for arm while obtaining reward feedback for all arms . One-sided full information bandit ca…
Neural networks provide a rich class of high-dimensional, non-convex optimization problems. Despite their non-convexity, gradient-descent methods often successfully optimize these models. This has motivated a recent spur in research attempting to characterize properties of their loss surface that may explain such succe…
In this paper, we study the stochastic combinatorial multi-armed bandit (CMAB) framework that allows a general nonlinear reward function, whose expected value may not depend only on the means of the input random variables but possibly on the entire distributions of these variables. Our framework enables a much larger c…
Kernel methods are powerful and flexible approach to solve many problems in machine learning. Due to the pairwise evaluations in kernel methods, the complexity of kernel computation grows as the data size increases; thus the applicability of kernel methods is limited for large scale datasets. Random Fourier Features (R…
This paper tackles online reinforcement learning for unseen tasks with unknown boundaries.
problem Real-world tasks violate assumptions of task distributions, independence, and clear task delineations.
method A mixture of Gaussian Processes models different dynamics, and a transition prior handles temporal dependencies.
result The approach reliably handles task distribution shifts and outperforms alternatives in non-stationary tasks.
Study shows SGD's generalization is not explained by implicit bias.
problem Explaining the generalization ability of overparameterized learning algorithms.
method Revisited Stochastic Convex Optimization with SGD, demonstrating limitations of implicit bias.
result No distribution-independent or distribution-dependent implicit regularizer can explain SGD's generalization.
Lower bound proves ridgeless regression performs poorly near interpolation threshold.
problem Proving performance of ridgeless regression near interpolation threshold.
method Distribution-independent lower bound for mean squared error in noisy ridgeless linear regression.
result Lower bound implies ridgeless regression performs poorly near interpolation threshold.
How does one find dimensions in multivariate data that are reliably expressed across repetitions? For example, in a brain imaging study one may want to identify combinations of neural signals that are reliably expressed across multiple trials or subjects. For a behavioral assessment with multiple ratings, one may want …
LDAO addresses imbalanced regression by learning local distribution structures.
problem Imbalanced regression with sparse target regions difficult for models.
method LDAO learns local distribution structures, models and samples from each, then merges.
result LDAO outperforms state-of-the-art methods on 45 imbalanced datasets.
Boltzmann exploration is a classic strategy for sequential decision-making under uncertainty, and is one of the most standard tools in Reinforcement Learning (RL). Despite its widespread use, there is virtually no theoretical understanding about the limitations or the actual benefits of this exploration scheme. Does it…
Analyzing deep neural networks (DNNs) via information plane (IP) theory has gained tremendous attention recently as a tool to gain insight into, among others, their generalization ability. However, it is by no means obvious how to estimate mutual information (MI) between each hidden layer and the input/desired output, …
Deep neural networks are often trained in the over-parametrized regime (i.e. with far more parameters than training examples), and understanding why the training converges to solutions that generalize remains an open problem. Several studies have highlighted the fact that the training procedure, i.e. mini-batch Stochas…
New findings on boosting sample complexity and implications for hardcore theorem.
problem Understanding the sample complexity of smooth boosting and its implications.
method Analyzing the sample complexity of smooth boosting and relating it to the hardcore theorem.
result The sample complexity of smooth boosting matches existing overhead and provides a separation from distribution-independent boosting.
New insights on robust learning under strong noise models.
problem Challenging label-noise models in robust learning.
method Extending statistical query framework to more general noise models and using evolutionary algorithms.
result First polynomial time algorithm for learning linear threshold functions with arbitrarily small excess error in presence of Tsybakov noise.
New denoisers improve signal recovery from noisy data without knowing noise distribution.
problem Denoising signals when only noise level is known, not distribution.
method Universal denoisers that shrink toward with higher-order accuracy.
result Achieves and accuracy in matching generalized moments and densities.
We consider learning under the constraint of local differential privacy (LDP). For many learning problems known efficient algorithms in this model require many rounds of communication between the server and the clients holding the data points. Yet multi-round protocols are prohibitively slow in practice due to network …
Classifiers deployed in the real world operate in a dynamic environment, where the data distribution can change over time. These changes, referred to as concept drift, can cause the predictive performance of the classifier to drop over time, thereby making it obsolete. To be of any real use, these classifiers need to d…
There is accumulating evidence in the literature that stability of learning algorithms is a key characteristic that permits a learning algorithm to generalize. Despite various insightful results in this direction, there seems to be an overlooked dichotomy in the type of stability-based generalization bounds we have in …
Record Statistics for Multiple Random Walkscond-mat.stat-mech
We study the statistics of the number of records R_{n,N} for N identical and independent symmetric discrete-time random walks of n steps in one dimension, all starting at the origin at step 0. At each time step, each walker jumps by a random length drawn independently from a symmetric and continuous distribution. We co…
Hardness proof for agnostically learning halfspaces from worst-case lattice problems.
problem Agnostically learning halfspaces in the presence of noise.
method Reduction to worst-case lattice problems (GapSVP, SIVP).
result No efficient algorithm can achieve misclassification error better than 1/2 - γ under given hardness assumptions.