Infinitesimal boosting converges to a deterministic process in large sample limit.
problem Characterizing the asymptotic behavior of infinitesimal gradient boosting in large sample sizes.
method Proving convergence to a deterministic process using large sample theory and differential equations.
result The test error decreases over time in the population limit.
We show through case studies that it is easier to estimate the fundamental limits of data processing than to construct explicit algorithms to achieve those limits. Focusing on binary classification, data compression, and prediction under logarithmic loss, we show that in the finite space setting, when it is possible to…
New method combines multiple data sources for optimal decision-making with limited outcomes.
problem Optimal decision-making with limited outcome data from multiple heterogeneous sources.
method Calibrated optimal decision-making method leveraging common intermediate outcomes.
result Proposed estimator of conditional mean outcome is asymptotically normal and more efficient.
Meta two-sample testing uses auxiliary data to quickly find powerful tests from limited samples.
problem Challenges in identifying powerful kernels for distinguishing complex distributions with limited data.
method Introduces meta two-sample testing (M2ST) to leverage abundant auxiliary data on related tasks.
result Proposed algorithms improve over baselines and identify powerful tests from scarce observations.
Method estimates model performance on external samples from limited statistical characteristics.
problem Limited access to multiple datasets due to privacy and commercial restrictions.
method Search for weights that match external statistics and are closest to uniform, using model performance on weighted internal sample as an estimation.
result Estimated external performance is closer to actual performance than internal performance.
Study the limits of discrete DPPs to continuous DPPs as set size grows.
problem Characterize the behavior of discrete DPPs as they approach continuous DPPs.
method Non-asymptotic characterization of the limit in terms of weak coherency.
result Sufficient conditions for weak coherency are identified.
Adaptive sampling detects local concept drift with limited labels.
problem Detecting local concept drift in dynamic environments with scarce labels.
method Combines residual-based exploration and exploitation with EWMA monitoring.
result Superior performance in label efficiency and drift detection accuracy.
Paper tackles offline RL with limited target samples using domain adaptation.
problem Limited samples in target dataset degrade offline RL performance.
method Proposes a framework to balance target and source datasets with theoretical guarantees.
result Establishes performance bounds and optimal weight for offline RL.
New methods for estimating causal effects with limited overlap, using Stable Probability Weighting.
problem Estimating causal effects with limited overlap in multivalued treatments.
method Stable Probability Weighting (SPW) and Finite-Sample Stable Probability Weighting (FPW) methods.
result SPW and FPW provide practical solutions for estimating and inferring causal effects with limited overlap.
The paper studies how more data affects prediction risk in high-dimensional models.
problem The impact of increasing data on prediction risk in high-dimensional models.
method Derives central limit theorem and provides finite-sample distribution and confidence interval for prediction risk.
result Demonstrates 'more data hurt' phenomenon in high-dimensional least squares estimation.
The study proves sampling-based GNNs can approximate training on full graphs with small subgraphs.
problem Training Graph Neural Networks (GNNs) on large graphs is computationally expensive.
method Theoretical framework using graph local limits to prove approximation of GNN training on small samples.
result Parameters learned from sampling-based GNNs on small subgraphs are close to those on full graphs.
A new approach models exploration in continuous-time RL using random measures.
problem Modeling exploration in continuous-time reinforcement learning.
method Random measure approach to control execution in continuous-time RL.
result Grid-sampling limit SDE can replace existing models for theoretical analysis and learning algorithms.
The paper analyzes phase retrieval under limited samples, ensuring a benign local landscape for convergence.
problem Ensuring a benign local landscape for phase retrieval under limited samples.
method Fine-grained analysis of local landscape properties under the regime of limited samples.
result Gradient descent can converge to an o d ( 1 ) o_d(1) o d ( 1 ) -loss solution exponentially fast under certain conditions. New bounds show limitations of sample-wise information-theoretic generalization.
problem Limitations of sample-wise information-theoretic generalization bounds.
method Analysis of existing bounds and derivation of new bounds.
result No sample-wise information-theoretic bounds exist for expected squared generalization gap.
Two parallel samplers enhance image quality in limited denoising steps.
problem Limited denoising steps in diffusion models reduce image quality.
method Two parallel samplers denoise at successive times, integrating their information.
result Two parallel samplers improve image quality compared to a single sampler.
This study improves uncertainty quantification in seismic inversion.
problem Uncertainty in seismic inversion due to limited data and model diversity.
method Integrates ensemble methods with importance sampling.
result More accurate uncertainty quantification in velocity models.
Extends Thompson sampling for RL with fewer episodes.
problem Limited episodes in RL settings.
method Batch Bayesian optimization over episodes to learn action bias terms.
result Significantly outperforms standard Thompson sampling.
Machine learning's predictive power is limited by sample size, as shown by the Limits-to-Learning Gap.
problem The limitations of machine learning in approximating true data-generating processes.
method Characterization of a universal lower bound (LLG) quantifying the discrepancy between empirical fit and population benchmark.
result Standard ML approaches can substantially understate true predictability in financial data.
We unify slice sampling and Hamiltonian Monte Carlo (HMC) sampling, demonstrating their connection via the Hamiltonian-Jacobi equation from Hamiltonian mechanics. This insight enables extension of HMC and slice sampling to a broader family of samplers, called Monomial Gamma Samplers (MGS). We provide a theoretical anal…
The paper analyzes SGD in high-dimensional networks, revealing new scaling limits.
problem Understanding SGD dynamics in high-dimensional networks.
method Analyzing the effective dynamics of SGD using recent work on the subject.
result A new correction term emerges at the critical scaling regime, changing the phase diagram.
Enhanced Sampling Scheme improves masked generative modeling.
problem Limitations of existing sampling schemes in masked non-autoregressive generative modeling.
method ESS consists of three stages: Naive Iterative Decoding, Critical Reverse Sampling, and Critical Resampling.
result ESS achieves significant performance gains in unconditional and class-conditional sampling.
LGES speeds up causal discovery while maintaining accuracy.
problem Causal discovery from observational data with computational and accuracy limitations.
method LGES modifies GES by avoiding certain edge insertions, using prior knowledge, and leveraging interventional data.
result LGES outperforms GES in speed, accuracy, and robustness to misspecified knowledge.
UD-SGD analysis shows efficient sampling by a few agents can outperform others.
problem Analyzing convergence speed and sampling strategies in UD-SGD.
method Asymptotic analysis of UD-SGD with various communication patterns and sampling strategies.
result Efficient sampling by a few agents can lead to better overall convergence.
Study best arm identification with limited precision sampling in bandits.
problem Limited precision sampling in multi-armed bandit problems.
method Proposed a modified tracking-based algorithm to handle non-unique optimal allocations and presented non-asymptotic bounds.
result Asymptotically optimal tracking-based algorithm for best arm identification.
We prove exact BNN posterior convergence to GP limit and provide sampling methods.
problem Theoretical and empirical challenges in obtaining exact posterior distributions of wide BNNs.
method Theoretical proof and rejection sampling for generating exact samples.
result Exact BNN posterior converges to GP limit as width increases.
RISA improves VFL by using imputed samples with low uncertainty.
problem Limited overlapping samples constrain VFL performance.
method Imputing non-overlapping samples and using evidence theory to select reliable imputed samples.
result Significant performance gains achieved, especially with limited overlapping samples.
Paper proposes efficient GCN learning method for limited data.
problem Learning GCNs from data with extremely limited annotations.
method Adaptive sampling strategy and model compression.
result Cut down annotation requirement by 90% and compress parameters 6x.
New method uses birth-death process and exploration component to accelerate sampling from multimodal distributions.
problem Sampling from multimodal probability distributions efficiently.
method Combines birth-death process and exploration component to accelerate sampling.
result Proves exponential asymptotic convergence under mild assumptions.
This work presents a theoretical and empirical evaluation of Anderson-Darling test when the sample size is limited. The test can be applied in order to backtest the risk factors dynamics in the context of Counterparty Credit Risk modelling. We show the limits of such test when backtesting the distributions of an intere…
Optimization algorithms and Monte Carlo sampling algorithms have provided the computational foundations for the rapid growth in applications of statistical machine learning in recent years. There is, however, limited theoretical understanding of the relationships between these two kinds of methodology, and limited unde…
Meta-learning enables a model to learn from very limited data to undertake a new task. In this paper, we study the general meta-learning with adversarial samples. We present a meta-learning algorithm, ADML (ADversarial Meta-Learner), which leverages clean and adversarial samples to optimize the initialization of a lear…
New Thompson sampling algorithm for stochastic partial monitoring achieves logarithmic regret.
problem Limited feedback in sequential learning problems.
method Developed a novel Thompson-sampling-based algorithm to sample from the posterior distribution exactly.
result Achieved logarithmic regret bound of O(log T) for a linearized variant of the problem.
Machine learning has been applied to a broad range of applications and some of them are available online as application programming interfaces (APIs) with either free (trial) or paid subscriptions. In this paper, we study adversarial machine learning in the form of back-box attacks on online classifier APIs. We start w…
Paper detects Trojan neural networks with limited or no data.
problem Robustness of DNNs against Trojan attacks in data-scarce scenarios.
method Proposes data-limited and data-free TrojanNet detectors.
result Demonstrates effectiveness of detectors on various datasets.
This work uses sampling theory to analyze smoothness and error bounds of finite neural networks.
problem Analyzing the function space of finite neural networks and providing error bounds.
method Applying sampling theory to finite neural networks with non-expansive activation functions, considering both deterministic and random sampling.
result Novel error bounds for univariate neural networks under band-limited input assumption, highlighting the advantage of deterministic uniform sampling.
Study examines influence diagnostics in high-dimensional M-estimation.
problem Understanding influence diagnostics in high-dimensional settings.
method Characterized the distribution of leave-one-out influences in high-dimensional Gaussian M-estimation.
result The distribution of influences converges to a limiting measure in high-dimensional settings.
Study mini-batch SGD noise and its limits, proving complexity guarantees.
problem Analyzing the noise in mini-batch SGD and its impact on optimization.
method Examined the conditional covariance and diffusion limits of SGD under different sampling designs.
result Proved mean-square upper bounds and Fisher van Trees lower bounds for SGD, linking them to effective dimension and condition number.
Characterizes uncertainty in high-dimensional linear classification models.
problem Assessing uncertainty in high-dimensional linear classification models.
method Approximate message passing algorithm for posterior marginals, closed-form formula for joint statistics.
result Closed-form formula for joint statistics between logistic classifier, Bayesian uncertainty, and ground-truth probit uncertainty.
Neural networks learn complex functions efficiently near information-theoretic limits.
problem Understanding how neural networks learn high-dimensional features.
method Gradient descent learning of a Gaussian Multi-index model with hidden subspace.
result A standard two-layer neural network can learn the target with optimal sample and time complexity.
This paper strengthens the central limit theorem for order statistics using relative entropy.
problem Establishing a stronger mode of convergence for central limit behavior of order statistics.
method Using relative entropy to ensure a stronger mode of convergence for central limit behavior of order statistics.
result An order O ( 1 / n ) O(1/\sqrt{n}) O ( 1/ n ) rate of convergence is established under mild conditions. Many popular dimensionality reduction procedures have out-of-sample extensions, which allow a practitioner to apply a learned embedding to observations not seen in the initial training sample. In this work, we consider the problem of obtaining an out-of-sample extension for the adjacency spectral embedding, a procedure…
Image modality recognition is essential for efficient imaging workflows in current clinical environments, where multiple imaging modalities are used to better comprehend complex diseases. Emerging biomarkers from novel, rare modalities are being developed to aid in such understanding, however the availability of these …
Extended study improves covariance matrix estimation for portfolio managers.
problem Limited sample sizes and poor performance of PCA estimator in high-dimensional returns.
method Developed a more general shrinkage framework targeting further information.
result Improves the PCA estimator of beta by shrinking it toward a target.
RLFA estimates misstated monetary fraction with weighted sampling without replacement.
problem Estimating misstated monetary fraction with given accuracy and confidence.
method Developed new confidence sequences for weighted average of unknown values using randomized weighted sampling and side information.
result Adaptive methods improve accuracy of estimates based on side information's predictive power.
Study evaluates synthetic data augmentation for small datasets, highlighting inconsistencies in traditional metrics.
problem Inconsistent validation of synthetic data generated for small sample sizes.
method Proposes a normalized Bottleneck distance metric to evaluate synthetic tabular data.
result Common metrics like propensity scoring and MMD fail for small datasets, showing instability and high variability.
The paper investigates the convergence of Vendi scores under finite samples and introduces a truncated version for better performance.
problem The Vendi score's convergence is hindered by computational limitations when using large sample sizes.
method The authors introduce the t-truncated Vendi score to address this issue by truncating the eigenspectrum of the kernel matrix.
result The t-truncated Vendi score converges to its asymptotic limit with a smaller number of samples, improving upon the standard Vendi score.
PRS improves rejection sampling by learning better proposals.
problem High rejection rate in traditional rejection sampling.
method PRS uses a kernel estimator to learn better sampling proposals.
result PRS guarantees a low number of accepted samples.
MGD combines maximum entropy and diffusion methods for efficient sampling.
problem Generating samples from limited information in high dimensions.
method Moment Guided Diffusion (MGD) using stochastic differential equations.
result MGD efficiently samples maximum entropy distributions in finite time.