Adapts DR objectives for both sample and feature size reduction.
problem Simultaneously reduce sample and feature sizes.
method Semi-relaxed Gromov-Wasserstein optimal transport.
result OT plan delivers competitive hard clustering.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “sample size reduction”
Paper proposes a method to estimate variance reduction in DNN training using importance sampling.
problem Challenges in assessing variance reduction during DNN training using importance sampling.
method Proposes a method for estimating variance reduction using minibatches sampled under importance sampling.
result Demonstrates consistent reduction in variance, improved training efficiency, and enhanced model accuracy.
Study finds that only a fraction of data is needed for accurate patient-level prediction models.
problem Developing predictive models for patient-level outcomes using large observational data.
method Empirical assessment of sample size effects on model performance and complexity using learning curves.
result A median reduction of 9.5% to 78.5% in the number of observations and 8.6% to 68.3% in the number of predictors can be achieved with adequate sample size.
SPreVcs.GR
SPREV simplifies visualization of complex labeled datasets.
problem Challenges of reducing dimensions and visualizing labeled datasets with small class size, high dimensionality, and low sample size.
method SPREV uses a novel dimensionality reduction technique integrating geometric principles.
result SPREV effectively visualizes hidden patterns in complex labeled datasets.
This paper explores how effective sample size, dimensionality, and model performance are related in covariate shift adaptation.
problem Understanding the relationship between effective sample size, dimensionality, and generalization in covariate shift adaptation.
method Building a unified theory connecting effective sample size, data dimensionality, and generalization in the context of covariate shift adaptation.
result Dimensionality reduction or feature selection can increase effective sample size, supporting the practice of reducing dimensionality before covariate shift adaptation.
New RL algorithm reduces sample complexity for optimal learning.
problem Achieving optimal learning with minimal samples in RL.
method Early-settled variance reduction method with Q-learning sequences.
result Near-optimal regret achieved with sample size .
MSTGD:A Memory Stochastic sTratified Gradient Descent Method with an Exponential Convergence Ratestat.ML
MSTGD optimizes gradient descent with stratified sampling for faster convergence.
problem Fluctuation in gradient expectation and variance between iterations.
method Memory Stochastic Stratified Gradient Descent (MSTGD) with stratified sampling and variance reduction.
result MSTGD achieves an exponential convergence rate independent of dataset size and batch size.
Reduces multiclass and regression compression schemes to binary ones.
problem Developing efficient learning algorithms for multiclass and regression problems.
method Reduces sample compression schemes for binary classes to multiclass and regression settings.
result Establishes new compression schemes for multiclass and regression problems.
Algorithme EM régulariséstat.ML
Regularized EM algorithm improves clustering performance with small sample sizes.
problem Performance reduction in EM algorithm due to small sample size and poorly conditioned covariance matrices.
method Regularized EM algorithm that uses prior knowledge to ensure positive definiteness of covariance matrices.
result The regularized EM algorithm outperforms standard EM in clustering tasks with small sample sizes.
In this paper we study a family of variance reduction methods with randomized batch size---at each step, the algorithm first randomly chooses the batch size and then selects a batch of samples to conduct a variance-reduced stochastic update. We give the linear convergence rate for this framework for composite functions…
In this paper, we propose a stochastic optimization method that adaptively controls the sample size used in the computation of gradient approximations. Unlike other variance reduction techniques that either require additional storage or the regular computation of full gradients, the proposed method reduces variance by …
Optimizes sampling in continuous domains by adjusting search distribution.
problem Improving sampling efficiency in continuous domains.
method Analyzes and refines the search distribution based on population size and dimension.
result Explicit values for reshaping the search distribution are provided.
Privacy affects how much data is needed for CVaR optimization.
problem Privacy constraints impact the effective sample size for CVaR optimization.
method Analyzes the privacy-relevant sample size and decomposes CVaR excess risk.
result The effective private tail sample size is εnτ, affecting CVaR learning rates.
We present a novel Metropolis-Hastings method for large datasets that uses small expected-size minibatches of data. Previous work on reducing the cost of Metropolis-Hastings tests yield variable data consumed per sample, with only constant factor reductions versus using the full dataset for each sample. Here we present…
Paper proposes a method to use in silico experiments with foundation models to reduce sample size.
problem Costly and uncertain randomized experiments.
method Integrates predictions from multiple foundation models with experimental data.
result Estimator offers substantial precision gains, equivalent to a 20% reduction in sample size.
Stochastic gradient Markov Chain Monte Carlo (SG-MCMC) has been developed as a flexible family of scalable Bayesian sampling algorithms. However, there has been little theoretical analysis of the impact of minibatch size to the algorithm's convergence rate. In this paper, we prove that under a limited computational bud…
Adaptive Importance Sampling for Finite-Sum Optimization and Sampling with Decreasing Step-Sizescs.LG
Avare improves optimization and sampling with adaptive importance sampling.
problem Improving convergence rate of stochastic gradient-based algorithms.
method Adaptive importance sampling with decreasing step-sizes.
result Achieves dynamic regret bounds of and .
Contrary to the situation with stochastic gradient descent, we argue that when using stochastic methods with variance reduction, such as SDCA, SAG or SVRG, as well as their variants, it could be beneficial to reuse previously used samples instead of fresh samples, even when fresh samples are available. We demonstrate t…
Bagging reduces variance in LID estimation by preserving local distribution of NN distances.
problem High estimation variance from limited data in small neighborhoods.
method Subbagging to preserve local distribution of NN distances, combined with ensemble size.
result Bagging significantly reduces variance and MSE in LID estimation.
Study compares under-bagging with other methods for imbalanced data.
problem Comparing under-bagging with other methods for imbalanced data.
method Replica analysis of under-bagging, comparing with under-sampling and simple weighting.
result Under-bagging improves performance by increasing the majority class size.
Improved TD learning reduces variance and bias errors.
problem Inefficient optimization variance in TD learning.
method Proposed a mathematically solid analysis of VRTD, showing linear convergence rate and reduced variance and bias errors.
result VRTD converges to a fixed-point solution with reduced variance and bias errors compared to vanilla TD.
Approximate Kernel-based Conditional Independence Tests for Fast Non-Parametric Causal Discoverystat.ME
Constraint-based causal discovery (CCD) algorithms require fast and accurate conditional independence (CI) testing. The Kernel Conditional Independence Test (KCIT) is currently one of the most popular CI tests in the non-parametric setting, but many investigators cannot use KCIT with large datasets because the test sca…
U-statistics improve gradient estimation in importance-weighted variational inference.
problem High variance in gradient estimation for importance-weighted variational inference.
method Use U-statistics to average base gradient estimators on overlapping batches of size m, achieving lower variance.
result U-statistic variance reduction leads to modest to significant improvements in inference performance.
Distributed estimation of principal support vector machines for sufficient dimension reductionstat.ML
The principal support vector machines method (Li et al., 2011) is a powerful tool for sufficient dimension reduction that replaces original predictors with their low-dimensional linear combinations without loss of information. However, the computational burden of the principal support vector machines method constrains …
We propose a new yet natural algorithm for learning the graph structure of general discrete graphical models (a.k.a. Markov random fields) from samples. Our algorithm finds the neighborhood of a node by sequentially adding nodes that produce the largest reduction in empirical conditional entropy; it is greedy in the se…
Improved variance reduction for Riemannian non-convex optimization with adaptive batch size.
problem Optimizing non-convex functions on Riemannian manifolds.
method Batch size adaptation in R-SVRG, R-SRG, and R-SPIDER.
result Achieves lower total complexities for various non-convex functions.
Graph convolutional networks (GCNs) are powerful deep neural networks for graph-structured data. However, GCN computes the representation of a node recursively from its neighbors, making the receptive field size grow exponentially with the number of layers. Previous attempts on reducing the receptive field size by subs…
Improved stochastic optimization outperforms standard methods.
problem Optimizing smooth, strongly convex functions with noisy data.
method Variance reduction strategy called VISOR.
result VISOR achieves optimal sample complexity and oracle complexity.
This paper tackles variance issues in GNN training by proposing a method to reduce both embedding and gradient variances.
problem High variance in estimating stochastic gradients in GNN training, especially in large graphs.
method The paper proposes a decoupled variance reduction strategy that employs approximate gradient information to adaptively sample nodes with minimal variance.
result The proposed method achieves faster convergence and better generalization compared to existing sampling methods.
New sampling strategy preserves relationships in multivariate scientific data.
problem Reducing storage and enabling efficient multivariate analyses on large scientific data.
method Uses principal component analysis for multivariate data and combines with existing univariate sampling algorithms.
result Efficacy demonstrated on real-world data sets, showing data reduction and multivariate analysis ease.
New auditors assess -DP privacy with adaptive sampling, avoiding large sample sizes.
problem Empirical auditing of -DP privacy with adaptive sampling.
method Shift focus to -DP, develop adaptive auditors for whitebox and blackbox settings.
result Adaptive auditors detect -DP violations across the privacy spectrum with statistical guarantees.
Paper reduces recommender system model size by 90%.
problem Large model size in recommender systems.
method Hybrid frequency and double hashing for model size reduction.
result 90% reduction in model size with no performance loss.
A new dimension reduction method based on Gaussian finite mixtures is proposed as an extension to sliced inverse regression (SIR). The model-based SIR (MSIR) approach allows the main limitation of SIR to be overcome, i.e., failure in the presence of regression symmetric relationships, without the need to impose further…
Eigen-stratified models reduce model size and improve performance.
problem Large model size in Laplacian-regularized stratified models.
method Formulate eigen-stratified models with linear combinations of bottom eigenvectors of the graph Laplacian.
result Significant reduction in model size with eigen-stratified models.
Divide and Conquer Kernel Ridge Regression: A Distributed Algorithm with Minimax Optimal Ratesmath.ST
We establish optimal convergence rates for a decomposition-based scalable approach to kernel ridge regression. The method is simple to describe: it randomly partitions a dataset of size N into m subsets of equal size, computes an independent kernel ridge regression estimator for each subset, then averages the local sol…
Reduced sample complexity for group-invariant distributions.
problem Improving sample complexity for estimating divergences of group-invariant distributions.
method Quantified reduction in sample complexity for Wasserstein-1 metric and Lipschitz-regularized α-divergences under finite and infinite groups.
result Sample complexity reduction proportional to group size for finite groups, and convergence rate depends on intrinsic dimension for infinite groups.
Consider a Markov decision process (MDP) that admits a set of state-action features, which can linearly express the process's probabilistic transition model. We propose a parametric Q-learning algorithm that finds an approximate-optimal policy using a sample size proportional to the feature dimension and invariant …
In this paper we propose the macroblock scaling (MBS) algorithm, which can be applied to various CNN architectures to reduce their model size. MBS adaptively reduces each CNN macroblock depending on its information redundancy measured by our proposed effective flops. Empirical studies conducted with ImageNet and CIFAR-…
Deep Neural Networks (DNNs) thrive in recent years in which Batch Normalization (BN) plays an indispensable role. However, it has been observed that BN is costly due to the reduction operations. In this paper, we propose alleviating this problem through sampling only a small fraction of data for normalization at each i…
New HMC framework reduces variance for sampling from log-concave distributions.
problem Efficient sampling from log-concave distributions with high precision.
method Unified formulation of biased and unbiased variance reduction methods for HMC.
result Unbiased and biased gradient estimators achieve different gradient complexities and accuracy.
New method for spatiotemporal data regression using Gaussian processes.
problem Regression in spatiotemporal random fields.
method Empirical Bayes approach, tight Gaussian measures, truncation scheme.
result Effective dimension reduction through time-varying angular spectra.
SDR outperforms IDR in multimodal data analysis, especially with fewer samples.
problem Understanding and optimizing data efficiency in multimodal representation learning.
method Generative linear model to synthesize multimodal data, comparing IDR and SDR methods.
result Linear SDR methods yield higher-quality, more succinct reduced-dimensional representations with smaller datasets.
To improve the efficiency of Monte Carlo estimation, practitioners are turning to biased Markov chain Monte Carlo procedures that trade off asymptotic exactness for computational speed. The reasoning is sound: a reduction in variance due to more rapid sampling can outweigh the bias introduced. However, the inexactness …
We study the complexity of sampling from a distribution over all index subsets of the set with the probability of a subset proportional to the determinant of the submatrix of some p.s.d. matrix , where corresponds to the entries of ind…
Proposes an online method for high-dimensional streaming data.
problem Increasing variable dimensions with sample size in online kernel sliced inverse regression.
method Introduces approximate linear dependence condition and dictionary variable sets to address the problem. Transforms into online generalized eigen-decomposition problem and uses stochastic optimization for updates.
result Achieves close performance to batch processing kernel sliced inverse regression.
A new method improves active learning for large batch sizes.
problem Challenges in scaling Bayesian active learning to large batch sizes.
method Derives Partial Batch Label Sampling (ParBaLS) for EPIG algorithm.
result ParBaLS EPIG outperforms top- selection and BatchBALD.
We propose a sample efficient stochastic variance-reduced cubic regularization (Lite-SVRC) algorithm for finding the local minimum efficiently in nonconvex optimization. The proposed algorithm achieves a lower sample complexity of Hessian matrix computation than existing cubic regularization based methods. At the heart…
A new method reduces variance in SGMCMC by preferentially subsampling data.
problem High variance in stochastic gradient estimates impacts sampler performance.
method Use a non-uniform probability distribution to preferentially subsample data points and adaptively adjust subsample size.
result Maintains accuracy while substantially reducing average subsample size.