Proofs high-dimensional spectrum convergence of weighted sample covariance.
problem High-dimensional spectrum convergence of weighted sample covariance.
method Proposes a new, concise proof with stronger assumptions.
result Spectrum convergence proven for different weight distributions.
The importance-weighted risk estimator can be skewed, leading to suboptimal regularization parameters.
problem Skewed sampling distribution of the importance-weighted risk estimator affects model selection.
method Empirical study of the sampling distribution of the importance-weighted risk estimator.
result The importance-weighted risk estimator produces overestimates for the majority of cases and underestimates for tail cases, leading to suboptimal regularization parameters.
Optimizes sample weights for representative data averages.
problem Achieving sample averages close to prescribed values.
method Formulates as an optimization problem, often convex and efficiently solvable.
result Heuristic methods based on convex optimization perform well.
Estimates Gaussian mixtures from weighted samples efficiently.
problem Estimating Gaussian mixtures from weighted samples with correct weight treatment.
method Density interpretation and expectation-maximization method considering weights.
result Correctly estimates Gaussian mixtures with weighted samples.
New method improves covariance estimation for weighted samples.
problem Improving covariance estimation for weighted sample data.
method Asymptotic non-linear shrinkage formulas for covariance and precision matrix estimators of weighted sample covariances.
result Asymptotic non-linear shrinkage formulas for covariance and precision matrix estimators of weighted sample covariances.
Improves cross-validation for biased data by adjusting risk estimator variance.
problem Cross-validation under sample selection bias produces suboptimal results.
method Introduces control variate to reduce variance of importance-weighted risk estimator.
result Control variate increases robustness to problematic weights.
Method identifies change points in high-dimensional models using sample weights.
problem Identifying change points in high-dimensional generalized linear models.
method Sample-weighted empirical risk minimization (Weighted ERM).
result Weighted ERM yields precise asymptotic performance characterization for Gaussian designs.
New summary measures reveal geometric structure in weighted measures on manifolds.
problem Lack of geometric information in standard weight-only summaries.
method Heat-kernel entropy profiles, tracking nonuniformity across scales.
result Geometric effective sample size discounts nearby or duplicate particles.
Recently theoretical guarantees have been obtained for matrix completion in the non-uniform sampling regime. In particular, if the sampling distribution aligns with the underlying matrix's leverage scores, then with high probability nuclear norm minimization will exactly recover the low rank matrix. In this article, we…
Adaptive learning of sample weights for better model performance.
problem Overfitting to biased training data with corrupted labels or class imbalance.
method Adaptive learning of an explicit weighting function using a meta-weight-net.
result Improves model accuracy in class imbalance and noisy label cases.
New method reduces bias in high-dimensional action spaces for efficient reinforcement learning.
problem Large bias and difficulty in reusing old samples in high-dimensional action spaces.
method Dimension-wise IS weight clipping to control bias and adaptively manage IS weights.
result Proposed method outperforms PPO and other RL algorithms in various tasks.
We provide rigorous guarantees on learning with the weighted trace-norm under arbitrary sampling distributions. We show that the standard weighted trace-norm might fail when the sampling distribution is not a product distribution (i.e. when row and column indexes are not selected independently), present a corrected var…
New sketches for weighted ℓp sampling without replacement improve accuracy and efficiency.
problem Efficiently sampling weighted data with high accuracy and minimal redundancy.
method Design of novel composable sketches for WOR ℓp sampling, based on CountSketch. result First to provide WOR sampling for p>1 and signed updates. Enhances physics-informed neural networks with adaptive sampling and weighting.
problem Challenges in training physics-informed neural networks on complex problems.
method Hybrid adaptive sampling and weighting method.
result Consistently improves prediction accuracy and training efficiency.
Word2vec skip-gram with negative sampling is explained as a weighted logistic PCA.
problem Understanding word2vec skip-gram with negative sampling.
method Equivalence to weighted logistic PCA, comparison to other methods, extension to higher dimensions.
result Word2vec skip-gram with negative sampling is a weighted logistic PCA.
NS-GAN mode collapse due to sample weighting inversion, solved with MM-nsat.
problem Mode collapse in GANs due to sample weighting inversion.
method Preserves MM-GAN sample weighting while avoiding saturation by rescaling gradients.
result MM-nsat improves mode coverage, stability, and FID on MNIST and CIFAR-10.
Adaptive sampling method improves efficiency in complex target distributions.
problem Efficiency of importance sampling in complex target distributions, especially multimodal distributions in high-dimensional spaces.
method Proposes an adaptive scheme combining global sampling with delayed weighting to promote efficient exploration of target distributions.
result The proposed algorithm is geometrically convergent under mild assumptions and demonstrates improved efficiency in various numerical experiments.
RLFA estimates misstated monetary fraction with weighted sampling without replacement.
problem Estimating misstated monetary fraction with given accuracy and confidence.
method Developed new confidence sequences for weighted average of unknown values using randomized weighted sampling and side information.
result Adaptive methods improve accuracy of estimates based on side information's predictive power.
Paper characterizes sampling distributions of optimal portfolio weights and characteristics.
problem Characterizing sampling distributions of optimal portfolio weights and characteristics.
method Derives exact sampling distribution by stochastic representation.
result High-dimensional asymptotic distribution of optimal portfolio weights is multivariate normal.
A new meta-learning framework that assigns weights to source tasks based on target samples.
problem Learning initialization for target tasks with limited labeled examples.
method A general framework that assigns weights to the loss of different source tasks, which can depend on the target samples. Provides upper bounds and develops a learning algorithm based on minimizing the error bound with respect to an empirical IPM.
result Empirically, the weighted meta-learning algorithm finds better initializations than uniformly-weighted meta-learning algorithms.
This paper examines probabilistic sampling weighted by uncertainty in active learning.
problem Improving efficiency and effectiveness in active learning.
method Probabilistic sampling weighted by uncertainty, implemented in a single-pass streaming fashion.
result Using probabilistic weighting often benefits active learning, especially with biased initial labeled points.
Proposes a new method to improve regression models with reweighted samples.
problem Improves regression models' performance under low sample sizes and covariate perturbations.
method Reparametrizes sample weights using a doubly non-negative matrix and solves the reweighted estimate efficiently.
result Adversarial reweighting strategy delivers promising results on various datasets.
A new algorithm improves credit scoring accuracy for imbalanced data.
problem Poor classification of minority class in credit scoring data sets.
method Weighted-Hybrid-Sampling-Boost (WHSBoost) algorithm with balanced data sampling.
result WHSBoost outperforms other methods in credit scoring accuracy.
New methods for estimating causal effects with limited overlap, using Stable Probability Weighting.
problem Estimating causal effects with limited overlap in multivalued treatments.
method Stable Probability Weighting (SPW) and Finite-Sample Stable Probability Weighting (FPW) methods.
result SPW and FPW provide practical solutions for estimating and inferring causal effects with limited overlap.
Proposes a method to transfer samples from source tasks to target tasks in RL.
problem Improving RL learning by selecting and weighting relevant samples from multiple tasks.
method Automatic estimation of importance weights for each source sample, applied to a batch RL algorithm.
result The proposed method achieves better learning performance and robustness to task differences.
A new method corrects weight values to improve treatment effect estimation.
problem Estimating heterogeneous treatment effects in high-dimensional data with sample selection bias.
method Differentiable Pareto-Smoothed Weighting (DPSW) framework.
result Our method outperforms existing methods in treatment effect estimation.
Generative models learn from biased data using weighted importance.
problem Learning from biased or related data distributions.
method Importance weighting to estimate loss with respect to target distribution.
result Effective in various settings with theoretical guarantees and good performance.
New method improves deep RL by combining emphatic weightings with replay data.
problem Improving sample efficiency and scaling model-free RL methods.
method Developed a multi-step emphatic weighting and time-reversed n-step TD learning algorithm. result The new approach reduces variance and provides convergence guarantees.
Extend CPS to non-exchangeable settings with observation-specific permutation weights
problem Calibrated predictive bands under distributional shifts
method Encoding distributional shifts through observation-specific permutation weights
result Shift-aware predictive systems remain valid
Differentially private weighted sampling improves privacy while maintaining utility.
problem Ensuring privacy in datasets with key-value pairs while preserving analytical utility.
method Private Weighted Sampling (PWS) that ensures element-level differential privacy.
result Significant performance gains in key reporting and estimation accuracy compared to prior methods.
SURF steers scalarization weights to uniformly traverse the Pareto front.
problem Non-uniform coverage of the Pareto front when using scalarization weights.
method Geometric analysis and CDF mapping to select weights for uniform coverage.
result SURF converges to uniform Pareto front coverage under provable conditions.
BONAS accelerates NAS while maintaining reliability.
problem Computational inefficiency in sample-based NAS.
method Bayesian Optimized Neural Architecture Search (BONAS) using weight-sharing.
result BONAS accelerates sample-based NAS significantly while maintaining reliability.
Importance sampling has become an important tool for the computation of tail-based risk measures. Since such quantities are often determined mainly by rare events standard Monte Carlo can be inefficient and importance sampling provides a way to speed up computations. This paper considers moderate deviations for the wei…
The study examines how weight sharing, equivariance, and locality affect the sample complexity of neural networks.
problem Understanding the impact of design choices on the generalization error of neural networks.
method Statistical learning theory applied to single hidden layer networks with weight sharing, equivariance, and locality.
result Lower and upper bounds for sample complexity are derived, showing that locality has benefits but comes with a trade-off.
Paper develops robust k-NN algorithm for few samples.
problem Learning robust classifier from limited samples.
method Distributionally robust formulation of weighted k-NN. result Robust classifier improves generalization with smaller Lipschitz norm.
WildWood improves Random Forest predictions using bootstrap out-of-bag samples.
problem Improving Random Forest predictions for supervised learning.
method Uses bootstrap out-of-bag samples to compute improved predictions by aggregating all possible subtrees with exponential weights.
result WildWood produces faster and more competitive predictions compared to other ensemble methods.
Diffusion models accurately recover mixture weights from generated samples despite score function insensitivity.
problem Score-based generative models often fail to learn correct relative mode amplitudes (mixture weights) from generated samples.
method Relate diffusion score matching (DSM) loss to mixture weight estimation error, define diffusion score sensitivity index (DSSI), and prove its governing role in mixture weight recovery.
result Generated samples can accurately recover mixture weights from the DSM loss, even when the target score is insensitive to mixture weights.
New algorithm for training GNNs with learned weights.
problem Optimal sampling for GNNs with learned weights is intractable.
method Formulated as an adversary bandit problem, optimizing exploration and exploitation.
result Asymptotically approaches optimal variance within a factor of 3.
Corrects bias in learned generative models using likelihood-free importance weighting.
problem Bias in learned generative models relative to true data distribution.
method Estimate likelihood ratio using a classifier, apply importance weighting.
result Consistently improves goodness-of-fit metrics for deep generative models.
New algorithms bound graph structure sampling and learning high-dimensional graphical models.
problem Learning high-dimensional graphical models and efficient graph structure sampling.
method Online learning framework with exponentially weighted average (EWA) or randomized weighted majority (RWM) forecasters using log loss function.
result New sample complexity bounds and efficient algorithms for learning Bayes nets, including trees and chordal skeletons.
WeSpeR speeds up non-linear shrinkage for high-dimensional weighted covariance.
problem Computing non-linear shrinkage formulas for high-dimensional weighted sample covariance.
method Derive extit{WeSpeR} algorithm using asymptotic sample spectrum properties.
result Significantly speeds up non-linear shrinkage in dimensions higher than 1000.
The paper explores when to prioritize easy or hard samples in learning tasks.
problem Determining the optimal order of learning easy or hard samples.
method Theoretical analyses and experiments were conducted to propose and validate four priority modes.
result Four priority modes (easy-first, hard-first, medium-first, two-ends-first) can be flexibly applied.
SWCCA identifies specific subsets of samples for better correlation analysis.
problem Identify specific subsets of samples contributing to correlation between two data matrices.
method Proposes SWCCA with weights to regularize different samples, solves using alternating iterative algorithm.
result Demonstrates effectiveness and superiority over related methods on synthetic and real-world data.
WiGS improves active learning for regression by dynamically selecting informative samples.
problem Reducing labeling costs in regression tasks.
method Formulated as a reinforcement learning problem, WiGS adapts the exploration-investigation balance.
result WiGS outperforms static methods in accuracy and labeling efficiency, especially in irregular data density.
A method to reduce memory usage in deep learning models by adding inducing weights.
problem Memory inefficiency in Bayesian neural networks and deep ensembles.
method Augmenting the weight matrix with inducing weights and using Matheron's conditional Gaussian sampling rule.
result Reduces parameter size to 24.3% of a single neural network while maintaining competitive performance.
Paper studies L2-consistency of nearest neighbor matching for biased samples.
problem Biased sampling and missing data in statistical problems.
method Nearest neighbor matching (NNM) to estimate population quantities from biased samples.
result NNM is L2-consistent in finite dimensions without smoothness and boundedness assumptions. Paper adapts multiplicative weights method to Gaussian graphical models.
problem Graphical model selection in Gaussian random fields.
method Adapted multiplicative weights method from Ising model to Gaussian model.
result Achieves sample complexity bound similar to existing methods.
Proposes a new sampling policy for ranking and selection problems.
problem Improving ranking and selection in adaptive sampling policies.
method Annealed entropic allocation, using soft-min weights and saddlepoint corrections.
result Consistently competitive performance in various settings.