The Lasso performs well in ultra-sparse linear models with finite support size.
problem Performance analysis of Lasso in ultra-sparse linear models.
method Novel application of replica method from statistical physics, rigorous analysis of average case performance.
result Average performance of Lasso assessed without scaling assumptions, offering sample complexity bounds.
The paper tackles matrix completion in ultra-sparse sampling, improving imputation accuracy.
problem Matrix completion in ultra-sparse sampling, where each row has only a few entries.
method Estimate row span of matrix or averaged second-moment matrix, normalize and impute missing entries.
result Gradient descent method normalizes and imputes missing entries, achieving low variance and unbiased estimation.
TRUST improves tree models' accuracy while maintaining interpretability.
problem Piecewise-constant regression trees lack in predictive accuracy compared to black-box models.
method Combines Random Forest accuracy with interpretability of shallow trees and sparsity of linear models, using LLMs for explanations.
result TRUST outperforms other interpretable models in predictive accuracy and matches Random Forest's accuracy.
GRASPEL learns large graphs from data efficiently.
problem Learning meaningful graphs from data for various applications.
method Highly scalable spectral approach using graph Laplacians and coarsening techniques.
result Ultra-sparse graphs with improved efficiency and accuracy in spectral clustering and t-SNE.
Sparse dictionary learning (SDL) has become a popular method for adaptively identifying parsimonious representations of a dataset, a fundamental problem in machine learning and signal processing. While most work on SDL assumes a training dataset of independent and identically distributed samples, a variant known as con…
In semi-supervised learning for classification, it is assumed that every ground truth class of data is present in the small labelled dataset. Many real-world sparsely-labelled datasets are plausibly not of this type. It could easily be the case that some classes of data are found only in the unlabelled dataset -- perha…
Develops a new point process model for detecting neural spike sequences.
problem Detecting sparse sequences of neural spikes in high-dimensional spike trains.
method A point process model that represents sequence occurrences as marked events in continuous time, with learnable time warping parameters.
result Demonstrates improved detection and modeling of neural spike sequences.
STR reparameterizes DNN weights with soft thresholds for better sparsity and accuracy.
problem Improving sparsity in DNNs for better accuracy and lower inference cost.
method Soft Threshold Reparameterization (STR) using the soft-threshold operator on DNN weights.
result STR achieves state-of-the-art accuracy and reduces FLOPs by up to 50%.
The eigendeomposition of nearest-neighbor (NN) graph Laplacian matrices is the main computational bottleneck in spectral clustering. In this work, we introduce a highly-scalable, spectrum-preserving graph sparsification algorithm that enables to build ultra-sparse NN (u-NN) graphs with guaranteed preservation of the or…
SOLAR improves search efficiency and accuracy with sparse, orthogonal embeddings.
problem Bottleneck of indexing large dense vectors and NNS for query efficiency and accuracy.
method Proposes SOLAR embeddings: sparse, orthogonal, learned, and random vectors across multiple GPUs.
result Successfully trains 500K dimensional SOLAR embeddings for 1.6M books and multi-label classification.
Enhanced Sampling Scheme improves masked generative modeling.
problem Limitations of existing sampling schemes in masked non-autoregressive generative modeling.
method ESS consists of three stages: Naive Iterative Decoding, Critical Reverse Sampling, and Critical Resampling.
result ESS achieves significant performance gains in unconditional and class-conditional sampling.
PRS improves rejection sampling by learning better proposals.
problem High rejection rate in traditional rejection sampling.
method PRS uses a kernel estimator to learn better sampling proposals.
result PRS guarantees a low number of accepted samples.
This paper reviews various sampling methods from statistics and machine learning.
problem Addressing sampling methods in statistics and machine learning.
method Explains and reviews simple random sampling, bootstrapping, stratified sampling, cluster sampling, multistage sampling, network sampling, snowball sampling, and sampling from cumulative distribution function.
result Summarizes characteristics, pros, and cons of different sampling methods.
RISA improves VFL by using imputed samples with low uncertainty.
problem Limited overlapping samples constrain VFL performance.
method Imputing non-overlapping samples and using evidence theory to select reliable imputed samples.
result Significant performance gains achieved, especially with limited overlapping samples.
Improved privacy-preserving methods for estimating multiple samples from distributions.
problem Estimating multiple samples from distributions while maintaining privacy.
method Developed new multi-sampling techniques for differentially private data estimation.
result Achieved significant reduction in sample complexity for multi-sampling from finite domains and Gaussian distributions.
Paper introduces a new sampling method combining Consistency Models with importance sampling.
problem Inherent errors in samples and high NFEs for high-quality samples in Boltzmann distributions.
method Combines Consistency Models with importance sampling to produce unbiased samples with minimal NFEs.
result Produces unbiased samples using only 6-25 NFEs, comparable to 100 NFEs for DDPMs.
Neural network accuracy improves with denser training samples.
problem Improving neural network accuracy on unseen test samples.
method Bounding empirical training error smoothed across activation regions and using it to discard high-risk test samples.
result Discarding high-risk test samples based on error bounds improves prediction accuracy by up to 20%.
Wedge Sampling improves tensor completion with nearly-linear sample complexity.
problem Efficiently completing low-rank tensors from a subset of entries.
method Non-adaptive wedge sampling to promote structured connections in tensor completion.
result Polynomial-time algorithms achieve weak and exact recovery with nearly linear sample complexity.
Algorithm samples constrained stochastic differential equations.
problem Sampling stochastic differential equations with complex constraints.
method Pathspace Metropolis-adjusted manifold sampling.
result Demonstrated effectiveness in various constrained conditions.
In modern data analysis, random sampling is an efficient and widely-used strategy to overcome the computational difficulties brought by large sample size. In previous studies, researchers conducted random sampling which is according to the input data but independent on the response variable, however the response variab…
Generative models map simple samples to complex target samples.
problem Improving Monte-Carlo sampling techniques.
method Variational learning of dynamical maps between base and target measures.
result Improved sampling efficiency through feedback loops.
Sampling is a fundamental problem in computer science and statistics. However, for a given task and stream, it is often not possible to choose good sampling probabilities in advance. We derive a general framework for adaptively changing the sampling probabilities via a collection of thresholds.In general, adaptive samp…
New sampling algorithm for non-smooth potentials.
problem Sampling from non-smooth potentials.
method Proximal algorithm based on rejection sampling.
result Achieves better complexity than existing methods.
Efficiently samples sequences without replacement for machine learning models.
problem Generating diverse outputs from sequential models without duplicates.
method Incremental sampling procedure for randomized programs, including neural models.
result Efficacy and flexibility of incremental sampling for large output spaces.
Optimizes biomolecular simulations by ranking adaptive sampling policies.
problem Efficiently sampling biomolecular systems to capture complex dynamical behaviors.
method Metric-driven ranking of adaptive sampling policies to identify the optimal policy for each round.
result Different adaptive sampling policies lead to faster convergence and improved sampling performance.
Ensemble sampling approximates Thompson sampling for complex models.
problem Computational intractability of exact posterior distributions.
method Thompson sampling approximation with information-theoretic concepts.
result Established a first rigorous regret bound for ensemble sampling.
Optimizes sample and round complexity in adaptive sampling from multiple distributions.
problem Adaptive sampling from multiple distributions with limited rounds and samples.
method Introduces OODS framework and analyzes tradeoffs between sample and round complexity.
result Achieves near-optimal sample complexity and sub-polynomial round complexity.
Meta-learners improve causal effect estimation in small samples.
problem Estimating causal effects using machine learning methods.
method Sample-splitting and cross-fitting to reduce overfitting bias.
result Meta-learners' performance depends on sample size and estimation procedure.
REP-GAN improves GANs by reparameterizing proposals for better sample quality and efficiency.
problem Poor sample efficiency in GANs due to independent proposal sampling.
method REParameterizing Markov chains into the latent space of the generator to create dependent proposals.
result Empirically shows significant improvement in sample efficiency and quality.
Unified framework for model-based RL with sample complexity guarantees.
problem Designing efficient posterior sampling methods for model-based RL.
method Optimistic posterior sampling, Hellinger distance reduction, data likelihood measurement.
result Unified algorithms with state-of-the-art sample complexity guarantees.
Proposes a neural network method to combine nonprobability and probability survey samples.
problem Combining nonprobability and probability survey samples for accurate population mean estimation.
method Uses a deep neural network to estimate sampling scores from nonprobability samples and combines them with probability sample information.
result Proposed estimators improve robustness to parametric propensity-score misspecification, especially for nonlinear selection mechanisms.
Dynamic Influence Tracker measures changing sample importance during model training.
problem Static influence measurements during training overlook how sample importance varies over time.
method Dynamic Influence Tracker (DIT) captures time-varying sample influence across arbitrary time windows.
result DIT reveals distinct learning phases with shifting priorities and detects corrupted samples more efficiently.
New sampling methods improve classifier performance estimation.
problem Efficiently selecting data points to estimate classifier performance.
method Introduced and compared Importance Sampling and Poisson Sampling.
result Poisson Sampling outperforms Importance Sampling.
Optimized sampling scheme for compressed sensing combining randomness and determinism.
problem Improving compressed sensing performance with deterministic sampling.
method Optimized sampling scheme combining random and deterministic selection of rows.
result Measurable improvements in image compressed sensing for generative and sparse priors.
Predictive sampling improves on Thompson sampling for non-stationary bandit environments.
problem Thompson sampling fails in non-stationary bandit environments.
method Proposes predictive sampling, which deprioritizes actions based on information loss rate.
result Predictive sampling outperforms Thompson sampling in all tested non-stationary environments.
Algorithm samples from Bingham distribution efficiently.
problem Sampling from the Bingham distribution on a sphere.
method Rejection sampling with polynomial approximation.
result Exact samples from Bingham distribution in polynomial time.
Self-taught learning is a technique that uses a large number of unlabeled data as source samples to improve the task performance on target samples. Compared with other transfer learning techniques, self-taught learning can be applied to a broader set of scenarios due to the loose restrictions on the source data. Howeve…
A new algorithm for sampling from complex distributions.
problem Sampling from high-dimensional multivariate probability densities.
method Combines kernel herding and Gibbs sampling for deterministic sampling.
result Significantly lower computation time compared to kernel herding.
Efficiently tests two distributions with few label queries.
problem Two-sample test with limited label information.
method Three-stage framework: classifier training, bimodal query, FR test.
result Significantly reduces Type II error compared to uniform querying.
A new graph-based sampling method improves classification of imbalanced COVID-19 datasets.
problem Classification of imbalanced COVID-19 datasets.
method Modified Random Walk Under-sampling based on graphs.
result The method increases the boundary of the minority class without raising outliers.
BIS uses bandits to efficiently sample from expensive-to-evaluate densities.
problem Sampling from computationally expensive target densities.
method Sequential selection through multi-armed bandits, optimizing sample set directly.
result BIS achieves accurate sampling with fewer evaluations than adaptive methods.
Enhances diffusion-based sampling for molecular systems.
problem Inefficiency and thermodynamic mode miss in diffusion-based samplers for molecular systems.
method Introduces a sequential bias along collective variables (CVs) to encourage exploration and increase temperature in the projected space.
result Improves efficiency, mode discovery, and free energy estimation; first to demonstrate reactive sampling.
Gradient-guided nested sampling improves posterior inference efficiency.
problem Efficiently sampling from complex posterior distributions.
method Gradient-guided nested sampling combining differentiable programming, Hamiltonian slice sampling, clustering, mode separation, dynamic nested sampling, and parallelization.
result Significantly faster mode discovery and more accurate partition function estimates.
Optimizes Thompson sampling policies using policy gradient methods.
problem Improving Thompson sampling in bandit problems.
method Applies policy gradient algorithms to optimize Thompson sampling policies.
result Direct policy search on Thompson sampling improves performance.
Thompson Sampling shows polynomial regret for combinatorial semi-bandits with subgaussian rewards.
problem Finding optimal solutions in combinatorial semi-bandits with suboptimal sampling.
method Proposes Thompson Sampling with polynomial regret for linear combinatorial semi-bandits.
result Demonstrates 'mismatched sampling paradox' where knowing distributions can lead to worse performance.
DAC enhances exploration in reinforcement learning with entropy regularization.
problem Improving exploration efficiency in reinforcement learning.
method Sample-aware entropy regularization using replay buffer action distributions.
result DAC significantly outperforms existing algorithms in reinforcement learning tasks.
Introduces t-CCS for flexible tensor sampling.
problem Lack of flexibility in tensor sampling methods.
method Tensor Cross-Concentrated Sampling (t-CCS).
result Effective tensor recovery from t-CCS samples.
Machine learning models, especially based on deep architectures are used in everyday applications ranging from self driving cars to medical diagnostics. It has been shown that such models are dangerously susceptible to adversarial samples, indistinguishable from real samples to human eye, adversarial samples lead to in…