Efficiently transforms samples from various statistical models.
problem Approximately transforming samples from one statistical model to another without knowing the source model's parameters.
method Constructs computationally efficient procedures to reduce uniform, Erlang, and Laplace models to general target families.
result Establishes nonasymptotic reductions between canonical high-dimensional problems, such as mixtures of experts, phase retrieval, and signal denoising.
This report concerns the problem of dimensionality reduction through information geometric methods on statistical manifolds. While there has been considerable work recently presented regarding dimensionality reduction for the purposes of learning tasks such as classification, clustering, and visualization, these method…
Scalability of statistical estimators is of increasing importance in modern applications and dimension reduction is often used to extract relevant information from data. A variety of popular dimension reduction approaches can be framed as symmetric generalized eigendecomposition problems. In this paper we outline how t…
Adaptive method reduces data dimensions for scalable statistical analysis.
problem Scalability of statistical estimators in massive data.
method Adapts randomized low-rank approximation algorithms for efficient PCA and LMM.
result Improves both computational and statistical performance in large-scale genomic studies.
New method corrects missing data bias in dimension reduction.
problem Missing data complicates high-dimensional data analysis.
method Developed a bias-corrected Gram matrix for heterogeneous missingness.
result Proposed method improves dimension reduction techniques significantly.
U-statistics improve gradient estimation in importance-weighted variational inference.
problem High variance in gradient estimation for importance-weighted variational inference.
method Use U-statistics to average base gradient estimators on overlapping batches of size m, achieving lower variance.
result U-statistic variance reduction leads to modest to significant improvements in inference performance.
New insights link diverse statistical problems via secret leakage planted clique.
problem Statistical-computational gaps in inference problems.
method Secret leakage planted clique as a new hardness assumption for reductions.
result Establishes tight statistical-computational tradeoffs for various problems.
The paper provides statistical guarantees for generative models using dimension reduction.
problem Improving the quality of generative models without increasing dimensionality.
method Modeling generative devices as smooth transformations of a lower-dimensional space and using integral probability metrics.
result Established a risk bound showing the impact of dimension reduction on generative model error.
Solla discusses neural processing using statistical physics and Bayesian methods.
problem Understanding neural information processing through statistical physics.
method Bayesian inference, Gibbs description, Generalized Linear Models, dimensionality reduction.
result Connection between neural processing and statistical physics.
Reduces IB problem to a simpler, lower-dimensional problem.
problem Information bottleneck problem in high-dimensional spaces.
method Identifies sufficient statistic that factors conditional distribution, reducing IB to a lower-dimensional problem.
result Preserves full IB curve and optimal representations, making IB tractable.
Efficiently transforms Gaussian data to simulate various target distributions.
problem Generating observations from different target distributions given a single Gaussian observation.
method Designs computationally efficient procedures to approximate target distributions.
result Establishes reduction-based computational lower bounds for high-dimensional statistical models.
New algorithms reduce computational burden for principal support vector machines.
problem High computational cost of principal support vector machines for large datasets.
method Two distributed estimation algorithms for principal support vector machines.
result Statistical efficiency is maintained with distributed algorithms.
Experimental life sciences like biology or chemistry have seen in the recent decades an explosion of the data available from experiments. Laboratory instruments become more and more complex and report hundreds or thousands measurements for a single experiment and therefore the statistical methods face challenging tasks…
This paper studies phase transitions in randomized dimension reduction.
problem How to ensure randomized dimension reduction succeeds with high probability.
method Natural family of randomized dimension reduction maps and a large class of data sets.
result There is a phase transition in the success probability of the dimension reduction map as the embedding dimension increases.
Paper tackles reinforcement learning with complex observations and simple latent dynamics.
problem Understanding reinforcement learning with complex observations and simple latent dynamics.
method Statistical and algorithmic analysis of reinforcement learning under general latent dynamics.
result Identifies latent pushforward coverability as a condition for statistical tractability.
Modified relative universality for unbiasedness and consistency in dimension reduction.
problem Gap in proof of unbiasedness and Fisher consistency in relative universality.
method Modified definition of relative universality using ǫ-measurability.
result Established unbiasedness and Fisher consistency rigorously.
Improved privacy-preserving statistical estimates with customizable noise reduction.
problem Balancing privacy and accuracy in statistical estimation.
method Introducing the Brownian mechanism, which adds Gaussian noise to a sequence of estimates, gradually reducing it based on the practitioner's needs.
result The Brownian mechanism produces more accurate estimates while maintaining strong privacy guarantees, outperforming existing methods.
Neural networks simplify SDR in regression tasks.
problem Sufficient dimension reduction in regression problems.
method Applying neural networks with rank regularization to estimate the central mean subspace.
result Neural networks effectively perform SDR, consistent with theoretical estimations.
Active learning method for ABC statistics selection reduces expert work and improves posterior estimates.
problem Handling intractable likelihood functions in models with domain knowledge.
method Active learning method for selecting summary statistics in ABC.
result Better posterior estimates than existing methods, especially with limited simulation budget.
Unified model for reducing dimensions and clustering high-dimensional data.
problem High-dimensional data clustering and dimensionality reduction.
method Hierarchical mixtures of Gaussians (HMoGs) with closed-form likelihood and inference.
result Efficiently models hundreds of latent dimensions, improving clustering performance.
This work optimizes statistical inference with neural networks for high-energy physics data.
problem Optimal dimensionality reduction with minimal loss of information in the presence of systematic uncertainties.
method Neural network optimization based on binned Poisson likelihoods with nuisance parameters.
result Estimates of parameters of interest close to optimal.
Introduces a probabilistic framework for dimension reduction methods.
problem Lack of clear probabilistic foundations for popular DR methods.
method A unifying statistical framework based on the coupling of hidden graphs using cross entropy.
result Existing DR methods suffer from a statistical deficiency that affects performance.
Reduced model helps estimate parameters from partially observed multiscale data.
problem Estimating parameters from high-dimensional partially observed multiscale data.
method Established convergence of filter from high-dimensional to reduced dimension model.
result Statistical estimation of parameters is possible in lower dimensions.
ML reduces high-dimensional data to reveal its underlying structure.
problem Handling large, high-dimensional data sets.
method Non-linear dimension reduction techniques.
result Reveals the geometric shape of high-dimensional data.
Estimates proportions of LLM-generated text in mixed documents.
problem Estimating the proportion of text generated by a pre-specified LLM in mixed documents.
method Developed estimators for two observation regimes: full observation and pivotal reduction, and established sample complexity bounds.
result Full observation estimators require fewer samples than pivotal reduction estimators.
Extends boosting to multiclass online agnostic classification.
problem Online multiclass classification with weak learners.
method Reduces multiclass online agnostic boosting to online convex optimization.
result First boosting algorithm for online agnostic multiclass classification.
Study on estimating Gumbel--Max watermark proportions in edited documents.
problem Estimating the proportion of a document generated from a watermarked LLM.
method Comparison of full observation and pivotal reduction observation regimes; development of estimators and information-theoretic lower bounds.
result Full observation yields a substantially smaller sample complexity compared to pivotal reduction.
Data balancing reduces variance in machine learning models.
problem Reduction of variance in machine learning models.
method Non-asymptotic statistical bound and eigenvalue decay of Markov operators.
result Data balancing across modalities and sources reduces variance.
New statistic κ-profile helps monitor weather, soundscapes, and dynamical systems.
problem Monitoring intrinsic dimensionality of large data sets.
method Optimization problem to find κ-profile, which is the norm of the shortest projected secant. result The κ-profile provides a useful statistic for understanding and monitoring large data sets. New algorithms improve privacy in statistical estimation by making them robust.
problem Improving privacy in statistical estimation methods.
method Black-box reduction from privacy to robustness, using Sum-of-Squares method.
result Design of polynomial-time private estimators with optimal tradeoffs among sample complexity, accuracy, and privacy.
New methods improve feature extraction and representation quality in supervised and unsupervised DR.
problem Statistical dependence, data diversity, contrast, and interpretability in conventional DR methods.
method Combines linear and nonlinear formulations for three new independence criteria.
result Significant improvements in contrast, accuracy, and interpretability over baselines.
SQFA learns features maximizing Fisher-Rao distance for better classification.
problem Improving classification accuracy through feature learning.
method SQFA learns linear features maximizing Fisher-Rao distance between class-conditional distributions.
result SQFA-H features achieve the best classification accuracy.
Algorithm finds function contours using multiple approximations.
problem Locating contours of expensive-to-evaluate functions.
method Uses multiple biased and noisy approximations to locate contours efficiently by maximizing entropy reduction.
result Maximizes reduction of contour entropy per unit cost.
The paper examines MSU measure and its effectiveness in feature selection.
problem Improving feature selection in multivariate data.
method Statistical simulation of MSU under different feature conditions.
result A condition is identified that maintains MSU quality across various feature combinations.
Directly estimates QMI derivative for robust supervised dimension reduction.
problem Estimating QMI derivative accurately for robust dimension reduction.
method Direct estimation of QMI derivative without estimating QMI itself.
result Direct estimation of QMI derivative is more accurate than estimated QMI derivative.
We present a very fast algorithm for general matrix factorization of a data matrix for use in the statistical analysis of high-dimensional data via latent factors. Such data are prevalent across many application areas and generate an ever-increasing demand for methods of dimension reduction in order to undertake the st…
We introduce a novel data-driven order reduction method for nonlinear control systems, drawing on recent progress in machine learning and statistical dimensionality reduction. The method rests on the assumption that the nonlinear system behaves linearly when lifted into a high (or infinite) dimensional feature space wh…
Paper develops a new method for reducing dimensions in longitudinal data.
problem Challenges in predicting and inferring from high-dimensional longitudinal data.
method Proposes supervised Kernel PCA (sklPCA) for longitudinal data. result Demonstrates superior model accuracy compared to existing methods.
Bayesian method reduces dimensions for small datasets.
problem Dimension reduction for small datasets with limited data.
method Bayesian approach using Gaussian process regression.
result Effective for small datasets, especially.
Proposes methods to combine local probabilistic models into a single efficient model.
problem Combining local probabilistic models into a single efficient model in distributed learning.
method Bootstrap samples from local models, then learn a joint model with variance reduction methods.
result Proposed methods correct bootstrap noise and improve performance.
Survey of SDR methods for high-dimensional regression and embedding.
problem Reducing dimensionality in high-dimensional data.
method Involves both statistical and machine learning approaches, covering inverse and forward regression methods.
result Supervised Kernel Dimension Reduction is equivalent to supervised PCA.
The paper shows how shared random seeds can reduce variance in machine learning evaluations.
problem The statistical structure of comparative evaluation under shared random seeds is not well understood.
method An extended learning-based multi-agent economic simulator was used to demonstrate the effects of shared random seeds on variance reduction.
result Pairing seeds can reduce variance in machine learning evaluations, especially when outcomes are positively correlated at the seed level.
We show how to reduce the process of predicting general order statistics (and the median in particular) to solving classification. The accompanying theoretical statement shows that the regret of the classifier bounds the regret of the quantile regression under a quantile loss. We also test this reduction empirically ag…
New auction design uses statistical learning to reduce costs and improve fairness.
problem Designing efficient multi-item auctions with reduced implementation costs and fairness.
method Nonparametric density estimation for credible intervals, two new strategies.
result Strategies consistently outperform alternative methods in revenue maximization and cost reduction.
Boosts weak online learners to strong ones with sublinear regret.
problem Online learning agnostic setting without strong guarantees.
method Reduction to online convex optimization, boosting via marginally-better-than-trivial regret guarantees.
result First agnostic online boosting algorithm with sublinear regret.
Spectral methods reduce the complexity of Markov processes.
problem Modeling and simplifying state-transition systems.
method Spectral decomposition and state aggregation.
result Developed methods to estimate low-rank Markov models.
A new method using energy distance for ensemble and scenario reduction.
problem Solving complex dynamic and stochastic programs, especially in energy systems.
method Proposes a new method based on energy distance for ensemble and scenario reduction.
result Reduced scenario sets exhibit better statistical properties for energy distance than Wasserstein distance.
We reduce variance in monetization metrics for ranking experiments.
problem Heavy-tailed monetization metrics lead to unreliable conclusions in A/B experiments.
method Post-stratification combined with CUPED.
result Significant reduction in variance and improved decision stability.