This work uses statistical mechanics to explain AI learning.
problem Understanding the statistical principles behind AI learning.
method Starting from sample concentration behaviors, the study applies statistical mechanics principles to AI and machine learning.
result Exponential families and statistical quantities are key in AI and machine learning.
Investigates transfer learning in spatial statistics.
problem Applying transfer learning to spatial statistics.
method Simple MLP models for spatial data.
result Potential of transfer learning in spatial statistics.
Machine learning improves official statistics but needs rigorous validation.
problem Lack of methodological robustness in machine learning for official statistics.
method Total Machine Learning Error (TMLE) framework to validate ML models.
result TMLE addresses representativeness and measurement errors in ML models.
Improves statistical learning bounds with self-concordant losses.
problem Statistical prediction with nuisance components.
method Orthogonal statistical learning with self-concordant loss.
result Non-asymptotic bounds on excess risk improved by a dimension factor.
Contrastive learning simplifies statistical inference for complex models.
problem Computational intractability of likelihood functions for certain models.
method Contrastive learning as an alternative for parameter estimation and inference.
result Contrastive learning enables practical methods for diverse statistical problems.
New framework connects online learning to statistical learning for better generalization bounds.
problem Deriving generalization bounds for statistical learning algorithms.
method Constructing an online learning game and showing a connection to statistical learning.
result Established a connection between online and statistical learning, leading to new generalization bounds.
Targeted Learning uses robust statistics for reproducible research.
problem Improving reproducibility and rigor in statistical analyses.
method Principled standard for statistical estimation and inference, minimizing assumptions.
result Enhances reliability of statistical conclusions.
Paper develops robust policy evaluation for reinforcement learning with outlier and heavy-tailed rewards.
problem Outlier contamination and heavy-tailed rewards in reinforcement learning.
method Develops a fully online robust policy evaluation procedure and efficient statistical inference.
result Establishes the Bahadur-type representation of the estimator and develops an online inference procedure.
Discusses new probabilistic morphisms and geometric methods in machine and statistical learning.
problem Addressing challenges in statistical, machine, and manifold learning.
method Introduces category of probabilistic morphisms and geometric methods.
result New insights and applications in various learning fields.
Research aims to bridge statistical learning to causal models in AI.
problem Challenges in machine learning and AI related to causality.
method Transition from statistical learning to causal models.
result Progress in AI may require advances in causal modeling.
Learning representations of data is an important problem in statistics and machine learning. While the origin of learning representations can be traced back to factor analysis and multidimensional scaling in statistics, it has become a central theme in deep learning with important applications in computer vision and co…
Unified platform for statistical and machine learning in bioinformatics.
problem Workflow inefficiencies in using multiple tools for data analysis.
method Automated hyperparameter optimization, feature importance analysis, statistical tests.
result Accelerates biological discovery workflows with methodological soundness.
A textbook on statistical machine learning for astronomy.
problem Uncertainty quantification in astronomical data analysis.
method Bayesian inference and classical statistical methods.
result Unified framework connecting modern and traditional methods.
Study uses online bootstrap for RL inference, showing effectiveness.
problem Statistical inference for RL parameters in online settings.
method Online bootstrap method applied to TD and GTD algorithms in RL.
result Method is distributionally consistent for policy evaluation inference.
This chapter reviews statistical tools for reinforcement learning.
problem Applying RL algorithms in healthcare and ride-sharing platforms.
method Statistical inference tools for RL, including hypothesis testing and confidence interval construction.
result Highlighting the value of statistical inference in RL for both communities.
In the first half of 2018, the Federal Statistical Office of Germany (Destatis) carried out a "Proof of Concept Machine Learning" as part of its Digital Agenda. A major component of this was surveys on the use of machine learning methods in official statistics, which were conducted at selected national and internationa…
This paper examines risks and uncertainties of changing data sources in machine learning for official statistics.
problem Risks and uncertainties associated with changing data sources in machine learning for official statistics.
method An overview of risks, causes, and repercussions of changing data sources, with a checklist of measures.
result Maintaining integrity, reliability, consistency, and relevance in official statistics.
Solla discusses neural processing using statistical physics and Bayesian methods.
problem Understanding neural information processing through statistical physics.
method Bayesian inference, Gibbs description, Generalized Linear Models, dimensionality reduction.
result Connection between neural processing and statistical physics.
Survey on statistical learning theory for control, focusing on linear systems.
problem Applying machine learning techniques to control systems, especially linear ones.
method Adapting tools from modern high-dimensional statistics and learning theory.
result Recent advances in statistical learning theory for control, particularly for linear systems.
Lecture notes on reinforcement learning using statistical methods.
problem Addressing the exploration-exploitation dilemma in decision making.
method Frequentist and Bayesian approaches, function approximation, neural networks.
result Unified framework for decision making and estimation.
Responds to a statistical method for policy learning.
problem Improving policy learning efficiency.
method Optimal retargeting approach.
result Discussion and response to previous work.
This paper introduces a novel measure-theoretic theory for machine learning that does not require statistical assumptions. Based on this theory, a new regularization method in deep learning is derived and shown to outperform previous methods in CIFAR-10, CIFAR-100, and SVHN. Moreover, the proposed theory provides a the…
DPpack offers R tools for private data analysis and machine learning.
problem Ensuring privacy in statistical analysis and machine learning.
method Differential privacy mechanisms (Laplace, Gaussian, exponential).
result User-friendly implementation of privacy-preserving models.
A quantum circuit designed for efficient statistical model preparation and training.
problem Challenges in preparing and learning statistical models on quantum processors.
method Utilizes the maximum entropy principle to design a statistics-informed parameterized quantum circuit (SI-PQC).
result Improves trainability and interpretability for learning quantum states and classical model parameters.
Diffusion models learn simple statistics before complex ones, revealing a sample complexity exponent.
problem Understanding the learning dynamics of diffusion models.
method Empirical observations and theoretical analysis of diffusion models and denoisers.
result Diffusion models learn simple statistics (pair-wise correlations) at linear sample complexity, while higher-order statistics (e.g., fourth cumulant) require cubic sample complexity.
Paper develops efficient algorithms for robust distributed learning with statistical guarantees.
problem Limited communication power and adversarial node behaviors in distributed learning.
method Surrogate likelihood framework and median/trimmed mean operations.
result Provable robustness against Byzantine failures and optimal statistical rates.
Active inference framework improves U-statistic estimation efficiency.
problem Costly acquisition of labels for U-statistics. method Active inference framework with optimal sampling rule.
result Substantial gains in estimation efficiency over baseline methods.
Paper uses learned summary statistics for Bayesian inference with difficult likelihood functions.
problem Difficult to obtain exact likelihood function for observation data and simulation model.
method Simulation-based inference with learned summary statistics, using Cressie-Read discrepancy criterion.
result Effective inference performed over selected sample sets of observation data.
Efficiently learns Ising model parameters with limited statistics.
problem Learning Ising model parameters with limited sample configurations.
method Examines trade-offs between computation and observation, using Ising model as example.
result Reconstructs model parameters with statistics up to order O(γ) for ℓ1 width γ. This paper connects functional data analysis with machine learning techniques.
problem Lack of theoretical analysis for functional depths.
method Viewing functional depths as kernel mean embeddings in machine learning.
result Facilitates answers to open questions about functional depths.
Statistical learning theory provides bounds of the generalization gap, using in particular the Vapnik-Chervonenkis dimension and the Rademacher complexity. An alternative approach, mainly studied in the statistical physics literature, is the study of generalization in simple synthetic-data models. Here we discuss the c…
This paper connects Wasserstein distances to MMD norms for compressive statistical learning.
problem Comparing and controlling Wasserstein distances between probability distributions.
method Establishing conditions under which Wasserstein distances can be controlled by MMD norms.
result Introducing Wasserstein regularity for compressive statistical learning.
Deep learning's success is puzzling from a statistical perspective.
problem Deep learning's success is puzzling from a statistical perspective.
method Physics-informed investigation of deep learning features and surprises.
result Neural scaling laws and their interplay with inductive biases.
MegazordNet combines stats and ML for better financial time series forecasting.
problem Forecasting financial time series is challenging due to its chaotic nature.
method MegazordNet integrates statistical features with a deep learning model.
result MegazordNet outperforms single statistical and machine learning methods in S&P 500 stock price prediction.
Noise Sensitivity Exponent controls statistical-computational gaps in learning.
problem Understanding when learning is statistically possible yet computationally hard in high-dimensional statistics.
method Investigating statistical-computational gaps in single- and multi-index models using Noise Sensitivity Exponent.
result Noise Sensitivity Exponent governs statistical-computational gaps in high-dimensional learning.
Breiman's paper sparked debate on the future of statistics and machine learning.
problem The tension between traditional statistical modeling and model-free machine learning approaches.
method Discussion of the implications of machine learning's success and the need for new inferential approaches.
result The importance of understanding 'why' and 'if' questions in machine learning is now recognized.
This paper analyzes Local SGD for federated learning, achieving both statistical and communication efficiency.
problem Statistical estimation and inference in federated learning with decentralized data.
method Local SGD, a multi-round estimation procedure using intermittent communication.
result Local SGD achieves both statistical efficiency and communication efficiency.
Paper introduces statistical learning for point processes.
problem Statistical learning for point processes in general spaces.
method Combines bivariate innovations and point process cross-validation.
result Statistical learning approach outperforms state of the art.
Statistical learning theory provides the theoretical basis for many of today's machine learning algorithms. In this article we attempt to give a gentle, non-technical overview over the key ideas and insights of statistical learning theory. We target at a broad audience, not necessarily machine learning researchers. Thi…
Statistical arbitrage is a class of financial trading strategies using mean reversion models. The corresponding techniques rely on a number of assumptions which may not hold for general non-stationary stochastic processes. This paper presents an alternative technique for statistical arbitrage based on online learning w…
New statistical theory explains contrastive learning effectiveness.
problem Understanding why contrastive learning works well for representation extraction.
method Developed a new theoretical framework based on approximate sufficient statistics.
result Near-sufficient encoders derived from contrastive learning can be adapted for downstream tasks.
Extreme value theory enhances statistical learning extrapolation for rare events.
problem Challenges in traditional machine learning methods for extreme data.
method Asymptotic theory and statistical tools for tail behavior.
result Effective extrapolation methods for extreme quantiles and anomalies.
Paper tackles reinforcement learning with complex observations and simple latent dynamics.
problem Understanding reinforcement learning with complex observations and simple latent dynamics.
method Statistical and algorithmic analysis of reinforcement learning under general latent dynamics.
result Identifies latent pushforward coverability as a condition for statistical tractability.
Statistical methods remain relevant for ODE inverse problems, especially with sparse data.
problem The relevance of statistical methods in the era of deep learning for ODE inverse problems.
method Employed physics-informed neural networks (PINN) and manifold-constrained Gaussian process inference (MAGI) to compare statistical and deep learning approaches.
result Statistically principled methods outperform deep learning models in tasks like parameter inference and trajectory reconstruction.
A new test statistic measures discrepancy between conditional distributions.
problem Measuring the discrepancy between two conditional distributions.
method Proposes a Bregman matrix divergence-based statistic that avoids explicit distribution estimation.
result The new statistic inherits high-order statistics and demonstrates utility in multi-task learning, concept drift detection, and feature selection.
Generative models are reinterpreted in statistical terms, enabling better understanding and inference.
problem Insufficient interpretability of generative models in statistical terms.
method Flow matching and orthogonalization/cross-fitting in double/debiased machine learning.
result Generative models can be used to estimate nuisance components while maintaining inferential validity.
Paper develops algorithms for nonsmooth, nonconvex statistical learning problems.
problem Nonsmooth and nonconvex objectives in statistical learning.
method Bregman-surrogate algorithm framework, including local linear approximation, mirror descent, iterative thresholding, DC programming.
result Global convergence rates for nonconvex and nonsmooth objectives in high dimensions.
Neuroimaging research has predominantly drawn conclusions based on classical statistics, including null-hypothesis testing, t-tests, and ANOVA. Throughout recent years, statistical learning methods enjoy increasing popularity, including cross-validation, pattern classification, and sparsity-inducing regression. These t…