MASS Learning trains models to use minimal sufficient statistics, improving performance and uncertainty quantification.
problem Training deep networks to use minimal sufficient statistics for better performance and uncertainty quantification.
method MASS Learning trains models to produce minimal sufficient statistics with respect to a class of functions, using Conserved Differential Information (CDI).
result Deep networks trained with MASS Learning achieve competitive performance on supervised learning and uncertainty quantification benchmarks.
We find minimal sufficient statistics for two variables that preserve mutual information.
problem Preserving mutual information when variables have high dimensionality.
method Developed a method to replace each variable with a lower-dimensional representation while preserving mutual information.
result Minimal sufficient statistics can be used to replace both variables simultaneously, preserving mutual information.
New principle in online learning: Regret can be expressed using sufficient statistics and a Burkholder function.
problem Achieving optimal online learning performance with limited memory.
method Introducing a Burkholder function that depends only on sufficient statistics, not the entire data sequence.
result Developed novel online strategies for matrix prediction and parameter-free supervised learning.
New statistical theory explains contrastive learning effectiveness.
problem Understanding why contrastive learning works well for representation extraction.
method Developed a new theoretical framework based on approximate sufficient statistics.
result Near-sufficient encoders derived from contrastive learning can be adapted for downstream tasks.
Boosting improves data fitting while maintaining fairness guarantees.
problem Ensuring fairness in data preprocessing.
method Boosting algorithm to learn sufficient statistics of exponential families.
result The learned distribution maintains fairness guarantees while fitting the data better.
A new principle and method improve out-of-distribution detection in generative models.
problem Out-of-distribution detection in deep generative models often fails due to poor likelihood estimates.
method Introducing the Likelihood Path (LPath) principle and new theoretical tools for OOD detection.
result Non-asymptotic provable OOD detection guarantees for variational autoencoders (VAEs).
Paper develops a theory explaining contrastive pre-training for multimodal AI.
problem Limited theoretical understanding of contrastive pre-training for multi-modal AI.
method Introduces approximate sufficient statistics and Joint Generative Hierarchical Model.
result Near-minimizers of contrastive loss are approximately sufficient, enabling diverse downstream tasks.
New statistics are introduced that maintain the Fisher metric structure closely, akin to sufficient statistics.
problem Maintaining the Fisher metric structure in statistical models.
method Characterizing statistics that maintain the Fisher metric structure bi-Lipschitz equivalently.
result Characterized statistics that preserve the Fisher metric structure closely.
Analyzes alternating minimization for nonconvex sets in high-dimensional statistics.
problem Optimizing loss functions over nonconvex sets in high-dimensional statistics.
method Local concavity coefficients for nonconvex sets, alternating minimization, inexact algorithms.
result Reveals distinctions between alternating and non-alternating methods, provides convergence conditions.
APG samplers use neural suff stats to improve deep model inference.
problem Efficient inference in deep generative models.
method Amortized population Gibbs, neural suff stats, KL divergence minimization.
result Significant improvement in inference accuracy.
Paper examines the relationship between maximizing and minimizing expected return in portfolio optimization.
problem Investment risk and return optimization in portfolio problems.
method Lagrange undetermined multiplier method and replica analysis.
result Derived mean square error and correlation coefficient of optimal portfolios as functions of risk tolerance.
Paper introduces data-dependent SSP for private linear and logistic regression.
problem Private linear and logistic regression with better performance.
method Data-dependent sufficient statistic perturbation (SSP) for linear and logistic regression.
result Data-dependent SSP outperforms state-of-the-art methods for linear and logistic regression.
New method uses sufficient statistics to infer causal relationships from observational data.
problem Inferring causal relationships from observational data with hidden variables.
method Information Bottleneck method applied to find functional sufficient statistics.
result New causal rules not obtainable from standard methods, validated on simulated and real data.
We clarify measurability assumptions in the agnostic PAC learning theorem.
problem Measurability assumptions in the Fundamental Theorem of Statistical Learning.
method Measure-theoretic scrutiny of existing proofs to extract minimal assumptions.
result Sound statement and detailed proof of the Fundamental Theorem in the agnostic setting.
FSRL balances fairness and sufficiency in learning representations.
problem Ensuring unbiased predictions and decisions in machine learning.
method A convex combination of sufficiency and fairness objectives using distance covariance.
result FSRL achieves a superior trade-off between fairness and accuracy.
New method for separating mixed signals with nonlinear functions.
problem Recovering source signals from nonlinear mixtures.
method Optimisation-based function approximation to minimize mutual statistical dependence.
result The method can recover source signals from nonlinear mixtures under certain conditions.
The article analyzes high-dimensional classification using empirical risk minimization with precise error predictions.
problem Classifying high-dimensional data with Gaussian mixture models.
method Theoretical analysis of ridge-regularized and unregularized empirical risk minimization for high-dimensional Gaussian mixture separation.
result The square loss is optimal for high-dimensional classification in both ridge-regularized and unregularized cases.
Study feature representations induced by dependence between variables.
problem Learning feature representations from dependent random variables.
method Characterized sufficient and necessary conditions for dependence-induced representations, and provided a family of loss functions.
result Features learned from the family of loss functions can be expressed as the composition of a loss-dependent function and the maximal correlation function.
Information geometry provides a geometric approach to families of statistical models. The key geometric structures are the Fisher quadratic form and the Amari-Chentsov tensor. In statistics, the notion of sufficient statistic expresses the criterion for passing from one model to another without loss of information. Thi…
A new framework for efficient large-scale learning using sketching of moments.
problem Efficiently learning from large datasets with limited computational resources.
method Compressing the training data into a low-dimensional sketch and solving a nonlinear least squares problem.
result Sufficient sketch sizes to control the generalization error of the procedure.
New insights into neural network training show some interpolating methods can generalize well, while others fail catastrophically.
problem Understanding why neural networks trained to interpolate can still generalize well or fail catastrophically.
method Analyzing empirical risk minimization (ERM) over large hypotheses classes, focusing on interpolating methods.
result Some interpolating ERM-like methods for large hypotheses classes provide good statistical guarantees, while others fail catastrophically.
New framework for predicting decisions that influence their own outcomes.
problem Predictions that affect the outcomes they predict, leading to undesirable distribution shift.
method Risk minimization framework combining statistics, game theory, and causality.
result Necessary and sufficient conditions for retraining to converge to a performatively stable point of minimal loss.
Proposes new fairness definitions for classification tasks combining statistical and individual fairness.
problem Combining statistical and individual fairness in classification tasks.
method Designs an oracle-efficient algorithm for fair empirical risk minimization.
result The ERM solution generalizes to new individuals and tasks.
Reduces IB problem to a simpler, lower-dimensional problem.
problem Information bottleneck problem in high-dimensional spaces.
method Identifies sufficient statistic that factors conditional distribution, reducing IB to a lower-dimensional problem.
result Preserves full IB curve and optimal representations, making IB tractable.
We address some theoretical guarantees for Schatten-p quasi-norm minimization (p∈(0,1]) in recovering low-rank matrices from compressed linear measurements. Firstly, using null space properties of the measurement operator, we provide a sufficient condition for exact recovery of low-rank matrices. This condition…
Differentially private learning of graphs improves on naive methods.
problem Learning discrete, undirected graphical models while preserving privacy.
method Developed a principled approach using collective graphical models within an expectation-maximization framework.
result The new method learns better models than competing approaches.
Improves A/B testing by detecting minor treatment effects.
problem Challenges in identifying small average treatment effects.
method Maximum probability-driven two-armed bandit (TAB) process with weighted mean volatility statistic.
result Significant improvement in A/B testing with reduced experimental costs.
Paper refines null space conditions for nuclear norm minimization in low-rank matrix recovery.
problem Establishing conditions for successful nuclear norm minimization recovery of low-rank matrices.
method Developed new null space conditions for nuclear norm minimization, proving their necessity and sufficiency.
result Weak null space condition is sufficient but not necessary for nuclear norm minimization recovery, providing a new necessary and sufficient condition.
Algorithm minimizes regret and converges to equilibria in Markov games.
problem Regret minimization and convergence to equilibria in general-sum Markov games under adversarial opponents.
method Decentralized algorithm that uses policy optimization and controls path length to achieve sublinear regret.
result Sublinear regret guarantees for convergence to correlated equilibrium in Markov games.
Affine and conformal submersions with horizontal distribution are studied in statistical manifolds.
problem Characterizing submersions and geodesics in statistical manifolds.
method Introducing conformal submersions with horizontal distribution and proving conditions for statistical manifold properties.
result Necessary and sufficient conditions for submersions and geodesics in statistical manifolds.
Counterfactual learning improves SMT by smoothing out deterministic logs.
problem Deterministic logging limits exploration in SMT systems.
method Additive and multiplicative control variates to smooth out deterministic components.
result Improvements of up to 2 BLEU points achieved through counterfactual learning.
FedLog reduces communication in federated learning by sharing data summaries.
problem Significant communication overhead in federated learning with large model parameters.
method Shares minimal sufficient statistics via Bayesian inference and differential privacy.
result High learning accuracy with low communication overhead.
A novel MM algorithm optimizes DCOV for SDR and SVS.
problem Dimension reduction and variable selection in nonparametric settings.
method Formulated as a DC program, MM algorithm solves quadratic subproblems on the Stiefel manifold.
result Improves computation efficiency and robustness across various settings.
Analyzes empirical risk minimization in finance, showing effectiveness and generalization issues.
problem Analyzing empirical risk minimization in finance for optimal hedging and investment decisions.
method Classical statistical machine learning techniques and non-asymptotic estimates based on Rademacher complexity.
result Over-training leads to anticipative decisions, but non-asymptotic estimates show convergence for large training sets.
Efficient method for tensor linear form inference with noisy incomplete data.
problem Statistical inference of tensor linear forms with incomplete and noisy observations.
method Initial estimate + debiasing + one-step power iteration.
result Optimal uncertainty quantification and statistical-to-computational gaps examined.
The study finds conditions for area-minimizing cones over submanifolds.
problem Conditions for area-minimizing cones over submanifolds.
method General configuration results for area-minimizing cones.
result Cone over the minimal product of submanifolds and spheres are area-minimizing.
This paper introduces SS-MAMP to address convergence issues in AMP algorithms.
problem Convergence issues in AMP algorithms for signal reconstruction.
method Proposes SS-MAMP algorithm framework for right-unitarily invariant sensing matrices and Lipschitz-continuous local processors.
result Covariance matrices of SS-MAMP are L-banded and convergent, ensuring optimal convergence.
A method for converting NIW parameters for better estimation.
problem Estimating parameters of multivariate normal distribution.
method Convergent procedure for converting mean parameters to natural parameters in NIW family.
result Maximum likelihood estimation of natural parameters from observed statistics.
Deep morphing detects bone structures in low-quality X-ray images.
problem Detecting bone structures in low-quality fluoroscopic X-ray images.
method Two-stage deep learning approach using deep segmentation networks and statistical shape models.
result Efficiently detects bone structures in low-quality X-ray images.
Efficiently learns Ising model parameters with limited statistics.
problem Learning Ising model parameters with limited sample configurations.
method Examines trade-offs between computation and observation, using Ising model as example.
result Reconstructs model parameters with statistics up to order O(γ) for ℓ1 width γ. Study proves conditions for area-minimizing surfaces in a specific space.
problem Existence and non-existence of area-minimizing surfaces in E(−1,τ). method Analyzes sufficient conditions for curves to be the asymptotic boundary of area-minimizing surfaces.
result Presented sufficient conditions for a curve to admit a solution to the asymptotic Plateau problem.
Deterministic GD can behave stochastically in large learning rates for multiscale functions.
problem Understanding deterministic GD's stochastic behavior in large learning rates for multiscale objectives.
method Established a sufficient condition for deterministic GD to converge to a rescaled Gibbs distribution in large learning rates for multiscale functions.
result Deterministic GD can converge to a statistical distribution in large learning rates for multiscale functions.
This paper provides a method for noise-calibrated inference from DP synthetic data.
problem Inference from DP synthetic data is often miscalibrated and lacks principled uncertainty quantification.
method Release DP sufficient statistics, perform noise-calibrated likelihood-based inference, and optional synthetic data generation.
result Asymptotic normality and valid confidence intervals for the plug-in DP MLE.
We address the question of detecting minimal virtual diagrams with respect to the number of virtual crossings. This problem is closely connected to the problem of detecting the minimal number of additional intersection points for a generic immersion of a singular link in R2. We tackle this problem by the so-called…
Improved Frank-Wolfe method reduces dependence on data size for empirical risk minimization.
problem Reducing dependence on number of data observations in Frank-Wolfe methods.
method Taylor-series approximated gradients applied to Frank-Wolfe method.
result Significant speed-ups over existing methods on real-world datasets.
We give an estimate of the first eigenvalue of the Laplace operator on a complete noncompact stable minimal hypersurface M in a complete simply connected Riemannian manifold with pinched negative sectional curvature. In the same ambient space, we prove that if a complete minimal hypersurface M has sufficiently smal…
A theorem simplifies mass-minimizing flat chains' regularity.
problem Understanding the regularity of mass-minimizing flat chains.
method Simple condition for fundamental regularity principle.
result Fundamental regularity principle holds for mass-minimizing chains.
The paper analyzes ℓq optimization methods for high-dimensional linear regression.
problem Estimating sparse parameters from noisy observations in high-dimensional settings.
method Introduces and analyzes ℓq optimization methods for sparse estimation. result Shows stable recovery properties and bounds for ℓq minimization and regularization methods.