Paper tackles complex risk in deep neural networks.
problem Complex risk in deep neural networks.
method Developed new approach for complex risk statistics.
result Derived dual representation for complex risk.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “statistical complexity”
The statistical complexity of quantum circuits is studied using Rademacher complexity.
problem Measuring the richness of quantum hypothesis spaces.
method Applying Rademacher complexity to quantum circuits, investigating dependencies on resources, depth, width, and input/output registers.
result Bounds on the capacity of quantum neural networks constrained by circuit depth, width, and resource measures.
The study examines how quantum resources enhance the complexity of quantum circuits.
problem Quantum resource enhancement on circuit complexity.
method Utilizing quantum resource theories, the study analyzes statistical complexities of quantum circuits with limited quantum resources.
result Bounds for statistical complexities of quantum circuits are derived and applied to specific cases.
Diffusion models learn simple statistics before complex ones, revealing a sample complexity exponent.
problem Understanding the learning dynamics of diffusion models.
method Empirical observations and theoretical analysis of diffusion models and denoisers.
result Diffusion models learn simple statistics (pair-wise correlations) at linear sample complexity, while higher-order statistics (e.g., fourth cumulant) require cubic sample complexity.
This paper simplifies computing higher-order -statistics efficiently.
problem The inefficiency of computing higher-order -statistics in practice.
method Decomposition, connection to Einstein summation, and treewidth-based complexity estimate.
result A new, more efficient algorithm to compute -statistics.
Improved computational complexity in statistical models using second-order information.
problem Polynomial convergence of gradient descent in singular statistical models.
method Normalized Gradient Descent (NormGD) algorithm with second-order information.
result NormGD reaches final statistical radius in logarithmic iterations of .
Rademacher complexity and spin glasses: A link between the replica and statistical theories of learningcond-mat.dis-nn
Statistical learning theory provides bounds of the generalization gap, using in particular the Vapnik-Chervonenkis dimension and the Rademacher complexity. An alternative approach, mainly studied in the statistical physics literature, is the study of generalization in simple synthetic-data models. Here we discuss the c…
Survey on using low-degree polynomials to assess statistical tasks complexity.
problem Understanding the complexity of statistical tasks using polynomial functions.
method Applying low-degree polynomials to measure the complexity of statistical tasks, including detection, recovery, and estimation.
result Low-degree polynomials provide a framework to predict and explain statistical-computational tradeoffs.
Fisher width is a geometric measure of complexity on statistical manifolds.
problem Complexity measures on statistical manifolds
method Introducing Fisher width as a Fisher-geometric analogue of Gaussian width
result Fisher width retains key structural features of Gaussian width while capturing anisotropic geometric effects
Establishes statistical and computational bounds for influence diagnostics.
problem Identifying influential datapoints or subsets in machine learning models.
method Finite-sample statistical bounds and computational complexity for influence functions and approximate maximum influence perturbations.
result Established statistical and computational guarantees for influence diagnostics.
This is a review article for Encyclopedia of Complexity and System Science, to be published by Springer http://refworks.springer.com/complexity/. The paper reviews statistical models for money, wealth, and income distributions developed in the econophysics literature since late 1990s.
The statistical leverage scores of a complex matrix record the degree of alignment between col and the coordinate axes in . These score are used in random sampling algorithms for solving certain numerical linear algebra problems. In this paper we present a max-plus algebr…
High-dimensional statistics advances in complex data domains.
problem Complex, rich datasets challenge traditional methods.
method Evolved to address sophisticated estimation and inference problems.
result Deepened connections with optimization, concentration, and information theory.
Efficient streaming algorithms for robust statistics with near-optimal memory.
problem High-dimensional robust statistics tasks in streaming model.
method First efficient streaming algorithms with near-optimal memory requirements.
result Near-optimal error guarantees and space complexity nearly-linear in the dimension for robust mean estimation.
Bayesian nonparametrics adapt model complexity to diverse datasets.
problem Complex challenges across statistics, computer science, and engineering.
method Flexible Bayesian nonparametric models that adapt model complexity.
result Bayesian nonparametrics offer innovative solutions to multi-object tracking.
New complexity measure for interactive learning reduces regret to near-optimal levels.
problem Challenges in sample-efficient, adaptive learning algorithms for interactive decision making.
method Introduces the Decision-Estimation Coefficient and the Estimation-to-Decisions (E2D) principle.
result Unified algorithm design principle E2D achieves optimal sample-efficient learning.
Paper extends SI method for detecting CPs in complex systems' frequency domain.
problem Identifying change points in complex systems' frequency domain.
method Extends SI framework to frequency domain using DFT properties and develops valid p-values.
result Reliable detection of genuine CPs with strong statistical guarantees.
Study shows computational and statistical gaps in Gaussian Single-Index Models.
problem Statistical and computational trade-offs in high-dimensional regression problems.
method Analysis of SQ and LDP frameworks, partial-trace algorithm.
result Computational algorithms require significantly more samples than information-theoretic limits.
Optimizes ICA performance in high dimensions with computational constraints.
problem Statistical optimality and computational tractability in ICA.
method Characterization of optimal sample complexity, development of computationally tractable estimates.
result Optimal sample complexity is linear in dimensionality, quadratic with low-degree polynomial algorithms.
We give nearly matching upper and lower bounds on the oracle complexity of finding -stationary points () in stochastic convex optimization. We jointly analyze the oracle complexity in both the local stochastic oracle model and the global oracle (or, statistical learning) model. This allows u…
Paper develops online statistical inference methods for stochastic optimization using Kiefer-Wolfowitz algorithms.
problem Online statistical inference of model parameters in stochastic optimization problems.
method Kiefer-Wolfowitz algorithm with random search directions, asymptotic distribution analysis.
result Developed valid confidence intervals for online statistical inference.
Two statistical tasks are shown to have equivalent sample complexity.
problem Determining if a function depends on only a few variables and identifying those variables.
method Proved statistical equivalence of feature selection and junta testing through sample complexity analysis.
result Brute-force algorithm is sample-optimal for both tasks with optimal sample size.
Statistical field theory aids in understanding deep learning complexities.
problem Complexity and lack of theoretical understanding in deep learning.
method Statistical field theory as a theoretical framework.
result Field theory provides insights into generalization, bias, and feature learning.
The paper extends statistical estimation techniques under differential privacy.
problem Establishing sample complexity bounds for estimation tasks under differential privacy.
method Proposes analogues of Le Cam's method, Fano's inequality, and Assouad's lemma under central differential privacy.
result Optimal sample complexity bounds for discrete distribution estimation under total variation and distances.
Polyak step size GD reaches final radius of convergence after log iterations.
problem Statistical and computational complexities of Polyak step size GD.
method Generalized smoothness and Lojasiewicz conditions, stability of gradients.
result Polyak step size GD reaches final statistical radius of convergence after logarithmic number of iterations.
The method to derive uniform bounds with Gaussian and Rademacher complexities is extended to the case where the sample average is replaced by a nonlinear statistic. Tight bounds are obtained for U-statistics, smoothened L-statistics and error functionals of l2-regularized algorithms.
Ridge regularization simplifies model complexity in data science.
problem Overfitting in statistical models.
method Adding a penalty on the magnitude of coefficients.
result Effective in reducing model complexity and improving generalization.
The paper explores Kähler and anti-Kähler structures on quasi-statistical manifolds.
problem Investigating Kähler and anti-Kähler structures on quasi-statistical manifolds.
method Analyzing conditions for integrability of almost complex structures and defining Kähler and anti-Kähler manifolds.
result Conditions for to be an anti-Kähler manifold are identified.
Complexity Dependent Error Rates for Physics-informed Statistical Learning via the Small-ball Methodstat.ML
Paper develops error rates for physics-informed learning, comparing it to data-driven methods.
problem Understanding the trade-off between soft penalties and hard constraints in PISL.
method Develops complexity-dependent error rates using the small-ball method.
result Physics-informed estimators have comparable error rates to hard constrained methods, differing only by constants.
The Morse-Smale complex of a function decomposes the sample space into cells where is increasing or decreasing. When applied to nonparametric density estimation and regression, it provides a way to represent, visualize, and compare multivariate functions. In this paper, we present some statistical results on es…
Study replicability in high-dimensional statistics, resolving open problems.
problem Ensuring consistent results in high-dimensional statistical tasks.
method Introduced replicable learning algorithms and established computational and statistical equivalence with high-dimensional isoperimetric tilings.
result Matching sample complexity upper and lower bounds for replicable mean estimation and coin problem.
New method tackles over-parameterized matrix sensing with FGD, improving statistical and computational complexity.
problem Solving low rank matrix sensing with over-specified factors when rank is unknown.
method Decomposing the factorized matrix into column spaces to capture extra ranks and analyze convergence.
result Convergence to a statistical error of after iterations.
Study on the Kodaira dimension of real parallelizable manifolds with almost complex structures.
problem Understanding the Kodaira dimension of real parallelizable manifolds with specific almost complex structures.
method Conditions and examples provided for calculating the Kodaira dimension of manifolds.
result Conditions under which the Kodaira dimension of a real parallelizable manifold is zero.
Framework for efficient statistical estimation with privacy guarantees.
problem Statistical estimation problems with differential privacy constraints.
method High-dimensional Propose-Test-Release (HPTR) framework combining exponential mechanism, robust statistics, and resilience.
result Near-optimal utility guarantees and tight local sensitivity bounds for various statistical problems.
DiffKnock improves feature selection in neural networks with complex dependencies and non-linear associations.
problem Selecting important features in neural networks with complex dependencies and non-linear associations.
method DiffKnock uses diffusion models to generate knockoffs and neural network statistics to measure feature importance.
result DiffKnock outperforms existing methods in detecting non-linear associations and preserving feature dependencies.
We apply information-based complexity analysis to support vector machine (SVM) algorithms, with the goal of a comprehensive continuous algorithmic analysis of such algorithms. This involves complexity measures in which some higher order operations (e.g., certain optimizations) are considered primitive for the purposes …
Near-optimal rates for multi-task learning with shared representations.
problem Approximation and statistical complexity of learning multiple operators.
method Multiple Neural Operators (MNO) architecture and comparison with DeepONet.
result Near-optimal upper and lower bounds for approximation and generalization.
Paper tightens statistical aggregation results using local complexity.
problem Combining predictors to achieve nearly optimal predictions.
method Replacing global complexity with local complexity, using PAC-Bayes localization.
result Localized versions of classical aggregation bounds proven, improving previous results.
New findings challenge the traditional U-shaped curve of model complexity and error, revealing a second descent in error as model size increases.
problem The traditional U-shaped curve of model complexity and prediction error is incomplete, with recent work suggesting a second descent in error as model size increases.
method Careful consideration of multiple complexity axes and a nonparametric statistics perspective were used to interpret the observed double descent curves.
result The observed double descent curves in classical statistical machine learning methods fold back into traditional convex shapes, resolving tensions with statistical intuition.
Study uses complex networks and machine learning to predict soccer match outcomes.
problem Predicting soccer match outcomes with complex networks and machine learning.
method Complex network metrics and match statistics were used to build machine learning models.
result Models based on passing networks were as effective as traditional models using match statistics.
New algorithm learns disjunctions faster than previous methods.
problem Learning Boolean disjunctions in the agnostic PAC model.
method Developed an agnostic learner with complexity .
result First separation between SQ and CSQ models in distribution-free agnostic learning.
Stock markets are complex systems exhibiting collective phenomena and particular features such as synchronization, fluctuations distributed as power-laws, non-random structures and similarity to neural networks. Such specific properties suggest that markets operate at a very special point. Financial markets are believe…
Study shows multi-distribution learning has slower rates than single-task learning.
problem Understanding the statistical complexity of learning from heterogeneous sources.
method Structured hypothesis-testing framework to capture the statistical cost of certifying near-optimality under bounded noise.
result Learning across multiple distributions incurs slow rates scaling with , even under constant noise levels.
The paper analyzes the statistical cost of tuning kernel hyperparameters in robust regression.
problem Finding the best interpolant from a class of kernels with unknown hyperparameters under adversarial noise.
method Finite-sample guarantees, subsampling guarantee for linear regression, ε-net argument for discretizing kernel parameterizations.
result Hyperparameter optimization increases sample complexity by just a logarithmic factor, compared to known parameters.
New methods improve confidence set calibration in complex models.
problem Challenges in maintaining confidence set coverage in complex models.
method TRUST and TRUST++ methods using simulated data for calibration.
result Methods achieve distribution-free conditional coverage and robust inference.
This study shows neural nets can approximate Turing machines with meaningful statistical properties.
problem Theoretical limitations in approximating Turing machines with neural networks.
method Formal definition of statistically meaningful approximation, analysis of boolean circuits and Turing machines using neural nets.
result Transformers can statistically meaningfully approximate Turing machines with polynomial sample complexity.
The paper establishes a nearly-sharp statistical threshold for efficient learning in Latent MDPs with separated components.
problem Learning Latent Markov Decision Processes (LMDPs) with separated components.
method The paper considers various notions of separation and establishes a nearly-sharp statistical threshold for efficient learning. It also presents a quasi-polynomial algorithm with time complexity scaling in terms of the statistical threshold under a weaker assumption of separability under the optimal policy, and a near-matching time complexity lower bound under the exponential time hypothesis.
result Establishes a nearly-sharp statistical threshold for efficient learning in Latent MDPs with separated components.
These are the written discussions of the paper "Bayesian measures of model complexity and fit" by D. Spiegelhalter et al. (2002), following the discussions given at the Annual Meeting of the Royal Statistical Society in Newcastle-upon-Tyne on September 3rd, 2013.