Proposes Neural Complexity (NC) for predicting and explaining generalization in deep neural networks.
problem Challenges in specifying a suitable complexity measure for deep neural networks to predict and explain generalization.
method A meta-learning framework that learns a scalar complexity measure through interactions with many heterogeneous tasks.
result Trained NC model can be added to standard training loss to regularize any task learner.
Paper introduces a new, tractable measure of model complexity.
problem Need for a reliable measure of model complexity.
method Mathematically rigorous measure based on gradient similarities.
result Generalizes to various model types and insights into double descent.
Majorizing measures control sequential complexities for online learning.
problem Extending classical empirical processes theory to sequential cases.
method Generic chaining, majorizing measures, fractional covering numbers.
result Sharp control of worst-case sequential Rademacher complexity.
Solves complex Monge-Ampère equation for measures with pluripolar parts.
problem Characterizing measures with complex Monge-Ampère equation solutions.
method Solves for measures with a pluripolar part in compact Kähler manifolds.
result Generalizes classical results in bounded hyperconvex domains.
The Bergman measure converges to the Zhang measure on a hybrid space.
problem Proving convergence of Bergman measures to Zhang measure.
method Analyzing convergence on a hybrid space and metrized curve complex.
result Bergman measure converges to Zhang measure on a hybrid space.
Study semiclassical measures on complex hyperbolic quotients, identifying measure supports.
problem Understanding Laplacian eigenfunctions on complex hyperbolic quotients.
method Combining fractal uncertainty principle and Ratner theory to analyze measure supports.
result Semiclassical measures support is either cosphere bundle or a compact submanifold.
Establishing associations between the structure and the generalisation ability of deep neural networks (DNNs) is a challenging task in modern machine learning. Producing solutions to this challenge will bring progress both in the theoretical understanding of DNNs and in building new architectures efficiently. In this w…
Transductive learning considers situations when a learner observes m labelled training points and u unlabelled test points with the final goal of giving correct answers for the test points. This paper introduces a new complexity measure for transductive learning called Permutational Rademacher Complexity (PRC) and …
We develop a complexity measure for large-scale economic systems based on Shannon's concept of entropy. By adopting Leontief's perspective of the production process as a circular flow, we formulate the process as a Markov chain. Then we derive a measure of economic complexity as the average number of bits required to e…
Introduces LLC, a new complexity measure for DNNs based on SLT.
problem Lack of effective complexity measures for DNNs.
method Uses Singular Learning Theory to define LLC and proposes scalable estimator.
result Empirical evidence shows LLC provides valuable insights into DNN complexity.
Paper develops a new generalization bound using PAC-Bayes theory and Gibbs distributions.
problem Limits of traditional generalization bounds due to complexity measures.
method Leverages PAC-Bayes bounds with Gibbs distributions to derive a flexible generalization bound.
result Derives a generalization bound that can adapt to both hypothesis class and task complexity.
Investor sentiment improves model accuracy but complexity doesn't always boost predictive power.
problem Determining the optimal complexity of investor sentiment measures in asset pricing models.
method Comprehensive review of 71 papers from 2000-2021, analyzing various sentiment measures and models.
result Higher complexity of sentiment measures does not necessarily improve predictive power.
Study shows simple vector quantization measures correlate with deep learning generalization.
problem Understanding and predicting generalization in deep learning models.
method Applying complexity measures from approximation and information theory to deep learning features.
result Simple vector quantization measures correlate well with generalization performance in deep learning.
Measures neural network complexity using tangent space diversity.
problem Estimating the true complexity of neural networks.
method Entropy-based measure of tangent spaces from different inputs.
result Captures effective complexity, not just theoretical capacity.
Study on entanglement complexity of confined ring polymers in lattice tubes.
problem Understanding the entanglement complexity of confined ring polymers in lattice tubes.
method Applied knot theory to extend and prove results about the complexity of 2SAPs.
result Proved that all but exponentially few size m 2SAPs have F complexity that grows at least linearly in m as m approaches infinity.
Post-hoc model-agnostic interpretation methods such as partial dependence plots can be employed to interpret complex machine learning models. While these interpretation methods can be applied regardless of model complexity, they can produce misleading and verbose results if the model is too complex, especially w.r.t. f…
The paper introduces optimal transport kernels for comparing cell complexes.
problem Lack of machine learning methods for CW complexes.
method Derives explicit expression for Wasserstein distance, extends Fused Gromov-Wasserstein, introduces novel kernels.
result Introduced novel kernels for comparing probability measures on CW complexes.
Paper proposes LANN to measure model complexity of neural networks with curve activation functions.
problem Measuring model complexity of neural networks with curve activation functions.
method Proposes LANN, a piecewise linear framework to approximate curve activation functions, and derives complexity measure based on the number of linear regions.
result Demonstrates positive correlation between overfitting and model complexity during training.
Characteristics extracted from the training datasets of classification problems have proven to be effective predictors in a number of meta-analyses. Among them, measures of classification complexity can be used to estimate the difficulty in separating the data points into their expected classes. Descriptors of the spat…
Measures neural network complexity via effective degrees of freedom.
problem Challenges in quantifying neural network complexity.
method Adapts generalized degrees of freedom (GDF) for binary outcomes and compares with cross-validation and null degrees of freedom.
result GDF provides a robust measure of model complexity for neural networks.
In this paper, we propose a new measure to gauge the complexity of image classification problems. Given an annotated image dataset, our method computes a complexity measure called the cumulative spectral gradient (CSG) which strongly correlates with the test accuracy of convolutional neural networks (CNN). The CSG meas…
There are many methods developed to approximate a cloud of vectors embedded in high-dimensional space by simpler objects: starting from principal points and linear manifolds to self-organizing maps, neural gas, elastic maps, various types of principal curves and principal trees, and so on. For each type of approximator…
New algorithm recovers sparse binary vectors from generalized linear measurements efficiently.
problem Recovering sparse binary vectors from generalized linear measurements.
method Linear estimation algorithm and information theoretic lower bounds.
result Optimal sample complexity of O((k+σ2)logn) for noisy one bit quantized linear measurements. Investigates regularity of solutions to complex Hessian equation.
problem Regularity of solutions to complex Hessian equation.
method Analyzes solutions to Dirichlet problem with specific density condition.
result Establishes conditions for regularity of solutions.
A new complexity measure for neural networks improves upon classical methods.
problem Lack of a refined complexity measure for comparing different neural network architectures, especially permutation-invariant ones.
method Introduced an equivalence relation among linear functions and counted them relative to this relation.
result The new complexity measure clearly distinguishes between different models and increases exponentially with depth.
New complexity measure ADL connects to classical complexity measures.
problem Deriving generalization bounds for neural networks.
method Exploring ADL's relationship to Covering Numbers and VC Dimension.
result ADL is equivalent to Covering Numbers and VC Dimension for real-valued functions.
Solves Calabi-Yau equation on symplectic manifolds using measurable Kahler metrics.
problem Solving the Calabi-Yau equation on symplectic manifolds.
method Global deformation of almost complex structures compatible with symplectic form, constructing measurable Lipschitz Kahler metric.
result Existence theorem for solutions to the one-form type Calabi-Yau equation on closed symplectic manifolds.
Complex performance measures, beyond the popular measure of accuracy, are increasingly being used in the context of binary classification. These complex performance measures are typically not even decomposable, that is, the loss evaluated on a batch of samples cannot typically be expressed as a sum or average of losses…
The local kinematic formulas on complex space forms induce the structure of a commutative algebra on the space CurvU(n)∗ of dual unitarily invariant curvature measures. Building on the recent results from integral geometry in complex space forms, we describe this algebra structure explicitly as a…
This work defines a complexity measure for BAMDP planning and introduces state abstraction for more efficient approximate planning.
problem The computational intractability of exact BAMDP planning solutions.
method Define a complexity measure for BAMDP planning, introduce state abstraction, and develop an approximate planning algorithm.
result Introduces a computationally tractable approximate planning algorithm using state abstraction.
In this short note we compare the weighted Laplacians on real and complex (Kähler) metric measure spaces. In the compact case Kähler metric measure spaces are considered on Fano manifolds for the study of Kähler-Einstein metrics while real metric measure spaces are considered with Bakry-Émery Ricci tensor. There are tw…
Develops a measure-theoretic framework for complex co-occurrence data.
problem Modeling and interpreting complex co-occurrences in high-dimensional data.
method Introduces measure-theoretic probability and conditional probability, investigates E-integrals.
result Establishes a rigorous measure-theoretic foundation for co-occurrence modeling.
Paper infers intrinsic dimension from quasi-convex measurements.
problem Inferring intrinsic dimension from measurements by quasi-convex functions.
method Developed a method using filtration of Dowker complexes based on discrete data of point orderings.
result Correct intrinsic dimension can be inferred in the limit of large data under generic assumptions.
We prove a compactness theorem for embedded measured hyperbolic Riemann surface laminations in a compact almost complex manifold (X,J). To prove compactness result, we show that there is a suitable topology on the space of measured Riemann surface laminations induced by Levy-Prokhorov metric. As an application of th…
Deep convolutional neural networks (CNNs) have been shown to be able to fit a random labeling over data while still being able to generalize well for normal labels. Describing CNN capacity through a posteriori measures of complexity has been recently proposed to tackle this apparent paradox. These complexity measures a…
Generalization of deep networks has been of great interest in recent years, resulting in a number of theoretically and empirically motivated complexity measures. However, most papers proposing such measures study only a small set of models, leaving open the question of whether the conclusion drawn from those experiment…
Defines Vassiliev complexity measures for open and closed curves in 3D space.
problem Measuring complexity of curves in 3D space.
method Using enhanced Jones polynomial coefficients and Gauss code diagrams.
result Second Vassiliev measure converges to knot invariants as curve ends coincide.
CantorNet tests geometric and topological complexity in neural networks.
problem Understanding self-similar patterns in neural networks.
method Inspired by Cantor set, CantorNet introduces novel complexity measures.
result CantorNet's decision boundaries are analytically known and can be arbitrarily ragged.
A measure of neural complexity quantifies how hard it is to access information across neurons.
problem Understanding how mutual information is distributed among neurons in neural networks.
method Partial Information Decomposition (PID) to disentangle contributions of single neurons, multiple neurons, and synergistic effects.
result Representational Complexity measures the difficulty of accessing information across multiple neurons.
New measure shows various training techniques control model complexity.
problem Understanding how to control model complexity in deep learning.
method Developed geometric complexity measure and demonstrated its effectiveness.
result Many training techniques control geometric complexity, providing a unified framework.
Despite existing work on ensuring generalization of neural networks in terms of scale sensitive complexity measures, such as norms, margin and sharpness, these complexity measures do not offer an explanation of why neural networks generalize better with over-parametrization. In this work we suggest a novel complexity m…
New method measures generalizability of deep neural networks based on decision boundary complexity.
problem Lack of generalization methods for deep neural networks.
method Created Decision Boundary Complexity (DBC) score to measure DNN complexity.
result Simpler decision boundaries lead to better generalizability, supporting Occam's Razor.
Quantum machine learning has received significant attention in recent years, and promising progress has been made in the development of quantum algorithms to speed up traditional machine learning tasks. In this work, however, we focus on investigating the information-theoretic upper bounds of sample complexity - how ma…
New method counts boundary pieces in ReLU classifiers for better complexity measure.
problem Current classification complexity measures are misleading and ineffective.
method Developed a novel method using tropical geometry to count exact boundary pieces.
result Boundary piece count is negatively correlated with robustness.
Study on convergence of Narasimhan-Simha measures on degenerating families of Riemann surfaces.
problem Analyzing the convergence of measures on degenerating families of Riemann surfaces.
method Hybrid space approach, using metrized curve complex and Hermitian pairing.
result Convergence of measures on hybrid space, extending to singular curves.
We survey recent results in hermitian integral geometry, i.e. integral geometry on complex vector spaces and complex space forms. We study valuations and curvature measures on complex space forms and describe how the global and local kinematic formulas on such spaces were recently obtained. While the local and global k…
This paper tackles denoising of complex measures using optimal transport and curvature analysis.
problem Denoising of complex, possibly non-log-concave measures.
method Score function and optimal transport theory to revert Langevin diffusion chains.
result The difficulty of denoising depends on the curvature complexity of the initial measure at specific SNR scales.
NeurIPS 2020 competition seeks to predict deep learning generalization.
problem Understanding and predicting generalization in deep learning models.
method Propose complexity measures to accurately predict generalization performance.
result A robust complexity measure could improve deep learning reliability.