Study proposes local effective dimension to measure model capacity and generalization error.
problem Capturing the generalization power of machine learning models.
method Proposes local effective dimension as a capacity measure.
result Local effective dimension bounds the generalization error and correlates well with it.
Solves a discrete logarithmic Minkowski problem for electrostatic p-capacity.
problem Characterize measures generated by electrostatic p-capacity.
method Solves the discrete logarithmic Minkowski problem for 1 < p < n.
result Solves the discrete logarithmic Minkowski problem for measures in general position.
Study on risk measures using distorted Choquet integrals with random distortions.
problem Developing risk measures under random distortions of capacities.
method Introducing and analyzing randomly distorted Choquet integrals with respect to a distorted capacity, establishing properties and providing representations.
result Representation of comonotonic additive conditional risk measures using G-randomly distorted Choquet integrals.
Continuous solutions found for complex geometry equations.
problem Finding solutions to complex geometry equations on Hermitian manifolds.
method Proving existence of continuous quasi-plurisubharmonic solutions for specific measures.
result Existence of continuous quasi-plurisubharmonic solutions for measures dominated by capacity.
This paper is devoted to a geometric-measure-theoretic study of the brand new affine BV-capacity which is essentially different from the classic BV-capacity in dimension greater than one.
Introduces Rashomon Capacity to measure predictive multiplicity in probabilistic classifiers.
problem Predictive multiplicity in classification models leading to unjustified decisions.
method Introduces Rashomon Capacity, a metric for probabilistic classifiers, and provides a rigorous derivation.
result Rashomon Capacity captures nuanced score variations and provides strategies for disclosing conflicting models.
Learning capacity measures model complexity, correlating with test loss and sample size.
problem Understanding model complexity and its relation to test performance.
method Formal correspondence between thermodynamics and inference; learning capacity as a measure of effective dimensionality.
result Learning capacity correlates with test loss and is a small fraction of model parameters.
We study the relationship between geometry and capacity measures for deep neural networks from an invariance viewpoint. We introduce a new notion of capacity --- the Fisher-Rao norm --- that possesses desirable invariance properties and is motivated by Information Geometry. We discover an analytical characterization of…
Study excess capacity in neural networks using Rademacher complexity.
problem Understanding how much capacity deep networks have beyond what's needed for classification.
method Unified Rademacher complexity bounds for function composition and convolutional layers, considering Lipschitz constants and initialization norms.
result There is substantial excess capacity per task, and capacity can be kept similar across different tasks.
New measure predicts Dutch housing market downturns.
problem Understanding causes of Dutch housing boom and bust.
method Modelled household lending capacity using bank formulas.
result New measure outperforms traditional measures in forecasting housing prices.
Measures neural network complexity using tangent space diversity.
problem Estimating the true complexity of neural networks.
method Entropy-based measure of tangent spaces from different inputs.
result Captures effective complexity, not just theoretical capacity.
New study shows how model complexity affects test risk, challenging classical theory.
problem Understanding how test risk scales with model complexity for large over-parametrized deep networks.
method Developed norm-based capacity measures for random features based estimators, providing precise characterization of estimator's norm concentration and test error.
result Predicted learning curve shows a phase transition from under- to over-parameterization, confirming classical U-shaped behavior with appropriate capacity measures.
Recently, path norm was proposed as a new capacity measure for neural networks with Rectified Linear Unit (ReLU) activation function, which takes the rescaling-invariant property of ReLU into account. It has been shown that the generalization error bound in terms of the path norm explains the empirical generalization b…
Despite existing work on ensuring generalization of neural networks in terms of scale sensitive complexity measures, such as norms, margin and sharpness, these complexity measures do not offer an explanation of why neural networks generalize better with over-parametrization. In this work we suggest a novel complexity m…
New method reduces overfitting in deep neural networks by measuring and regulating hidden unit diversity.
problem Overfitting in deep neural networks.
method Introduces a new redundancy measure based on mutual information to improve generalization.
result Reduction of redundancy improves generalization capacity, reducing overfitting.
The paper derives a new theorem for predicting batches of data.
problem Finding lower bounds on minimal batch regret.
method Derives a conditional version of the regret-capacity theorem.
result Reveals a connection between conditional Rényi divergence and conditional Sibson's mutual information.
New measure FTC quantifies how much a ReLU network can fine-tune.
problem Analyzing memorization capacity in fine-tuned neural networks.
method Defined Fine-Tuning Capacity (FTC) for additive fine-tuning of ReLU networks.
result Upper and lower bounds on FTC for 2 and 3-layer ReLU networks.
We conduct an axiomatic study of the problem of estimating the strength of a known causal relationship between a pair of variables. We propose that an estimate of causal strength should be based on the conditional distribution of the effect given the cause (and not on the driving distribution of the cause), and study d…
ADHD is being recognized as a diagnosis which persists into adulthood impacting economic, occupational, and educational outcomes. There is an increased need to accurately diagnose and recommend interventions for this population. One consideration is the development and implementation of reliable and valid outcome measu…
Study Poincaré inequality in metric spaces via separating sets.
problem Geometric characterization of Poincaré inequality in metric spaces.
method Properties of separating sets and various notions of energy.
result Equivalence of conditions for 1-Poincaré inequality.
Generalizes memory and forecasting capacities for nonlinear recurrent networks with dependent inputs.
problem Understanding memory and forecasting capabilities in networks with dependent inputs.
method Formulated bounds for memory and forecasting capacities in terms of network size and input properties.
result Proved that memory capacity for linear recurrent networks with independent inputs is given by the rank of the controllability matrix.
The study explores how to infer the geometry of space forms from similarity comparisons.
problem Inferring the geometry of space forms from unreliable similarity measurements.
method Introducing ordinal capacity and spread, proving their relation to space form properties, and using statistical analysis of similarity measurements.
result The statistical behavior of ordinal spread variables can identify the underlying space form.
Researchers solve Dirichlet problem for complex Monge-Ampère equation on Hermitian manifolds.
problem Solving the Dirichlet problem for the complex Monge-Ampère equation on Hermitian manifolds with boundary.
method Weak quasi-plurisubharmonic solutions and optimal subsolution theorems for bounded and Hölder continuous quasi-plurisubharmonic functions.
result Proves continuity of solutions for measures well dominated by capacity, including Lp densities and moderate measures. Study on existence and properties of continuous solutions to complex Hessian equations.
problem Existence and properties of continuous solutions to complex Hessian equations.
method Established new capacity estimates and weak stability estimates for the m-Hessian measure. result Existence of continuous solutions to the complex Hessian equation under certain conditions.
Paper investigates Lambda Value-at-Risk under ambiguity and risk sharing.
problem Investigates Lambda Value-at-Risk under ambiguity and risk sharing.
method Establishes equivalence of robust ΛVaR and traditional ΛVaR under ambiguity sets, analyzes properties, derives explicit formulas, and explores risk sharing. result Unified and extended the concept of Value-at-Risk under ambiguity, derived explicit formulas for specific ambiguity sets, and explored risk sharing.
Study shows how correlations between neural activity affect classification capacity.
problem Understanding how correlations between neural activity impact classification performance.
method Calculated the capacity of neural activity on spherical manifolds with and without correlations between centroids and axes.
result Introducing correlations between neural activity centroids pushes spheres closer together, while correlations between axes shrink their radii, revealing a duality between correlations and geometry in classification.
The paper proposes a probabilistic autoencoder for discovering causal directions between variables.
problem Finding the causal direction between two associated variables.
method Building an autoencoder of the joint distribution and maximizing its estimation capacity relative to marginal distributions.
result The higher estimation capacity is consistent with the unconstrained choice of a distribution representing the cause, while the lower capacity reflects the constraints imposed by the mechanism on the distribution of the effect.
There are (at least) three approaches to quantifying information. The first, algorithmic information or Kolmogorov complexity, takes events as strings and, given a universal Turing machine, quantifies the information content of a string as the length of the shortest program producing it. The second, Shannon information…
New complexity measure helps in agnostic reinforcement learning with or without access to MDP dynamics.
problem Understanding the number of rounds needed to learn an ε-suboptimal policy in unknown MDPs.
method Introducing spanning capacity as a new complexity measure and developing POPLER algorithm.
result There is a separation between generative and online access models for agnostic learnability.
We information-theoretically reformulate two measures of capacity from statistical learning theory: empirical VC-entropy and empirical Rademacher complexity. We show these capacity measures count the number of hypotheses about a dataset that a learning algorithm falsifies when it finds the classifier in its repertoire …
gLSTM improves graph neural networks by increasing storage capacity to prevent over-squashing.
problem Over-squashing in GNNs collapses information from a large receptive field into a single vector, creating an information bottleneck.
method Introduced a new synthetic task to measure over-squashing and adapted ideas from sequence modeling to develop gLSTM, a novel GNN architecture with improved capacity.
result gLSTM architecture demonstrates strong performance on synthetic and real-world graph benchmarks, mitigating over-squashing.
We introduce the concept of hereditarily non uniformly perfect sets, compact sets for which no compact subset is uniformly perfect, and compare them with the following: Hausdorff dimension zero sets, logarithmic capacity zero sets, Lebesgue 2-dimensional measure zero sets, and porous sets. In particular, we give an exa…
Adding noise controls capacity of function compositions.
problem Large capacity of function compositions with bounded capacity classes.
method Adding Gaussian noise to the output of F before composing with H. result Noise effectively controls the capacity of H∘F, offering a general recipe for modular design. The affine-additive group is hyperbolic with a non-vanishing 4-capacity.
problem Characterizing the hyperbolicity of the affine-additive group.
method Proving local 4-Ahlfors regularity and hyperbolicity using a left-invariant metric and measure.
result The affine-additive group is hyperbolic with a non-vanishing 4-capacity.
Introduces LDM to estimate machine learning algorithm capacity.
problem Estimating the performance of supervised learning algorithms.
method Characterizes algorithm flexibility using the diversity of possible outputs.
result LDM provides valuable insight into algorithm prediction behavior.
Graph neural networks struggle to distinguish certain graph structures.
problem Difficulty in distinguishing graphs with graph neural networks.
method Analysis of communication capacity in message-passing model of graph neural networks.
result Capacity of MPNN needs to grow linearly for trees and quadratically for general connected graphs.
Framework for understanding overfitting and underfitting using information theory.
problem Understanding and preventing overfitting and underfitting in machine learning.
method Information-theoretic framework measuring algorithm capacity and dataset information transfer.
result Upper-bounding algorithm capacity and establishing its relationship to machine learning quantities.
Enhanced Hopfield model boosts memory retrieval capacity.
problem Memory retrieval in modern Hopfield models with limited capacity.
method Introduces a learnable feature map transforming energy function into kernel space, minimizing separation loss for uniform memory distribution.
result Significant reduction in metastable states, enhancing memory capacity and retrieval accuracy.
Study on uniquely determining thermal properties from boundary temperature and heat flux measurements.
problem Determine thermal conductivity and volumetric heat capacity from boundary measurements.
method Uniqueness proof for isotropic and anisotropic media under thermal diffusivity assumption.
result Uniqueness of thermal properties in all dimensions and up to a gauge in two dimensions.
Modeling alignment as resource-limited cognitive processes, researchers derive performance bounds.
problem Systematic deviations in feedback-based alignment of large language models.
method Modeling alignment as a two-stage cascade UoHoY given S, with cognitive and total capacities. result Capacity-coupled Alignment Performance Interval derived from Fano and PAC-Bayes bounds.
In this paper, we introduce the anisotropic Sobolev capacity with fractional order and develop some basic properties for this new object. Applications to the theory of anisotropic fractional Sobolev spaces are provided. In particular, we give geometric characterizations for a nonnegative Radon measure μ that naturall…
We investigate under and overfitting in Generative Adversarial Networks (GANs), using discriminators unseen by the generator to measure generalization. We find that the model capacity of the discriminator has a significant effect on the generator's model quality, and that the generator's poor performance coincides with…
Theory developed for complex Hessian measures on Hermitian manifolds.
problem Defining and analyzing complex Hessian measures on Hermitian manifolds.
method Potential theory for m-subharmonic functions with respect to a Hermitian metric.
result Equivalence between polar sets and negligible sets for m-subharmonic functions.
Differential privacy, a notion of algorithmic stability, is a gold standard for measuring the additional risk an algorithm's output poses to the privacy of a single record in the dataset. Differential privacy is defined as the distance between the output distribution of an algorithm on neighboring datasets that differ …
Proposes PIC and POIC for measuring task difficulty in RL.
problem Lack of metrics to measure task difficulty in RL.
method Introduces policy information capacity (PIC) and policy-optimal information capacity (POIC) as metrics based on mutual information.
result Empirically shows PIC and POIC correlate with task solvability better than alternatives.
Study on removing sets and uniqueness of diffusion operators on various spaces.
problem Determining the effect of removing small sets on the self-adjointness and uniqueness of diffusion operators.
method Analyzes symmetric diffusion operators on metric measure spaces, proving a truncation result for potentials.
result Characterizes the critical size of removed sets and their effect on operator properties.
The paper defines capacities for minimal graphs over manifolds and proves the half-space property.
problem Characterizing minimal graphs and their properties over manifolds.
method Defining capacities using relative volume, studying solutions of bounded variation, and analyzing boundary behavior.
result Proves the half-space property for M-parabolic manifolds. New risk measures for quantiles under ambiguity improve risk sharing.
problem Risk optimization under ambiguity using quantiles.
method Introducing Choquet quantiles and Choquet Expected Shortfall.
result Optimal allocations for quantile agents under ambiguity.