In this article, we propose the notion of the general -affine capacity and prove some basic properties for the general -affine capacity, such as affine invariance and monotonicity. The newly proposed general -affine capacity is compared with several classical geometric quantities, e.g., the volume, the -var…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “generalization capacity”
We study various capacities on compact Kähler manifolds which generalize the Bedford-Taylor Monge-Ampère capacity. We then use these capacities to study the existence and the regularity of solutions of complex Monge-Ampère equations.
Solves a discrete logarithmic Minkowski problem for electrostatic p-capacity.
problem Characterize measures generated by electrostatic p-capacity.
method Solves the discrete logarithmic Minkowski problem for 1 < p < n.
result Solves the discrete logarithmic Minkowski problem for measures in general position.
We introduce the concept of pseudo symplectic capacities which is a mild generalization of that of symplectic capacities. As a generalization of the Hofer-Zehnder capacity we construct a Hofer-Zehnder type pseudo symplectic capacity and estimate it in terms of Gromov-Witten invariants. The (pseudo) symplectic capacitie…
Generalizes memory and forecasting capacities for nonlinear recurrent networks with dependent inputs.
problem Understanding memory and forecasting capabilities in networks with dependent inputs.
method Formulated bounds for memory and forecasting capacities in terms of network size and input properties.
result Proved that memory capacity for linear recurrent networks with independent inputs is given by the rank of the controllability matrix.
Study binary perceptrons' capacity using random duality theory.
problem Characterize the capacity of binary perceptrons with general thresholds.
method Utilized fully lifted random duality theory (fl RDT) to characterize the capacity.
result Characterizations match replica symmetry breaking predictions and uncover the capacity for zero-threshold scenario.
While symplectic manifolds have no local invariants, they do admit many global numerical invariants. Prominent among them are the so-called symplectic capacities. Different capacities are defined in different ways, and so relations between capacities often lead to surprising relations between different aspects of sympl…
Derives an empirical capacity model for self-attention neural networks.
problem Theoretical capacity of large transformer models is not fully utilized by current optimization algorithms.
method Analyzes memory capacity of transformers using synthetic training data and common training algorithms.
result Derives an empirical capacity model (ECM) for a generic transformer.
Study proposes local effective dimension to measure model capacity and generalization error.
problem Capturing the generalization power of machine learning models.
method Proposes local effective dimension as a capacity measure.
result Local effective dimension bounds the generalization error and correlates well with it.
New method uses relative capacities of geodesic balls to determine scalar curvature.
problem Determining scalar curvature from geodesic ball volumes.
method Using relative capacities of concentric small geodesic balls.
result Scalar curvature is determined by relative capacities of geodesic balls.
Study semicontinuity of capacity in non-smooth spaces using intrinsic flat convergence.
problem Investigate semicontinuity of capacity in non-smooth spaces.
method Analyze sequences of local integral current spaces converging in the pointed Sormani-Wenger intrinsic flat sense.
result Prove upper semicontinuity of capacity for balls and Lipschitz sublevel sets under volume-preserving convergence.
We investigate the capacity, convexity and characterization of a general family of norm-constrained feed-forward networks.
The paper proposes a probabilistic autoencoder for discovering causal directions between variables.
problem Finding the causal direction between two associated variables.
method Building an autoencoder of the joint distribution and maximizing its estimation capacity relative to marginal distributions.
result The higher estimation capacity is consistent with the unconstrained choice of a distribution representing the cause, while the lower capacity reflects the constraints imposed by the mechanism on the distribution of the effect.
Existence and uniqueness of the solution to the discrete Lp Minkowski problem for -capacity are proved when and . For general Lp Minkowski problem for -capacity, existence and uniqueness of the solution are given when and . These r…
Learning capacity measures model complexity, correlating with test loss and sample size.
problem Understanding model complexity and its relation to test performance.
method Formal correspondence between thermodynamics and inference; learning capacity as a measure of effective dimensionality.
result Learning capacity correlates with test loss and is a small fraction of model parameters.
Study shows mass-capacity inequality for specific geometric manifolds.
problem Establishing mass-capacity inequality for certain geometric manifolds.
method Using conformally flat manifolds with nonnegative scalar curvature.
result Equality implies harmonically conformal to a specific subset of Euclidean space.
Study shows how activation functions impact the storage capacity of treelike neural networks.
problem Understanding the role of activation functions in neural network expressive power.
method Analysis of treelike two-layer networks with various activation functions in the infinite-width limit.
result Activation functions affect storage capacity and robustness, with nonlinearity increasing capacity and decreasing robustness.
Normalization layers control deep neural network capacity, improving stability and generalization.
problem Excessive capacity in deep neural networks leads to overfitting and poor generalization.
method Developed a theoretical framework to explain normalization's role in capacity control.
result Normalization layers reduce the Lipschitz constant exponentially, smoothing the loss landscape and enhancing generalization.
Exact capacity of the \emph{wide} hidden layer treelike neural networks with generic activationsstat.ML
Wide hidden layer TCM nets capacity analyzed using RDT and fl RDT.
problem Capacity analysis of wide hidden layer TCM nets.
method Employed Fully Lifted Random Duality Theory (fl RDT) for capacity characterization.
result Explicit, closed form capacity characterizations for a generic class of hidden layer activations.
Label-Based Diversity Measure Among Hidden Units of Deep Neural Networks: A Regularization Methodcs.LG
New method reduces overfitting in deep neural networks by measuring and regulating hidden unit diversity.
problem Overfitting in deep neural networks.
method Introduces a new redundancy measure based on mutual information to improve generalization.
result Reduction of redundancy improves generalization capacity, reducing overfitting.
Recurrent neural networks are powerful models for processing sequential data, but they are generally plagued by vanishing and exploding gradient problems. Unitary recurrent neural networks (uRNNs), which use unitary recurrence matrices, have recently been proposed as a means to avoid these issues. However, in previous …
WGANs improve probability distribution approximation with depth and width trade-offs.
problem Approximating complex probability distributions accurately.
method Wasserstein GANs with GroupSort discriminators, quantified generalization bound.
result High-capacity discriminators are crucial for WGANs' performance.
New analysis shows capacity of treelike neural networks with various activations.
problem Analyzing the capacity of treelike neural networks with diverse activations.
method Utilized Random Duality Theory and its partially lifted version to handle various activations.
result The capacity of treelike neural networks decreases for large network width but converges to a constant value.
Developing Knowledge States: Technology and the Enhancement of National Statistical Capacityq-fin.GN
National statistical systems are the enterprises tasked with collecting, validating and reporting societal attributes. These data serve many purposes - they allow governments to improve services, economic actors to traverse markets, and academics to assess social theories. National statistical systems vary in quality, …
Study sharp decay of capacity for subharmonic functions on compact Hermitian manifolds.
problem Sharp decay of capacity of sublevel sets of -subharmonic functions.
method Generalizes previous results on Kähler manifolds and obtains full characterizations of polar sets.
result Full characterizations of polar sets and extremal functions.
Solution space and storage capacity of fully connected two-layer neural networks with generic activation functionscond-mat.dis-nn
Study on neural networks' storage capacity and solution space structure.
problem Understanding the storage capacity and solution space structure of neural networks.
method Replica method from statistical physics.
result Storage capacity per parameter remains finite even with infinite width and weights exhibit negative correlations.
Dropout controls model capacity in deep learning and matrix completion.
problem Controlling model capacity in deep learning and matrix completion problems.
method Investigates dropout's effect on model capacity and Rademacher complexity.
result Dropout induces a regularizer that controls model capacity in expectation.
New algorithm for shareable arms with load-dependent rewards in stochastic bandits.
problem Learning optimal play strategy with shareable finite-capacity arms in stochastic bandits.
method Developed a capacity estimator and online learning algorithm for MP-MAB with shareable arms.
result Regret upper bound matches the lower bound, validating the algorithm's performance.
Scattering networks maximize separation on low-dimensional data.
problem Maximizing separation capacity on low-dimensional datasets.
method Characterize and bound separation capacity for feature extractors, then apply to scattering networks with specific criteria.
result Design criteria for scattering networks to maximize separation on low-dimensional data.
Maximizes capacity of extensions with fixed boundary data.
problem Maximizing the capacity of extensions with nonnegative scalar curvature.
method Using the method of Lagrange multipliers on the constraint space of scalar-flat extensions.
result Derives variational condition for maximal capacity extensions and proves they have constant scalar curvature.
CapOptix uses options theory to price capacity in electricity markets.
problem Traditional capacity market designs fail to account for risk and price shocks.
method Interprets capacity commitments as reliability options and uses Markov Regime Switching Process.
result CapOptix provides more accurate pricing of capacity premia compared to existing mechanisms.
Study non-squeezing phenomena in contact geometry using specific capacities.
problem Detect and quantify non-squeezing in contact geometry.
method Defined and computed two contact capacities, using spectral selectors and Givental's non-linear Maslov index.
result Discovered and quantified non-squeezing phenomena in lens spaces and strongly order able closed prequantizations.
Recently, path norm was proposed as a new capacity measure for neural networks with Rectified Linear Unit (ReLU) activation function, which takes the rescaling-invariant property of ReLU into account. It has been shown that the generalization error bound in terms of the path norm explains the empirical generalization b…
The Rules-and-Facts Model for Simultaneous Generalization and Memorization in Neural Networksstat.ML
RAF model explains neural networks' dual rule learning and fact memorization.
problem Understanding how neural networks learn rules and memorize facts simultaneously.
method Introduces the Rules-and-Facts (RAF) model to bridge generalization and memorization.
result Characterizes conditions for simultaneous rule learning and fact memorization in neural networks.
The paper establishes inequalities for -capacitary functions in flat half-spaces.
problem Understanding -capacitary functions in asymptotically flat half-spaces.
method Establishes monotone quantities and mass-capacity inequalities.
result Sharp inequalities attain equality on a Schwarzschild half-space.
Despite existing work on ensuring generalization of neural networks in terms of scale sensitive complexity measures, such as norms, margin and sharpness, these complexity measures do not offer an explanation of why neural networks generalize better with over-parametrization. In this work we suggest a novel complexity m…
Study excess capacity in neural networks using Rademacher complexity.
problem Understanding how much capacity deep networks have beyond what's needed for classification.
method Unified Rademacher complexity bounds for function composition and convolutional layers, considering Lipschitz constants and initialization norms.
result There is substantial excess capacity per task, and capacity can be kept similar across different tasks.
Study rigidity by logarithmic capacity and related functions.
problem Rigidity phenomena in kernel functions and capacities.
method Exploration of Bergman kernel, logarithmic capacity, Green's function, and Euclidean distance/volume.
result Established rigidity theorems by logarithmic capacity.
A latent function decomposition method is proposed for forecasting the capacity of lithium-ion battery cells. The method uses the Multi-Output Gaussian Process, a generative machine learning framework for multi-task and transfer learning. The MCGP decomposes the available capacity trends from multiple battery cells int…
Study capacity constraints in continual learning with a simple model.
problem Understanding optimal resource allocation for agents with limited memory and compute resources.
method Analyzes a capacity-constrained linear-quadratic-Gaussian (LQG) sequential prediction problem and demonstrates optimal capacity allocation strategies.
result Derives a solution to the capacity-constrained LQG sequential prediction problem and shows how to optimally allocate capacity across sub-problems in the steady state.
New complete panel dataset for LMICs helps analyze innovation and development.
problem Lack of complete data for empirical analyses in LMICs.
method Predictive Mean Matching multiple imputation technique.
result Created a large dataset of 47 variables for 82 LMICs from 2005-2019.
Adding noise controls capacity of function compositions.
problem Large capacity of function compositions with bounded capacity classes.
method Adding Gaussian noise to the output of before composing with .
result Noise effectively controls the capacity of , offering a general recipe for modular design.
Upper bounds for Lagrangian capacities of Liouville domains
problem Lagrangian capacity of Liouville domains
method Using -equivariant techniques
result Extremal Lagrangian torus on the boundary of ellipsoid
Memory capacity of DAM scales exponentially with feature separation, unaffected by correlations.
problem Understanding how feature correlations impact DAM's capacity.
method Developed an empirical framework to analyze DAM's capacity under varying feature correlations and pattern separations.
result Memory capacity scales exponentially with feature separation, unaffected by correlations.
Proves local maximizers for higher Ekeland-Hofer capacities in 4D star-shaped domains.
problem Finding local maximizers for higher Ekeland-Hofer capacities in specific domains.
method Analogous to 4D local Viterbo conjecture, proving maximizers for rational ellipsoids.
result Local maximizers of the k-th Ekeland-Hofer capacities are symplectomorphic to rational ellipsoids.
A long standing open problem in the theory of neural networks is the development of quantitative methods to estimate and compare the capabilities of different architectures. Here we define the capacity of an architecture by the binary logarithm of the number of functions it can compute, as the synaptic weights are vari…
Develops a theory for mth order p-affine capacity for convex bodies containing the origin.
problem Defines and studies the mth order p-affine capacity for convex bodies containing the origin.
method Provides equivalent definitions, proves properties, and establishes inequalities.
result Establishes inequalities comparing to other geometric measures.
Given a surface in an asymptotically flat 3-manifold with nonnegative scalar curvature, we derive an upper bound for the capacity of the surface in terms of the area of the surface and the Willmore functional of the surface. The capacity of a surface is defined to be the energy of the harmonic function which equals 0 o…