Proves mass-capacity inequalities for critical area-normalized capacitors, improving Schwarzschild metric uniqueness.
problem Proving mass-capacity inequalities for critical area-normalized capacitors.
method Analyzes asymptotically flat manifolds with boundary capacity potential satisfying an overdetermined problem.
result Improves Schwarzschild metric uniqueness and results for spin asymptotically flat spacetimes.
A variety of large-scale machine learning problems can be cast as instances of constrained submodular maximization. Existing approaches for distributed submodular maximization have a critical drawback: The capacity - number of instances that can fit in memory - must grow with the data set size. In practice, while one c…
Kernel networks' stability edge linked to Fisher Information singularity.
problem Understanding the stability edge in high-capacity kernel Hopfield networks.
method Statistical manifold analysis and Riemannian geometry.
result The Ridge of Optimization corresponds to the Edge of Stability, revealing a dual equilibrium.
External or internal shocks may lead to the collapse of a system consisting of many agents. If the shock hits only one agent initially and causes it to fail, this can induce a cascade of failures among neighoring agents. Several critical constellations determine whether this cascade remains finite or reaches the size o…
Maximizes capacity of extensions with fixed boundary data.
problem Maximizing the capacity of extensions with nonnegative scalar curvature.
method Using the method of Lagrange multipliers on the constraint space of scalar-flat extensions.
result Derives variational condition for maximal capacity extensions and proves they have constant scalar curvature.
Truncated Singular Value Decomposition (SVD) calculates the closest rank-k approximation of a given input matrix. Selecting the appropriate rank k defines a critical model order choice in most applications of SVD. To obtain a principled cut-off criterion for the spectrum, we convert the underlying optimization prob…
Normalization layers control deep neural network capacity, improving stability and generalization.
problem Excessive capacity in deep neural networks leads to overfitting and poor generalization.
method Developed a theoretical framework to explain normalization's role in capacity control.
result Normalization layers reduce the Lipschitz constant exponentially, smoothing the loss landscape and enhancing generalization.
Double descent found in DRL, improving generalization with model capacity.
problem Generalization in over-parameterized DRL models.
method Actor-Critic framework, Policy Entropy metric.
result Policy Entropy significantly reduces as model capacity increases, indicating improved generalization.
DMs emerge from DenseAMs, transitioning from memorization to generalization.
problem Hindered memory retrieval in DenseAMs due to spurious states.
method Examined diffusion models through the lens of DenseAMs, focusing on their generative process.
result Identified a critical phase in DMs transitioning from memorization to generalization.
This study investigates self-organizing dynamics in a stochastic exponential DAM model using Temporal Complexity.
problem Understanding self-organizing behavior in artificial neural systems.
method Investigation of a stochastic exponential DAM model through Temporal Complexity analysis.
result The model exhibits regimes of complex intermittency with nontrivial temporal correlations and scale-free behavior.
The capacity of a neural network to absorb information is limited by its number of parameters. Conditional computation, where parts of the network are active on a per-example basis, has been proposed in theory as a way of dramatically increasing model capacity without a proportional increase in computation. In practice…
We study several quantities associated to the Green's function of a multiply connected domain in the complex plane. Among them are some intrinsic properties such as geodesics, curvature, and L2-cohomology of the capacity metric and critical points of the Green's function. The principal idea used is an affine scaling…
Deep learning has enabled traditional reinforcement learning methods to deal with high-dimensional problems. However, one of the disadvantages of deep reinforcement learning methods is the limited exploration capacity of learning agents. In this paper, we introduce an approach that integrates human strategies to increa…
This paper identifies critical cases for evaluating PV investment impacts on MV networks efficiently.
problem Challenges in maintaining and controlling voltages in MV distribution networks due to increasing PV generation.
method Clustering MV nodes based on electrical adjacency and time blocks, identifying critical cases for further study.
result A scalable method to time efficiently identify critical cases for PV investment evaluation.
Investigates the rotating Kepler problem for energy values ≤ -3/2.
problem Understanding periodic orbits and symplectic structures in rotating celestial mechanics.
method Ligon-Schaaf and Levi-Civita symplectic regularizations, special concave toric domain construction.
result Identification of a special concave toric domain (SCTD) for the RKP phase space.
Slice-based Learning improves model performance on critical data subsets.
problem Low performance on critical data subsets in machine learning models.
method Proposes a new programming model (Slice-based Learning) that uses slicing functions to specify critical data subsets and combines these with an attention mechanism.
result Improves model performance by up to 19.0 F1 on slices and 4.6 F1 overall.
Generative Adversarial Networks (GANs) are powerful models for learning complex distributions. Stable training of GANs has been addressed in many recent works which explore different metrics between distributions. In this paper we introduce Fisher GAN which fits within the Integral Probability Metrics (IPM) framework f…
This work addresses causal inference challenges in networked interference and proposes GNN-based estimators for individual treatment effects.
problem Estimating individual treatment effects in randomized experiments with networked interference.
method Uses Graph Neural Networks (GNNs) to capture network dependencies and derive causal effect estimators.
result Provides policy regret bounds and heuristic error bounds for GNN-based causal estimators under network interference and treatment capacity constraints.
This paper presents an infinite variational autoencoder (VAE) whose capacity adapts to suit the input data. This is achieved using a mixture model where the mixing coefficients are modeled by a Dirichlet process, allowing us to integrate over the coefficients when performing inference. Critically, this then allows us t…
FACMAC combines deep policy gradients with factored critic for multi-agent reinforcement learning.
problem Cooperative multi-agent reinforcement learning in discrete and continuous action spaces.
method FACMAC uses a centralised but factored critic, combining per-agent utilities into a joint action-value function.
result FACMAC outperforms MADDPG and other baselines on multi-agent particle environments and StarCraft II tasks.
Study on removing sets and uniqueness of diffusion operators on various spaces.
problem Determining the effect of removing small sets on the self-adjointness and uniqueness of diffusion operators.
method Analyzes symmetric diffusion operators on metric measure spaces, proving a truncation result for potentials.
result Characterizes the critical size of removed sets and their effect on operator properties.
This work improves communication efficiency in federated learning over wireless networks by optimizing energy consumption.
problem Optimizing energy consumption in federated learning over wireless networks.
method Adopting SignSGD for gradient sign exchange, considering channel capacity with outage, and proposing a stochastic sign-based algorithm for uneven data distribution.
result Proposed methods achieve a balance between learning performance and energy consumption.
Dreaming neural networks learn and consolidate patterns during sleep.
problem Maximizing information storage and critical capacity in neural networks.
method Daily routine of learning during awake state and consolidation during sleep, using Guerra's interpolation techniques.
result The network achieves perfect retrieval regime after sleep, storing the same number of patterns as neurons.
Complexity helps identify sparse risk factors in asset pricing.
problem Tension between feature richness and economic parsimony in high-dimensional asset pricing.
method Expanding feature space and using basis pursuit to discover sparse risk factors.
result Nonlinear feature expansions combined with basis pursuit yield superior out-of-sample performance.
We study various capacities on compact Kähler manifolds which generalize the Bedford-Taylor Monge-Ampère capacity. We then use these capacities to study the existence and the regularity of solutions of complex Monge-Ampère equations.
Solves a discrete logarithmic Minkowski problem for electrostatic p-capacity.
problem Characterize measures generated by electrostatic p-capacity.
method Solves the discrete logarithmic Minkowski problem for 1 < p < n.
result Solves the discrete logarithmic Minkowski problem for measures in general position.
CapOptix uses options theory to price capacity in electricity markets.
problem Traditional capacity market designs fail to account for risk and price shocks.
method Interprets capacity commitments as reliability options and uses Markov Regime Switching Process.
result CapOptix provides more accurate pricing of capacity premia compared to existing mechanisms.
In this article, we propose the notion of the general p-affine capacity and prove some basic properties for the general p-affine capacity, such as affine invariance and monotonicity. The newly proposed general p-affine capacity is compared with several classical geometric quantities, e.g., the volume, the p-var…
Extends capacity analysis to neural networks, showing how capacity is distributed across layers.
problem How capacity is distributed in neural networks with non-linear layers.
method Introduces layer decoupling to quantify non-linear activation's impact, and uses a markovian rule for capacity propagation in deep networks.
result Shows that under certain conditions, capacity allocation in neural networks is equivalent to linear capacity allocation in an extended input space.
While symplectic manifolds have no local invariants, they do admit many global numerical invariants. Prominent among them are the so-called symplectic capacities. Different capacities are defined in different ways, and so relations between capacities often lead to surprising relations between different aspects of sympl…
Study excess capacity in neural networks using Rademacher complexity.
problem Understanding how much capacity deep networks have beyond what's needed for classification.
method Unified Rademacher complexity bounds for function composition and convolutional layers, considering Lipschitz constants and initialization norms.
result There is substantial excess capacity per task, and capacity can be kept similar across different tasks.
Study rigidity by logarithmic capacity and related functions.
problem Rigidity phenomena in kernel functions and capacities.
method Exploration of Bergman kernel, logarithmic capacity, Green's function, and Euclidean distance/volume.
result Established rigidity theorems by logarithmic capacity.
A new data-driven model forecasts electricity prices efficiently.
problem Forecasting electricity prices using traditional methods.
method Integrates data-driven and fundamental models, learns from historical data.
result Significantly improves forecasting accuracy compared to existing models.
Study binary perceptrons' capacity using random duality theory.
problem Characterize the capacity of binary perceptrons with general thresholds.
method Utilized fully lifted random duality theory (fl RDT) to characterize the capacity.
result Characterizations match replica symmetry breaking predictions and uncover the capacity for zero-threshold scenario.
Study capacity constraints in continual learning with a simple model.
problem Understanding optimal resource allocation for agents with limited memory and compute resources.
method Analyzes a capacity-constrained linear-quadratic-Gaussian (LQG) sequential prediction problem and demonstrates optimal capacity allocation strategies.
result Derives a solution to the capacity-constrained LQG sequential prediction problem and shows how to optimally allocate capacity across sub-problems in the steady state.
New complete panel dataset for LMICs helps analyze innovation and development.
problem Lack of complete data for empirical analyses in LMICs.
method Predictive Mean Matching multiple imputation technique.
result Created a large dataset of 47 variables for 82 LMICs from 2005-2019.
Upper bounds for Lagrangian capacities of Liouville domains
problem Lagrangian capacity of Liouville domains
method Using S1-equivariant techniques result Extremal Lagrangian torus on the boundary of ellipsoid
Memory capacity of DAM scales exponentially with feature separation, unaffected by correlations.
problem Understanding how feature correlations impact DAM's capacity.
method Developed an empirical framework to analyze DAM's capacity under varying feature correlations and pattern separations.
result Memory capacity scales exponentially with feature separation, unaffected by correlations.
Proves local maximizers for higher Ekeland-Hofer capacities in 4D star-shaped domains.
problem Finding local maximizers for higher Ekeland-Hofer capacities in specific domains.
method Analogous to 4D local Viterbo conjecture, proving maximizers for rational ellipsoids.
result Local maximizers of the k-th Ekeland-Hofer capacities are symplectomorphic to rational ellipsoids.
The paper introduces capacity allocation analysis for neural networks, focusing on spatial capacity.
problem Designing neural network architectures is challenging due to the interplay of intuition, experimentation, and luck.
method Introduces capacity allocation analysis, focusing on spatial capacity allocation in linear settings.
result Quantitative comparison of classical architectures on various synthetic tasks reveals insights into model capacity allocation.
Develops a theory for mth order p-affine capacity for convex bodies containing the origin.
problem Defines and studies the mth order p-affine capacity for convex bodies containing the origin.
method Provides equivalent definitions, proves properties, and establishes inequalities.
result Establishes inequalities comparing to other geometric measures.
Derives an empirical capacity model for self-attention neural networks.
problem Theoretical capacity of large transformer models is not fully utilized by current optimization algorithms.
method Analyzes memory capacity of transformers using synthetic training data and common training algorithms.
result Derives an empirical capacity model (ECM) for a generic transformer.
Recurrent neural networks (RNNs) have drawn interest from machine learning researchers because of their effectiveness at preserving past inputs for time-varying data processing tasks. To understand the success and limitations of RNNs, it is critical that we advance our analysis of their fundamental memory properties. W…
Improves online learning algorithms for functional models with capacity assumptions.
problem Convergence rates of online stochastic gradient descent algorithms for functional linear models.
method Characterizations of slope function regularity, kernel space capacity, and sampling process covariance operator.
result Capacity assumptions can alleviate saturation of convergence rates as function regularity increases.
We introduce the concept of pseudo symplectic capacities which is a mild generalization of that of symplectic capacities. As a generalization of the Hofer-Zehnder capacity we construct a Hofer-Zehnder type pseudo symplectic capacity and estimate it in terms of Gromov-Witten invariants. The (pseudo) symplectic capacitie…
Study relates symplectic homology capacity to periodic orbits in Liouville domains.
problem Relating symplectic homology capacity to periodic orbits in Liouville domains.
method Uses positive symplectic homology and Hofer-Zehnder capacity to establish bounds and existence of periodic points.
result Non-zero positive symplectic homology implies finite upper bound for Hofer-Zehnder capacity relative to skeleton and Hamiltonian diffeomorphisms.
Learning capacity measures model complexity, correlating with test loss and sample size.
problem Understanding model complexity and its relation to test performance.
method Formal correspondence between thermodynamics and inference; learning capacity as a measure of effective dimensionality.
result Learning capacity correlates with test loss and is a small fraction of model parameters.
Generalizes memory and forecasting capacities for nonlinear recurrent networks with dependent inputs.
problem Understanding memory and forecasting capabilities in networks with dependent inputs.
method Formulated bounds for memory and forecasting capacities in terms of network size and input properties.
result Proved that memory capacity for linear recurrent networks with independent inputs is given by the rank of the controllability matrix.