arXiv research

Improves online learning algorithms for functional models with capacity assumptions.

New analysis tightens memory capacity of Hopfield models using spherical codes.

New analysis shows capacity of treelike neural networks with various activations.

Improved neural network capacity analysis using simplified RDT.

Designing neural network architectures is a task that lies somewhere between science and art. For a given task, some architectures are eventually preferred over others, based on a mix of intuition, experience, experimentation and luck. For many tasks, the final word is attributed to the loss function, while for some ot…

Enhances CNN feature extractors' separation capacity analysis.

Study on Privileged ERM showing limitations and providing capacity analysis.

Memory capacity of DAM scales exponentially with feature separation, unaffected by correlations.

Capacity analysis has been recently introduced as a way to analyze how linear models distribute their modelling capacity across the input space. In this paper, we extend the notion of capacity allocation to the case of neural networks with non-linear layers. We show that under some hypotheses the problem is equivalent …

Model predicts COVID-19 growth in Senegal, highlighting health care capacity importance.

Wide hidden layer TCM nets capacity analyzed using RDT and fl RDT.

RAF model explains neural networks' dual rule learning and fact memorization.

National statistical systems are the enterprises tasked with collecting, validating and reporting societal attributes. These data serve many purposes - they allow governments to improve services, economic actors to traverse markets, and academics to assess social theories. National statistical systems vary in quality, …

In this paper, the echo state network (ESN) memory capacity, which represents the amount of input data an ESN can store, is analyzed for a new type of deep ESNs. In particular, two deep ESN architectures are studied. First, a parallel deep ESN is proposed in which multiple reservoirs are connected in parallel allowing …

New measure FTC quantifies how much a ReLU network can fine-tune.

Modeling alignment as resource-limited cognitive processes, researchers derive performance bounds.

We study the relationship between geometry and capacity measures for deep neural networks from an invariance viewpoint. We introduce a new notion of capacity --- the Fisher-Rao norm --- that possesses desirable invariance properties and is motivated by Information Geometry. We discover an analytical characterization of…

An online learning framework optimizes pricing and capacity in service systems.

Survival analysis models predict economic convergence across Americas.

Study optimizes pricing under uncertainty and capacity constraints.

Reservoir computing's success depends on mapping different input time series to separable states.

Measures neural network complexity using tangent space diversity.

Study on CNNs' learning rates and approximation capacities.

The paper analyzes the performance of delay-based reservoir computing using eigenvalue analysis.

Paper develops an online learning algorithm for functional data models.

New analysis shows over-squashing limits GNNs' power.

ADHD is being recognized as a diagnosis which persists into adulthood impacting economic, occupational, and educational outcomes. There is an increased need to accurately diagnose and recommend interventions for this population. One consideration is the development and implementation of reliable and valid outcome measu…

New learning rates derived for Tikhonov-regularized problems without kernel assumptions.

We extend manifold capacity to nonlinear neural representations with contextual information.

New analysis explains pathology of deep Gaussian processes.

A major challenge in understanding the generalization of deep learning is to explain why (stochastic) gradient descent can exploit the network architecture to find solutions that have good generalization performance when using high capacity models. We find simple but realistic examples showing that this phenomenon exis…

Graph neural networks struggle to distinguish certain graph structures.

Introduces geometric formulation of EM algorithm for robust inference and various applications.

Semi-supervised node classification in attributed graphs, i.e., graphs with node features, involves learning to classify unlabeled nodes given a partially labeled graph. Label predictions are made by jointly modeling the node and its' neighborhood features. State-of-the-art models for node classification on such attrib…

We study various capacities on compact Kähler manifolds which generalize the Bedford-Taylor Monge-Ampère capacity. We then use these capacities to study the existence and the regularity of solutions of complex Monge-Ampère equations.

Wide neural networks can degrade performance, contrary to conventional wisdom.

It has been known for a long time that the classical spherical perceptrons can be used as storage memories. Seminal work of Gardner, \cite{Gar88}, started an analytical study of perceptrons storage abilities. Many of the Gardner's predictions obtained through statistical mechanics tools have been rigorously justified. …

Solves a discrete logarithmic Minkowski problem for electrostatic p-capacity.

CapOptix uses options theory to price capacity in electricity markets.

In this paper we study asymptotic behavior of $n$-superharmonic functions at isolated singularity using the Wolff potential and $n$-capacity estimates in nonlinear potential theory. Our results are inspired by and extend those of Arsove-Huber and Taliaferro in 2 dimensions. To study $n$-superharmonic functions we use a…

New insights into experience replay in RL algorithms.

In this article, we propose the notion of the general $p$-affine capacity and prove some basic properties for the general $p$-affine capacity, such as affine invariance and monotonicity. The newly proposed general $p$-affine capacity is compared with several classical geometric quantities, e.g., the volume, the $p$-var…

Optimal hidden-target learning for online inventory optimization on general convex sets.

Unweighted matrix factorization can match or outperform weighted methods in recommender systems.

While symplectic manifolds have no local invariants, they do admit many global numerical invariants. Prominent among them are the so-called symplectic capacities. Different capacities are defined in different ways, and so relations between capacities often lead to surprising relations between different aspects of sympl…

Study excess capacity in neural networks using Rademacher complexity.

Study rigidity by logarithmic capacity and related functions.

Study binary perceptrons' capacity using random duality theory.