arXiv research

Dropout controls model capacity in deep learning and matrix completion.

Paper tackles inventory management with deep learning, improving performance and adherence to constraints.

We investigate the capacity, convexity and characterization of a general family of norm-constrained feed-forward networks.

Normalization layers control deep neural network capacity, improving stability and generalization.

Adding noise controls capacity of function compositions.

We study a stochastic, continuous time model on a finite horizon for a firm that produces a single good. We model the production capacity as an Ito diffusion controlled by a nondecreasing process representing the cumulative investment. The firm aims to maximize its expected total net profit by choosing the optimal inve…

Nonlinear RNNs' memory capacity varies widely, making it impractical.

We study the volume of extrinsic balls and the capacity of extrinsic annuli in minimal submanifolds which are properly immersed with controlled radial sectional curvatures into an ambient manifold with a pole. The key results are concerned with the comparison of those volumes and capacities with the corresponding entit…

Study capacity constraints in continual learning with a simple model.

Exploiting capacity of sewer system using decentralized control is a cost effective mean of minimizing the overflow. Given the size of the real sewer system, exploiting all the installed control structures in the sewer pipes can be challenging. This paper presents a divide and conquer solution to implement decentralize…

Sparse codes improve optimal control tasks with correlated inputs.

Study optimizes pricing under uncertainty and capacity constraints.

Generalizes memory and forecasting capacities for nonlinear recurrent networks with dependent inputs.

This paper presents a general framework for norm-based capacity control for $L_{p,q}$ weight normalized deep neural networks. We establish the upper bound on the Rademacher complexities of this family. With an $L_{p,q}$ normalization where $q\le p^*$, and $1/p+1/p^{*}=1$, we discuss properties of a width-independent ca…

Deep learning models can generalize well even when they fit training data perfectly.

Remove symmetries to improve model optimization and performance.

We consider the problem of finding optimal strategies that maximize the average growth-rate of multiplicative stochastic processes. For a geometric Brownian motion the problem is solved through the so-called Kelly criterion, according to which the optimal growth rate is achieved by investing a constant given fraction o…

Modeling alignment as resource-limited cognitive processes, researchers derive performance bounds.

Paper applies theorem to find optimal investment boundary in stochastic capacity expansion.

The paper tackles imbalanced classification under operational constraints, proposing a framework to maximize sensitivity.

Truncated Singular Value Decomposition (SVD) calculates the closest rank-$k$ approximation of a given input matrix. Selecting the appropriate rank $k$ defines a critical model order choice in most applications of SVD. To obtain a principled cut-off criterion for the spectrum, we convert the underlying optimization prob…

New study shows how model complexity affects test risk, challenging classical theory.

Optimal trading strategy in Proof-of-Stake blockchain using continuous-time control.

We information-theoretically reformulate two measures of capacity from statistical learning theory: empirical VC-entropy and empirical Rademacher complexity. We show these capacity measures count the number of hypotheses about a dataset that a learning algorithm falsifies when it finds the classifier in its repertoire …

Proposes PIC and POIC for measuring task difficulty in RL.

RAF model explains neural networks' dual rule learning and fact memorization.

A new framework uses deep reinforcement learning to improve aircraft separation in busy airspace.

The memory capacity of linear echo state networks is accurately calculated using new numerical methods.

PINNs struggle with increasingly complex ODEs, especially when parameters control their complexity.

There are (at least) three approaches to quantifying information. The first, algorithmic information or Kolmogorov complexity, takes events as strings and, given a universal Turing machine, quantifies the information content of a string as the length of the shortest program producing it. The second, Shannon information…

The presence of Long Distance Dependencies (LDDs) in sequential data poses significant challenges for computational models. Various recurrent neural architectures have been designed to mitigate this issue. In order to test these state-of-the-art architectures, there is growing need for rich benchmarking datasets. Howev…

ADHD is being recognized as a diagnosis which persists into adulthood impacting economic, occupational, and educational outcomes. There is an increased need to accurately diagnose and recommend interventions for this population. One consideration is the development and implementation of reliable and valid outcome measu…

Recently, path norm was proposed as a new capacity measure for neural networks with Rectified Linear Unit (ReLU) activation function, which takes the rescaling-invariant property of ReLU into account. It has been shown that the generalization error bound in terms of the path norm explains the empirical generalization b…

We present experiments demonstrating that some other form of capacity control, different from network size, plays a central role in learning multilayer feed-forward networks. We argue, partially through analogy to matrix factorization, that this is an inductive bias that can help shed light on deep learning.

Residual networks' depth is mathematically equivalent to expanding an implicit ensemble size.

New insights into experience replay in RL algorithms.

Study on spectral stability of an embedded annulus under curve shortening and Ricci flows.

Let $(M,g)$ be a complete non-compact Riemannian manifold together with a function $e^h$, which weights the Hausdorff measures associated to the Riemannian metric. In this work we assume lower or upper radial bounds on some weighted or unweighted curvatures of $M$ to deduce comparisons for the weighted isoperimetric qu…

Reinforcement learning (RL) constitutes a promising solution for alleviating the problem of traffic congestion. In particular, deep RL algorithms have been shown to produce adaptive traffic signal controllers that outperform conventional systems. However, in order to be reliable in highly dynamic urban areas, such cont…

An array system of coupled maps is proposed as a model for economy evolution. The local dynamics of each map or agent is controlled by two parameters. One of them represents the growth capacity of the agent and the other one is a control term representing the local environmental pressure which avoids an exponential gro…

Optimizes query routing to LLMs under cost and resource constraints.

Optimal hidden-target learning for online inventory optimization on general convex sets.

ECLIPSE detects AI hallucinations in finance with high accuracy.

We give a singular control approach to the problem of minimizing an energy functional for measures with given total mass on a compact real interval, when energy is defined in terms of a completely monotone kernel. This problem occurs both in potential theory and when looking for optimal financial order execution strate…

State of the art deep reinforcement learning algorithms take many millions of interactions to attain human-level performance. Humans, on the other hand, can very quickly exploit highly rewarding nuances of an environment upon first discovery. In the brain, such rapid learning is thought to depend on the hippocampus and…

Deep learning has enabled traditional reinforcement learning methods to deal with high-dimensional problems. However, one of the disadvantages of deep reinforcement learning methods is the limited exploration capacity of learning agents. In this paper, we introduce an approach that integrates human strategies to increa…

Proposes a new reinforcement learning method to improve agent performance in control tasks.

Proves generalization bounds for SGD using Feller processes and Hausdorff dimension.