arXiv research

New theory predicts which large DNNs will have best test accuracy.

There are (at least) three approaches to quantifying information. The first, algorithmic information or Kolmogorov complexity, takes events as strings and, given a universal Turing machine, quantifies the information content of a string as the length of the shortest program producing it. The second, Shannon information…

The paper derives a new theorem for predicting batches of data.

Dropout controls model capacity in deep learning and matrix completion.

Paper tackles inventory management with deep learning, improving performance and adherence to constraints.

This paper controls the capacity of weight-normalized deep neural networks using rectified linear units.

We investigate the capacity, convexity and characterization of a general family of norm-constrained feed-forward networks.

Normalization layers control deep neural network capacity, improving stability and generalization.

Adding noise controls capacity of function compositions.

Economic growth is unpredictable unless demand is quantified. We solve this problem by introducing the demand for unpaid spare time and a user quantity named human capacity. It organizes and amplifies spare time required for enjoying affluence like physical capital, the technical infrastructure for production, organize…

We study a stochastic, continuous time model on a finite horizon for a firm that produces a single good. We model the production capacity as an Ito diffusion controlled by a nondecreasing process representing the cumulative investment. The firm aims to maximize its expected total net profit by choosing the optimal inve…

Improved neural network capacity analysis using simplified RDT.

Nonlinear RNNs' memory capacity varies widely, making it impractical.

TVS-FNNs can approximate any continuous function on expanded input spaces.

We study the volume of extrinsic balls and the capacity of extrinsic annuli in minimal submanifolds which are properly immersed with controlled radial sectional curvatures into an ambient manifold with a pole. The key results are concerned with the comparison of those volumes and capacities with the corresponding entit…

Study capacity constraints in continual learning with a simple model.

Normalizing flows are shown to be equivalent to Bayesian networks, revealing new insights.

Sparse codes improve optimal control tasks with correlated inputs.

Study optimizes pricing under uncertainty and capacity constraints.

This paper uses unsupervised learning and dimensionality reduction to optimize sewer system control.

Deep residual networks can approximate any continuous function using control theory.

Enhanced Neural ODEs outperform traditional models in image classification and video prediction.

Generalizes memory and forecasting capacities for nonlinear recurrent networks with dependent inputs.

Study of Gödel Universe as Lie group with specific metric.

HardNet adds hard constraints to neural networks without sacrificing performance.

New capacity measure for ReLU networks reduces generalization error.

Deep learning models can generalize well even when they fit training data perfectly.

With the advent of large labelled datasets and high-capacity models, the performance of machine vision systems has been improving rapidly. However, the technology has still major limitations, starting from the fact that different vision problems are still solved by different models, trained from scratch or fine-tuned o…

Remove symmetries to improve model optimization and performance.

Benchmarking deep time series models for equity portfolios

The paper compares isoperimetric quotients and capacities in weighted manifolds.

Paper interprets ResNets via gate-network controls and deep-layer classifications.

We consider the problem of finding optimal strategies that maximize the average growth-rate of multiplicative stochastic processes. For a geometric Brownian motion the problem is solved through the so-called Kelly criterion, according to which the optimal growth rate is achieved by investing a constant given fraction o…

Modeling alignment as resource-limited cognitive processes, researchers derive performance bounds.

Graph neural networks struggle to distinguish certain graph structures.

Paper applies theorem to find optimal investment boundary in stochastic capacity expansion.

The massive amount of available data potentially used to discover patters in machine learning is a challenge for kernel based algorithms with respect to runtime and storage capacities. Local approaches might help to relieve these issues. From a statistical point of view local approaches allow additionally to deal with …

The paper tackles imbalanced classification under operational constraints, proposing a framework to maximize sensitivity.

Truncated Singular Value Decomposition (SVD) calculates the closest rank-$k$ approximation of a given input matrix. Selecting the appropriate rank $k$ defines a critical model order choice in most applications of SVD. To obtain a principled cut-off criterion for the spectrum, we convert the underlying optimization prob…

New study shows how model complexity affects test risk, challenging classical theory.

Transformers learn to cluster Gaussian mixtures as well as the EM algorithm.

Optimal trading strategy in Proof-of-Stake blockchain using continuous-time control.

Hybrid controller combines model-based and policy-based reinforcement learning.

We information-theoretically reformulate two measures of capacity from statistical learning theory: empirical VC-entropy and empirical Rademacher complexity. We show these capacity measures count the number of hypotheses about a dataset that a learning algorithm falsifies when it finds the classifier in its repertoire …

Proposes PIC and POIC for measuring task difficulty in RL.

RAF model explains neural networks' dual rule learning and fact memorization.

A new framework uses deep reinforcement learning to improve aircraft separation in busy airspace.

The memory capacity of linear echo state networks is accurately calculated using new numerical methods.