A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Learning capacity measures model complexity, correlating with test loss and sample size.
BestChanID identifies the channel with maximal capacity using training sequences.
Paper develops an online learning algorithm for functional data models.
Model for cross-border markets with limited transmission capacities.
A long standing open problem in the theory of neural networks is the development of quantitative methods to estimate and compare the capabilities of different architectures. Here we define the capacity of an architecture by the binary logarithm of the number of functions it can compute, as the synaptic weights are vari…
A new memory system handles non-stationary environments by self-sizing and retaining memories.
In this paper, we investigate the common scenario where every candidate item for recommendation is characterized by a maximum capacity, i.e., number of seats in a Point-of-Interest (POI) or size of an item's inventory. Despite the prevalence of the task of recommending items under capacity constraints in a variety of s…
An online learning framework optimizes pricing and capacity in service systems.
Capacity analysis has been recently introduced as a way to analyze how linear models distribute their modelling capacity across the input space. In this paper, we extend the notion of capacity allocation to the case of neural networks with non-linear layers. We show that under some hypotheses the problem is equivalent …
We study the computational capacity of a model neuron, the Tempotron, which classifies sequences of spikes by linear-threshold operations. We use statistical mechanics and extreme value theory to derive the capacity of the system in random classification tasks. In contrast to its static analog, the Perceptron, the Temp…
A variety of large-scale machine learning problems can be cast as instances of constrained submodular maximization. Existing approaches for distributed submodular maximization have a critical drawback: The capacity - number of instances that can fit in memory - must grow with the data set size. In practice, while one c…
External or internal shocks may lead to the collapse of a system consisting of many agents. If the shock hits only one agent initially and causes it to fail, this can induce a cascade of failures among neighoring agents. Several critical constellations determine whether this cascade remains finite or reaches the size o…
gLSTM improves graph neural networks by increasing storage capacity to prevent over-squashing.
Despite existing work on ensuring generalization of neural networks in terms of scale sensitive complexity measures, such as norms, margin and sharpness, these complexity measures do not offer an explanation of why neural networks generalize better with over-parametrization. In this work we suggest a novel complexity m…
Enhances sequence memory capacity in neural networks.
Study on neural networks' storage capacity and solution space structure.
Modeling alignment as resource-limited cognitive processes, researchers derive performance bounds.
Researchers analyze neural process architectures and their representational capacities.
Reduced reservoir size for faster edge computing.
New study shows how model complexity affects test risk, challenging classical theory.
Analyzes factors affecting flow VI performance.
Sparse superposition codes were recently introduced by Barron and Joseph for reliable communication over the AWGN channel at rates approaching the channel capacity. The codebook is defined in terms of a Gaussian design matrix, and codewords are sparse linear combinations of columns of the matrix. In this paper, we prop…
New algorithm interpolates data with neural nets, independent of sample size.
Theory of MoE Transformers' generalization and scaling.
Two-layer neural networks learn features through a few gradient descent steps, improving approximation capacity.
We define a capacity which measures the size of Weinstein tubular neighbourhoods of Lagrangian submanifolds. In symplectic vector spaces this leads to bounds on the codisc radius for any closed Lagrangian submanifold in terms of Viterbo's isoperimetric inequality. Moreover, we prove a generalization of Gromov's packing…
Sparse codes improve optimal control tasks with correlated inputs.
New bounds on neural network capacity for treelike sign perceptrons using RDT.
We present experiments demonstrating that some other form of capacity control, different from network size, plays a central role in learning multilayer feed-forward networks. We argue, partially through analogy to matrix factorization, that this is an inductive bias that can help shed light on deep learning.
Residual networks' depth is mathematically equivalent to expanding an implicit ensemble size.
Graph neural networks struggle to distinguish certain graph structures.
Modern machine learning models are typically trained using Stochastic Gradient Descent (SGD) on massively parallel computing resources such as GPUs. Increasing mini-batch size is a simple and direct way to utilize the parallel computing capacity. For small batch an increase in batch size results in the proportional red…
Proposes a new graph representation method using tensor products.
We present a novel framework of knowledge distillation that is capable of learning powerful and efficient student models from ensemble teacher networks. Our approach addresses the inherent model capacity issue between teacher and student and aims to maximize benefit from teacher models during distillation by reducing t…
Expressive efficiency refers to the relation between two architectures A and B, whereby any function realized by B could be replicated by A, but there exists functions realized by A, which cannot be replicated by B unless its size grows significantly larger. For example, it is known that deep networks are exponentially…
Study on CNNs' learning rates and approximation capacities.
Automatically adjusts model size for continual Gaussian processes.
Binary Neural Networks (BNNs) are neural networks which use binary weights and activations instead of the typical 32-bit floating point values. They have reduced model sizes and allow for efficient inference on mobile or embedded devices with limited power and computational resources. However, the binarization of weigh…
Variational methods that rely on a recognition network to approximate the posterior of directed graphical models offer better inference and learning than previous methods. Recent advances that exploit the capacity and flexibility in this approach have expanded what kinds of models can be trained. However, as a proposal…
This work studies scaling laws for low-precision training in high-dimensional linear regression.
Recurrent neural networks (RNNs) have drawn interest from machine learning researchers because of their effectiveness at preserving past inputs for time-varying data processing tasks. To understand the success and limitations of RNNs, it is critical that we advance our analysis of their fundamental memory properties. W…
A new ensemble learning method called Residual Likelihood Forests improves performance and reduces model size.
ChatGPT predicts stock market reactions from news headlines without financial training.
Accurately predicting the future health of batteries is necessary to ensure reliable operation, minimise maintenance costs, and calculate the value of energy storage investments. The complex nature of degradation renders data-driven approaches a promising alternative to mechanistic modelling. This study predicts the ch…
Increasing the mini-batch size for stochastic gradient descent offers significant opportunities to reduce wall-clock training time, but there are a variety of theoretical and systems challenges that impede the widespread success of this technique. We investigate these issues, with an emphasis on time to convergence and…
Credit networks represent a way of modeling trust between entities in a network. Nodes in the network print their own currency and trust each other for a certain amount of each other's currency. This allows the network to serve as a decentralized payment infrastructure---arbitrary payments can be routed through the net…
Exploiting capacity of sewer system using decentralized control is a cost effective mean of minimizing the overflow. Given the size of the real sewer system, exploiting all the installed control structures in the sewer pipes can be challenging. This paper presents a divide and conquer solution to implement decentralize…
Paper studies a robust online learning algorithm for regression.