The paper introduces capacity allocation analysis for neural networks, focusing on spatial capacity.
problem Designing neural network architectures is challenging due to the interplay of intuition, experimentation, and luck.
method Introduces capacity allocation analysis, focusing on spatial capacity allocation in linear settings.
result Quantitative comparison of classical architectures on various synthetic tasks reveals insights into model capacity allocation.
Extends capacity analysis to neural networks, showing how capacity is distributed across layers.
problem How capacity is distributed in neural networks with non-linear layers.
method Introduces layer decoupling to quantify non-linear activation's impact, and uses a markovian rule for capacity propagation in deep networks.
result Shows that under certain conditions, capacity allocation in neural networks is equivalent to linear capacity allocation in an extended input space.
Study capacity constraints in continual learning with a simple model.
problem Understanding optimal resource allocation for agents with limited memory and compute resources.
method Analyzes a capacity-constrained linear-quadratic-Gaussian (LQG) sequential prediction problem and demonstrates optimal capacity allocation strategies.
result Derives a solution to the capacity-constrained LQG sequential prediction problem and shows how to optimally allocate capacity across sub-problems in the steady state.
Investigates scaling deep neural networks to avoid capacity issues.
problem Avoiding the shattering problem in deep neural networks.
method Formulates conjecture and studies various architectures to determine scaling relations.
result Reveals scaling relations for deep residual networks and recurrent networks.
Deep imagination optimizes decision-making in large trees with limited resources.
problem Optimal planning in large decision trees with limited resources and time.
method Analytical solutions and numerical analysis of sampling capacity allocation.
result Optimal policy is to allocate few samples per level for deep exploration, favoring depth over breadth.
LDTA expands LDA's topic modeling capacity with tree-structured priors.
problem Limited expressiveness of Dirichlet priors in LDA for complex topic relationships.
method Introduces Latent Dirichlet-Tree Allocation (LDTA) with Dirichlet-Tree (DT) priors, and develops universal mean-field variational inference and Expectation Propagation.
result LDTA enables expressive, tree-structured priors over topic proportions, expanding modeling capacity of LDA.
Sparse superposition codes were recently introduced by Barron and Joseph for reliable communication over the AWGN channel at rates approaching the channel capacity. The codebook is defined in terms of a Gaussian design matrix, and codewords are sparse linear combinations of columns of the matrix. In this paper, we prop…
New CNN layer selects important channels to improve model capacity.
problem Improving model capacity under resource constraints.
method Selective allocation of channels in convolutional layers.
result New layer allows new optima that generalize better.
New method optimizes Gaussian process allocation for BO.
problem Existing methods for inducing point allocation in BO hinder performance.
method Proposes a new allocation strategy using quality-diversity decomposition.
result Demonstrates improved BO performance through local high-fidelity modeling.
New method optimizes resource allocation for uncertain tasks.
problem Optimal resource allocation for uncertain tasks with limited capacity.
method Formulated as an assignment problem, optimized using learning to rank with net discounted cumulative gain.
result Achieves higher expected profit and precision compared to classification methods.
Optimizes query routing to LLMs under cost and resource constraints.
problem Non-uniform or adversarial batching in per-query routing methods leads to cost inefficiency.
method Batch-level, resource-aware routing framework that jointly optimizes model assignment for each batch.
result Robust routing framework improves accuracy by 1-14% over non-robust methods.
Optimizes power allocation for WDM in RoFSO systems.
problem Maximizing total capacity with power and eye safety constraints.
method Model-based Stochastic Dual Gradient algorithm and model-free Primal-Dual Deep Learning algorithm.
result Deep Learning algorithm outperforms average equal power allocation.
We discuss investment allocation to multiple alpha streams traded on the same execution platform with internal crossing of trades and point out differences with allocating investment when alpha streams are traded on separate execution platforms with no crossing. First, in the latter case allocation weights are non-nega…
Millimeter Wave (MmWave) communication is one of the key technology of the fifth generation (5G) wireless systems to achieve the expected 1000x data rate. With large bandwidth at mmWave band, the link capacity between users and base stations (BS) can be much higher compared to sub-6GHz wireless systems. Meanwhile, due …
Overprocuring reserves can improve network efficiency by using excess reserves for congestion management.
problem Optimizing energy and reserve allocation between zones to minimize costs and ensure deliverability.
method Developed allocation models for co-allocating traded energy and reserve products, considering both deterministic and stochastic flows.
result Excess reserve supplies can be used for congestion management, leading to additional network benefits.
Modeling alignment as resource-limited cognitive processes, researchers derive performance bounds.
problem Systematic deviations in feedback-based alignment of large language models.
method Modeling alignment as a two-stage cascade UoHoY given S, with cognitive and total capacities. result Capacity-coupled Alignment Performance Interval derived from Fano and PAC-Bayes bounds.
Unified formula for arbitrary liquidity operations in weighted AMMs
problem Decentralized resource allocation in intelligent transportation systems
method Weighted invariant adapted from Balancer-type AMMs
result Unified formula for four resource allocation operations
Paper studies optimal investing for retirees with risk constraints.
problem Retirees' longevity and living standard risks in a fluctuating market.
method Formulated as a portfolio choice problem under time-varying risk capacity constraint. Derived optimal investment strategy using differential equations. Demonstrated endogenous spending measure and active investment strategy.
result Time-varying risk capacity constraint impacts asset allocation in retirement.
RAF model explains neural networks' dual rule learning and fact memorization.
problem Understanding how neural networks learn rules and memorize facts simultaneously.
method Introduces the Rules-and-Facts (RAF) model to bridge generalization and memorization.
result Characterizes conditions for simultaneous rule learning and fact memorization in neural networks.
In these notes we discuss investment allocation to multiple alpha streams traded on the same execution platform, including when trades are crossed internally resulting in turnover reduction. We discuss approaches to alpha weight optimization where one maximizes P&L subject to bounds on volatility (or Sharpe ratio). The…
Study optimal treatment assignment policies under strategic agent responses.
problem Learning optimal treatment policies with strategic agents complicates estimation.
method Dynamic model with threshold convergence to mean-field equilibrium, consistent estimator for policy gradient.
result Threshold for treatment assignment converges to mean-field equilibrium threshold under large but finite number of agents.
Paper investigates Lambda Value-at-Risk under ambiguity and risk sharing.
problem Investigates Lambda Value-at-Risk under ambiguity and risk sharing.
method Establishes equivalence of robust ΛVaR and traditional ΛVaR under ambiguity sets, analyzes properties, derives explicit formulas, and explores risk sharing. result Unified and extended the concept of Value-at-Risk under ambiguity, derived explicit formulas for specific ambiguity sets, and explored risk sharing.
We propose a novel regularizer to improve the training of Generative Adversarial Networks (GANs). The motivation is that when the discriminator D spreads out its model capacity in the right way, the learning signals given to the generator G are more informative and diverse. These in turn help G to explore better and di…
PT network optimizes asset weights without forecasting returns.
problem Traditional asset allocation methods are error-prone and limit portfolio performance.
method PT network uses attention mechanisms to directly optimize Sharpe ratio.
result PT outperforms other algorithms in risk-adjusted performance.
Develops a new framework for integrating satellite allocations in small portfolios.
problem Feasibility constraints in small portfolios, not return predictability, are the primary concerns.
method A four-layer feasibility framework: physical, economic, structural, and epistemic.
result Closed-form feasibility bounds on satellite size, turnover, and breadth without return forecasts.
New risk measures for quantiles under ambiguity improve risk sharing.
problem Risk optimization under ambiguity using quantiles.
method Introducing Choquet quantiles and Choquet Expected Shortfall.
result Optimal allocations for quantile agents under ambiguity.
Paper optimizes battery storage in multiple energy markets for better profits.
problem Optimizing battery storage participation in multiple energy markets to balance supply and demand.
method Developed a joint bidding strategy combining intraday and frequency markets using mixed integer linear programming and a learned classifier strategy.
result The LCS increases overall profits by over 4% compared to static strategies and by more than 3% over a naive dynamic benchmark.
A new memory system handles non-stationary environments by self-sizing and retaining memories.
problem Non-stationary environments where memories arrive over time and must be distinguished from noise.
method A self-sizing continual associative memory that generalizes Hopfield's network, handling adaptation and novelty.
result The memory system grows to the intrinsic memory demand of the environment and retains memories without forgetting.
NPAS trains neural networks with a fixed parameter budget, improving performance and compactness.
problem Training neural networks requires memory, and existing methods struggle with arbitrary parameter budgets.
method NPAS learns to share parameters automatically, covering low and high budgets.
result NPAS and SSNs improve network performance and compactness across various tasks.
Product Kanerva Machines dynamically combine smaller models for better memory organization.
problem Limited organization in the Kanerva Machine.
method Introducing Product Kanerva Machines that dynamically combine multiple smaller Kanerva Machines.
result Product Kanerva Machines can discover spatial tunings that approximately factorize simple images by object.
Flow-SSN improves segmentation efficiency and accuracy.
problem Challenges in medical imaging segmentation, especially high-rank pixel-wise covariances.
method Generative segmentation model using discrete-time autoregressive and continuous-time flow variants.
result Flow-SSNs can estimate high-rank pixel-wise covariances efficiently without assuming rank or storing parameters.
Analyzes COVID-19 data to predict mortality, forecast spread, and optimize resource allocation.
problem Challenges in patient triage, treatment, and care management during the pandemic.
method Integrated four-step approach combining descriptive, predictive, and prescriptive analytics.
result Optimized resource allocation and informed policy decisions.
Cultural activity is an inherent aspect of urban life and the success of a modern city is largely determined by its capacity to offer generous cultural entertainment to its citizens. To this end, the optimal allocation of cultural establishments and related resources across urban regions becomes of vital importance, as…
A two-stage training method improves GNN graph classification accuracy.
problem Maximizing GNN model capacity for graph classification.
method Two-stage training framework using triplet loss.
result Consistent improvement in accuracy up to 5.4\% points.
Robust forecast framework reduces distribution error by 63%.
problem Accurate distribution forecast for planning decisions.
method Backtest-based bootstrap and adaptive residual selection.
result Reduces Absolute Coverage Error by more than 63%.
Graph neural networks optimize radio resource management policies for wireless networks.
problem Optimizing user selection and power control in wireless networks with fairness constraints.
method Formulated as a Lagrangian dual problem, RRM policies are parameterized by a GNN architecture trained on channel conditions.
result The method achieves superior tradeoff between average and 5th percentile rates, demonstrating fairness.
Intelligent behaviour in the real-world requires the ability to acquire new knowledge from an ongoing sequence of experiences while preserving and reusing past knowledge. We propose a novel algorithm for unsupervised representation learning from piece-wise stationary visual data: Variational Autoencoder with Shared Emb…
Estimates the generalization error of deep neural networks without relying on capacity measures.
problem Understanding how generalization error scales with training data for deep neural networks.
method Derives estimates of generalization error for deep networks based on two assumptions: zero training error and error probability proportional to distance to nearest training point.
result Estimates the generalization error of DNNs as O(1/(δN^{1/d})), matching experimental behavior.
We study various capacities on compact Kähler manifolds which generalize the Bedford-Taylor Monge-Ampère capacity. We then use these capacities to study the existence and the regularity of solutions of complex Monge-Ampère equations.
Solves a discrete logarithmic Minkowski problem for electrostatic p-capacity.
problem Characterize measures generated by electrostatic p-capacity.
method Solves the discrete logarithmic Minkowski problem for 1 < p < n.
result Solves the discrete logarithmic Minkowski problem for measures in general position.
The paper optimizes DIA purchase policies using lifecycle models and asset allocation.
problem Determining the optimal allocation to Deferred Income Annuities (DIAs).
method Employed a lifecycle model with utility of consumption and bequest, formalized optimization process, analyzed results, and extended model to include asset allocation.
result Optimal DIA allocation varies based on refundability, asset allocation, and perceived longevity.
Study shows cognitive load impacts financial market efficiency, especially for less sophisticated investors.
problem Cognitive load's effect on financial market information processing.
method Developed a theoretical framework and tested it with exogenous disclosure complexity variation.
result Cognitive load significantly impairs price discovery, particularly for less sophisticated investors.
Study optimizes resource allocation in noisy systems for better control.
problem Limited attention in stochastic systems with multiplicative noise.
method Analytical and numerical methods for optimal attention allocation.
result Effective resource allocation enhances noise estimation and control decisions.
CapOptix uses options theory to price capacity in electricity markets.
problem Traditional capacity market designs fail to account for risk and price shocks.
method Interprets capacity commitments as reliability options and uses Markov Regime Switching Process.
result CapOptix provides more accurate pricing of capacity premia compared to existing mechanisms.
The paper proposes an asset allocation strategy using the Sortino ratio for better performance.
problem Traditional asset allocation methods like the Sharpe ratio do not penalize negative returns adequately.
method The Sortino ratio is used to maximize asset allocation, penalizing only negative return variances.
result The Sortino ratio-based strategy outperforms traditional methods like the Kelly criterion.
In this article, we propose the notion of the general p-affine capacity and prove some basic properties for the general p-affine capacity, such as affine invariance and monotonicity. The newly proposed general p-affine capacity is compared with several classical geometric quantities, e.g., the volume, the p-var…
This paper tackles post-trade allocation inefficiencies and presents a uniform return allocation method.
problem Return divergence among accounts after trade allocation.
method Systematic treatment of trade allocation risk, presenting a uniform return allocation method.
result Uniform allocation of returns irrespective of the number of accounts and trade sizes.
While symplectic manifolds have no local invariants, they do admit many global numerical invariants. Prominent among them are the so-called symplectic capacities. Different capacities are defined in different ways, and so relations between capacities often lead to surprising relations between different aspects of sympl…