Iran's FiT policy boosts REs but risks financial crisis and plant failure.
problem Financial challenges in maintaining renewable energy growth after policy targets.
method System dynamics model considering budget status, social tolerance, and investor trust.
result Adjusting tax on electricity consumption based on budget status prevents financial crises and maintains stable growth.
A variety of large-scale machine learning problems can be cast as instances of constrained submodular maximization. Existing approaches for distributed submodular maximization have a critical drawback: The capacity - number of instances that can fit in memory - must grow with the data set size. In practice, while one c…
Deep learning models can generalize well even when they fit training data perfectly.
problem Generalization in over-parameterized deep learning models.
method Combining empirical risk minimization with capacity control, exploring inductive biases and smooth empirical risk minimizers.
result Double descent phenomenon: test error can decrease after interpolation point.
Modeling alignment as resource-limited cognitive processes, researchers derive performance bounds.
problem Systematic deviations in feedback-based alignment of large language models.
method Modeling alignment as a two-stage cascade UoHoY given S, with cognitive and total capacities. result Capacity-coupled Alignment Performance Interval derived from Fano and PAC-Bayes bounds.
The multivariate version of the Mixed Tempered Stable is proposed. It is a generalization of the Normal Variance Mean Mixtures. Characteristics of this new distribution and its capacity in fitting tails and capturing dependence structure between components are investigated. We discuss a random number generating procedu…
This work uses a recommendation system to enhance exporting countries' fitness.
problem Predicting and optimizing the evolution of international trade networks.
method A recommendation system was used to identify overlooked products for countries.
result Countries can improve their national competitiveness by diversifying exported products.
Data selection boosts fact memorization in language models.
problem Language models struggle to accurately memorize factual knowledge.
method Formalizes fact memorization, proposes data selection schemes based on training loss.
result Data selection boosts fact accuracy to model capacity and improves performance.
Neural networks fit fewer samples than their parameters suggest in practice.
problem Understanding the practical limitations of neural network flexibility.
method Examination of neural network optimization, parameter efficiency, and loss surfaces.
result Neural networks can only fit training sets with significantly fewer samples than their parameters suggest.
The paper explores how approximation theory can improve understanding of smooth kernels in machine learning.
problem Understanding the inferential properties of smooth kernels in machine learning.
method Analysis of eigenvalue decay, properties of eigenfunctions/eigenvectors, and fitting capacity of kernels.
result Eigenvalues of kernel matrices show nearly exponential decay, highlighting the 'approximation beats concentration' phenomenon.
New method lowers spherical perceptron capacity using fully lifted random duality theory.
problem Tackles the negative spherical perceptron capacity, a long-standing open problem.
method Develops fully lifted random duality theory (fl RDT) to characterize capacity.
result Shows remarkable closed-form analytical relations for practical capacity values.
New VAE model improves data fitting without sacrificing computational efficiency.
problem Limitation of Gaussian assumption in VAE for continuous variable fitting.
method Infinite mixture of asymmetric Laplace distribution in decoder, nonparametric M-estimator for quantile estimation.
result Model demonstrates superior data privacy adjustment and better distribution fitting.
Neural networks struggle with extrapolation, but a new framework allows them to learn counterfactual invariances.
problem Neural networks' inability to extrapolate beyond training data distribution.
method Introduces a learning framework that allows neural networks to extrapolate over group transformations based on counterfactual invariances.
result Neural networks can learn counterfactual invariances from a single environment, overcoming their limitations in extrapolation.
Modern machine learning practices contradict traditional bias-variance theory.
problem Modern machine learning models often fit data perfectly, yet perform well.
method Introducing a 'double descent' curve to reconcile classical and modern practices.
result Increasing model capacity beyond interpolation improves performance.
This paper explores memorization in adversarial training and proposes a mitigation algorithm.
problem Understanding and mitigating robust overfitting in adversarial training.
method Demonstrated the capacity of deep networks to memorize adversarial examples, analyzed convergence and generalization issues, and proposed a new mitigation algorithm.
result Identified robust overfitting as a significant drawback of adversarial training and proposed a mitigation algorithm.
New theory predicts which large DNNs will have best test accuracy.
problem Predicting which large pre-trained DNNs will have the best test accuracy.
method Heavy-Tailed Self-Regularization (HT-SR) and Universal capacity control metric based on power law exponents.
result Universal capacity control metric correlates well with reported test accuracies of large-scale DNNs.
Improves two-stage hashing methods for better image retrieval.
problem Developing efficient binary codes for image retrieval.
method Theoretical analysis and empirical improvements of two-stage hashing methods using high-capacity hash functions.
result Proposes a novel two-stage hashing method significantly outperforming previous studies.
AON improves neural network generalization by making weights approximately orthogonal.
problem Improving generalization of deep neural networks.
method Approximated orthonormal normalisation (AON) technique to make weight vectors approximately orthogonal.
result AON yields promising validation performance compared to orthonormal regularisation.
Novel GNN model tackles few-shot learning with improved performance.
problem Few-shot learning with GNN suffers from over-fitting and over-smoothing.
method Proposes Attentive GNN with triple-attention mechanism.
result Improves GNN performance for few-shot learning tasks.
Neural interaction discoveries can be real or artifacts of model flexibility.
problem Identifying real neural interactions from data.
method Using a multiplicative-gating extension of neural additive vector autoregression.
result Effective rank of the joint lag-block covariance predicts interaction recoverability.
Neural networks generalize well despite overfitting due to high capacity.
problem Understanding why deep neural networks generalize well in overparameterized settings.
method High-dimensional asymptotic analysis of generalization under kernel regression with Neural Tangent Kernel.
result Test error exhibits non-monotonic behavior and can have additional peaks and descents in the overparameterized regime.
Generative Adversarial Networks (GANs) are powerful models for learning complex distributions. Stable training of GANs has been addressed in many recent works which explore different metrics between distributions. In this paper we introduce Fisher GAN which fits within the Integral Probability Metrics (IPM) framework f…
We study various capacities on compact Kähler manifolds which generalize the Bedford-Taylor Monge-Ampère capacity. We then use these capacities to study the existence and the regularity of solutions of complex Monge-Ampère equations.
Solves a discrete logarithmic Minkowski problem for electrostatic p-capacity.
problem Characterize measures generated by electrostatic p-capacity.
method Solves the discrete logarithmic Minkowski problem for 1 < p < n.
result Solves the discrete logarithmic Minkowski problem for measures in general position.
Study finds macroeconomic indicators predict health workforce and infrastructure measures.
problem Evaluating the predictive value of macroeconomic indicators for public health targets.
method Examined multiple forecasting approaches including neural networks, generalized additive models, random forests, and time series models with exogenous indicators.
result Macroeconomic indicators provide consistent and reproducible predictive signals for health workforce and infrastructure measures, but less so for other targets.
CapOptix uses options theory to price capacity in electricity markets.
problem Traditional capacity market designs fail to account for risk and price shocks.
method Interprets capacity commitments as reliability options and uses Markov Regime Switching Process.
result CapOptix provides more accurate pricing of capacity premia compared to existing mechanisms.
In this article, we propose the notion of the general p-affine capacity and prove some basic properties for the general p-affine capacity, such as affine invariance and monotonicity. The newly proposed general p-affine capacity is compared with several classical geometric quantities, e.g., the volume, the p-var…
Extends capacity analysis to neural networks, showing how capacity is distributed across layers.
problem How capacity is distributed in neural networks with non-linear layers.
method Introduces layer decoupling to quantify non-linear activation's impact, and uses a markovian rule for capacity propagation in deep networks.
result Shows that under certain conditions, capacity allocation in neural networks is equivalent to linear capacity allocation in an extended input space.
New algorithm learns and unlearns from streaming data efficiently.
problem Continuous learning and unlearning from production data streams.
method Translated batch unlearning techniques to online setting using regret, sample complexity, and deletion capacity.
result Achieved logarithmic regret bound of O(lnT) for online unlearning. While symplectic manifolds have no local invariants, they do admit many global numerical invariants. Prominent among them are the so-called symplectic capacities. Different capacities are defined in different ways, and so relations between capacities often lead to surprising relations between different aspects of sympl…
Study excess capacity in neural networks using Rademacher complexity.
problem Understanding how much capacity deep networks have beyond what's needed for classification.
method Unified Rademacher complexity bounds for function composition and convolutional layers, considering Lipschitz constants and initialization norms.
result There is substantial excess capacity per task, and capacity can be kept similar across different tasks.
Study rigidity by logarithmic capacity and related functions.
problem Rigidity phenomena in kernel functions and capacities.
method Exploration of Bergman kernel, logarithmic capacity, Green's function, and Euclidean distance/volume.
result Established rigidity theorems by logarithmic capacity.
Study binary perceptrons' capacity using random duality theory.
problem Characterize the capacity of binary perceptrons with general thresholds.
method Utilized fully lifted random duality theory (fl RDT) to characterize the capacity.
result Characterizations match replica symmetry breaking predictions and uncover the capacity for zero-threshold scenario.
We introduce a new discrepancy score between two distributions that gives an indication on their similarity. While much research has been done to determine if two samples come from exactly the same distribution, much less research considered the problem of determining if two finite samples come from similar distributio…
Study capacity constraints in continual learning with a simple model.
problem Understanding optimal resource allocation for agents with limited memory and compute resources.
method Analyzes a capacity-constrained linear-quadratic-Gaussian (LQG) sequential prediction problem and demonstrates optimal capacity allocation strategies.
result Derives a solution to the capacity-constrained LQG sequential prediction problem and shows how to optimally allocate capacity across sub-problems in the steady state.
New complete panel dataset for LMICs helps analyze innovation and development.
problem Lack of complete data for empirical analyses in LMICs.
method Predictive Mean Matching multiple imputation technique.
result Created a large dataset of 47 variables for 82 LMICs from 2005-2019.
Subjective expected utility theory assumes that decision-makers possess unlimited computational resources to reason about their choices; however, virtually all decisions in everyday life are made under resource constraints - i.e. decision-makers are bounded in their rationality. Here we experimentally tested the predic…
Upper bounds for Lagrangian capacities of Liouville domains
problem Lagrangian capacity of Liouville domains
method Using S1-equivariant techniques result Extremal Lagrangian torus on the boundary of ellipsoid
New model prevents forgetting in continual learning.
problem Learning from a continuous stream of tasks without forgetting.
method Bayesian Optimized Continual Learning with Attention Mechanism (BOCL).
result BOCL outperforms state-of-the-art in preventing catastrophic forgetting and fitting new tasks better.
Memory capacity of DAM scales exponentially with feature separation, unaffected by correlations.
problem Understanding how feature correlations impact DAM's capacity.
method Developed an empirical framework to analyze DAM's capacity under varying feature correlations and pattern separations.
result Memory capacity scales exponentially with feature separation, unaffected by correlations.
A new unsupervised learning method calibrates rough volatility models efficiently.
problem Efficient calibration of rough volatility models with minimal data.
method Unsupervised learning using BSDE representation and neural networks.
result The proposed scheme minimizes loss and approximates BSDE solution.
Proves local maximizers for higher Ekeland-Hofer capacities in 4D star-shaped domains.
problem Finding local maximizers for higher Ekeland-Hofer capacities in specific domains.
method Analogous to 4D local Viterbo conjecture, proving maximizers for rational ellipsoids.
result Local maximizers of the k-th Ekeland-Hofer capacities are symplectomorphic to rational ellipsoids.
The paper introduces capacity allocation analysis for neural networks, focusing on spatial capacity.
problem Designing neural network architectures is challenging due to the interplay of intuition, experimentation, and luck.
method Introduces capacity allocation analysis, focusing on spatial capacity allocation in linear settings.
result Quantitative comparison of classical architectures on various synthetic tasks reveals insights into model capacity allocation.
Develops a theory for mth order p-affine capacity for convex bodies containing the origin.
problem Defines and studies the mth order p-affine capacity for convex bodies containing the origin.
method Provides equivalent definitions, proves properties, and establishes inequalities.
result Establishes inequalities comparing to other geometric measures.
Derives an empirical capacity model for self-attention neural networks.
problem Theoretical capacity of large transformer models is not fully utilized by current optimization algorithms.
method Analyzes memory capacity of transformers using synthetic training data and common training algorithms.
result Derives an empirical capacity model (ECM) for a generic transformer.
Improves online learning algorithms for functional models with capacity assumptions.
problem Convergence rates of online stochastic gradient descent algorithms for functional linear models.
method Characterizations of slope function regularity, kernel space capacity, and sampling process covariance operator.
result Capacity assumptions can alleviate saturation of convergence rates as function regularity increases.
We introduce the concept of pseudo symplectic capacities which is a mild generalization of that of symplectic capacities. As a generalization of the Hofer-Zehnder capacity we construct a Hofer-Zehnder type pseudo symplectic capacity and estimate it in terms of Gromov-Witten invariants. The (pseudo) symplectic capacitie…
Study relates symplectic homology capacity to periodic orbits in Liouville domains.
problem Relating symplectic homology capacity to periodic orbits in Liouville domains.
method Uses positive symplectic homology and Hofer-Zehnder capacity to establish bounds and existence of periodic points.
result Non-zero positive symplectic homology implies finite upper bound for Hofer-Zehnder capacity relative to skeleton and Hamiltonian diffeomorphisms.
Learning capacity measures model complexity, correlating with test loss and sample size.
problem Understanding model complexity and its relation to test performance.
method Formal correspondence between thermodynamics and inference; learning capacity as a measure of effective dimensionality.
result Learning capacity correlates with test loss and is a small fraction of model parameters.