We extend the adaptive regression spline model by incorporating saturation, the natural requirement that a function extend as a constant outside a certain range. We fit saturating splines to data using a convex optimization problem over a space of measures, which we solve using an efficient algorithm based on the condi…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,694 papers · 148 categories
Trend · papers per month
48 results for “saturation condition”
A general nonlinear logistic equation has been proposed to model long-time saturation in industrial growth. An integral solution of this equation has been derived for any arbitrary degree of nonlinearity. A time scale for the onset of nonlinear saturation in industrial growth can be estimated from an equipartition cond…
New formulations for Ricci flows without smoothness.
problem Characterize Ricci flows without smooth solutions.
method Weak formulations of super Ricci flows with saturation condition.
result Generalized formulations for singular settings.
This paper explores saturation effects in spectral algorithms over large dimensions.
problem Saturation effects in spectral algorithms over large dimensions.
method Improved minimax lower bound and gradient flow with early stopping strategy.
result Exact convergence rates of spectral algorithms in large dimensional settings.
Paper proves KRR saturation effect for smooth functions.
problem Kernel ridge regression fails to reach theoretical limits for smooth functions.
method Proof of conjectured saturation lower bound for KRR.
result Proved the conjectured saturation lower bound for KRR.
A method to analyze neural network performance by measuring layer saturation.
problem Understanding which layers contribute to network performance.
method Layer saturation method: restricts layer output to eigenspace of variance matrix.
result Layer saturation indicates which layers contribute to network performance.
New examples show deletion type admissible pairs can be rigid under rational saturation.
problem Rigidity of admissible pairs of rational homogeneous spaces of Picard number one.
method Application of Mok's general criterion for non-subdiagram type admissible pairs.
result Examples of deletion type admissible pairs are rigid under rational saturation.
The paper extends kernel ridge regression to product kernels and reveals new convergence behaviors.
problem Understanding kernel ridge regression in large dimensions with various kernels.
method Established a broad family of large dimensional kernels and derived convergence rates.
result Revealed new phenomena including minimax optimality, saturation effect, and multiple descent behavior.
New method resolves density ratio estimation saturation issues.
problem Error saturation in density ratio estimation methods.
method Iterated regularization to improve kernel methods.
result Achieves fast error rates on regular learning problems.
We propose a metric, Layer Saturation, defined as the proportion of the number of eigenvalues needed to explain 99% of the variance of the latent representations, for analyzing the learned representations of neural network layers. Saturation is based on spectral analysis and can be computed efficiently, making live ana…
Efficient NTF algorithm for large sparse tensors.
problem Sparse multi-dimensional data and limitations of existing NTF algorithms.
method Saturating Coordinate Descent with element selection based on Lipschitz continuity.
result Proposes a scalable NTF algorithm for large tensors.
New insights into neural network training efficiency.
problem Understanding the optimal initialization for deep neural networks.
method Exploring the edge of chaos and saturation of tanh activation function.
result The line of uniformity in phase space intersects the edge of chaos, indicating saturation begins to hinder training efficiency.
The paper normalizes Poisson saturation of coregular submanifolds.
problem Normalizing the Poisson saturation of coregular submanifolds.
method Normal form construction and Poisson geometry analysis.
result Local Poisson saturation of coregular submanifolds is an embedded Poisson submanifold with a normal form.
Accelerating Non-Negative and Bounded-Variable Linear Regression Algorithms with Safe Screeningcs.LG
The paper accelerates regression algorithms by identifying saturated coordinates.
problem Non-negative and bounded-variable linear regression problems.
method Safe screening technique to identify saturated coordinates.
result The approach provides theoretical guarantees for identifying saturated coordinates.
A homogeneously saturated equation for the time development of the price of a financial asset is presented and investigated for the pricing of European call options using noise that is distributed as a Student's t-distribution. In the limit that the saturation parameter of the equation equals zero, the standard model o…
33 curves on a 3-genus surface, all intersecting at most once.
problem Finding a saturated system of curves on a surface of genus 3.
method Constructing 33 essential curves pairwise non-homotopic and intersecting at most once.
result The constructed system is saturated, not properly contained in any other system.
Water saturation is an important property in reservoir engineering domain. Thus, satisfactory classification of water saturation from seismic attributes is beneficial for reservoir characterization. However, diverse and non-linear nature of subsurface attributes makes the classification task difficult. In this context,…
A recent paper suggests that Deep Neural Networks can be protected from gradient-based adversarial perturbations by driving the network activations into a highly saturated regime. Here we analyse such saturated networks and show that the attacks fail due to numerical limitations in the gradient computations. A simple s…
Modelling long-term dependencies is a challenge for recurrent neural networks. This is primarily due to the fact that gradients vanish during training, as the sequence length increases. Gradients can be attenuated by transition operators and are attenuated or dropped by activation functions. Canonical architectures lik…
NS-GAN mode collapse due to sample weighting inversion, solved with MM-nsat.
problem Mode collapse in GANs due to sample weighting inversion.
method Preserves MM-GAN sample weighting while avoiding saturation by rescaling gradients.
result MM-nsat improves mode coverage, stability, and FID on MNIST and CIFAR-10.
Mitigating Performance Saturation in Neural Marked Point Processes: Architectures and Loss Functionscs.LG
Neural marked point processes show saturation with complexity, leading to new simple architectures.
problem Performance saturation in neural marked point processes with complex architectures.
method Proposed GCHP with graph convolutional layers and likelihood ratio loss.
result GCHP reduces training time and improves model performance.
Entrocraft addresses RL performance saturation in LLMs by customizing entropy curves.
problem Performance saturation in RL algorithms for LLMs.
method Entrocraft uses rejection sampling to bias advantage distributions for customized entropy schedules.
result Entrocraft significantly improves generalization, output diversity, and long-term training in 4B models.
Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updatescs.LG
New insights explain speedup saturation in distributed learning with large batches and delays.
problem Understanding and optimizing speedup in distributed learning with large batches and delays.
method Theoretical analysis of strongly convex, convex, and non-convex settings, considering data sparsity.
result Identification of a data-dependent parameter explaining speedup saturation in both batch size and gradient staleness.
The time development of the price of a financial asset is considered by constructing and solving Langevin equations for a homogeneously saturated model, and for comparison, for a standard model and for a logistic model. The homogeneously saturated model uses coupled rate equations for the money supply and for the price…
The closed string field theory minimal-area problem asks for the conformal metric of least area on a Riemann surface with the condition that all non-contractible closed curves have length at least 2π. Through every point in such a metric there is a geodesic that saturates the length condition, and saturating geodesics …
New theory explains GAN's high quality but low diversity.
problem Lack of theoretical justification for non-saturating GAN training.
method Showed non-saturating GAN training approximately minimizes a specific f-divergence.
result Non-saturating GAN training minimizes a particular f-divergence.
As one of standard approaches to train deep neural networks, dropout has been applied to regularize large models to avoid overfitting, and the improvement in performance by dropout has been explained as avoiding co-adaptation between nodes. However, when correlations between nodes are compared after training the networ…
Sign information is the key to overcoming the inevitable saturation error in compressive sensing systems, which causes information loss and results in bias. For sparse signal recovery from saturation, we propose to use a linear loss to improve the effectiveness from existing methods that utilize hard constraints/hinge …
We give a method to compute presentations of saturated cluster modular groups. Using this, we obtain finite presentations of the saturated cluster modular groups of finite mutation type and . We verify that the cluster modular groups of finite mutation type , , $\widetilde{E…
The paper develops a theory for iterative self-improvement of models, proving conditions for better performance with easy-to-hard curricula.
problem Lack of theoretical foundation for iterative self-improvement in practical settings.
method Modeling self-improvement as maximum-likelihood fine-tuning on reward-filtered distributions and proving finite-sample guarantees.
result Explicit feedback loop and conditions for better performance with easy-to-hard curricula.
Common nonlinear activation functions used in neural networks can cause training difficulties due to the saturation behavior of the activation function, which may hide dependencies that are not visible to vanilla-SGD (using first order gradients only). Gating mechanisms that use softly saturating activation functions t…
Much effort has been devoted to understanding the decisions of deep neural networks in recent years. A number of model-aware saliency methods were proposed to explain individual classification decisions by creating saliency maps. However, they are not applicable when the parameters and the gradients of the underlying m…
If is a compact real analytic Riemannian manifold, we give a necessary and sufficient condition for there to be a sequence of quasimodes of order saturating sup-norm estimates. In particular, it gives optimal conditions for existence of eigenfunctions satisfying maximal sup norm bounds. The condition is …
Study shows scaling up models doesn't always improve downstream tasks.
problem Understanding why scaling up models doesn't always improve downstream performance.
method Systematic study of 4800 experiments on various models, analyzing performance on 20 downstream tasks.
result Performance on downstream tasks saturates as model size increases, revealing a nonlinear relationship.
Shallow Neural Network can Perfectly Classify an Object following Separable Probability Distributioncs.LG
Guiding the design of neural networks is of great importance to save enormous resources consumed on empirical decisions of architectural parameters. This paper constructs shallow sigmoid-type neural networks that achieve 100% accuracy in classification for datasets following a linear separability condition. The separab…
Soft-Radial Projection solves gradient saturation in constrained deep learning.
problem Gradient saturation in deep learning models when integrating hard constraints.
method Introduces Soft-Radial Projection, a differentiable layer that maps predictions onto constraint boundaries without rank-deficient Jacobians.
result Improves convergence and solution quality over state-of-the-art methods.
The paper derives upper bounds on eigenvalues of Laplace-Beltrami operator on hyperbolic surfaces.
problem Finding upper bounds on eigenvalues of Laplace-Beltrami operator on hyperbolic surfaces.
method Using spectral decompositions and consistency conditions derived from quadruple overlap integrals in terms of triple overlap integrals.
result Derives upper bounds on eigenvalues, nearly saturated by the Bolza surface.
EoS selectively shapes learning, affecting some groups more than others.
problem EoS affects learning differently across the data distribution.
method Branching intervention to enter or exit EoS regime, controlled perturbation to isolate mechanisms.
result EoS redistributes learning, amplifying progress on some groups and suppressing others.
We undertake a fundamental study of network equilibria modeled as solutions of fixed point equations for monotone linear functions with saturation nonlinearities. The considered model extends one originally proposed to study systemic risk in networks of financial institutions interconnected by mutual obligations and is…
Surrogate strategies are used widely for uncertainty quantification of groundwater models in order to improve computational efficiency. However, their application to dynamic multiphase flow problems is hindered by the curse of dimensionality, the saturation discontinuity due to capillarity effects, and the time-depende…
Open, connected, saturated sets W without holonomy in codimension one foliations play key roles as fundamental building blocks. Here, for the case of foliated 3-manifolds, we produce a finite system of closed, convex, non-overlapping polyhedral cones in the first cohomology of W with real coefficients such that the iso…
This paper presents the development of a hybrid learning system based on Support Vector Machines (SVM), Adaptive Neuro-Fuzzy Inference System (ANFIS) and domain knowledge to solve prediction problem. The proposed two-stage Domain Knowledge based Fuzzy Information System (DKFIS) improves the prediction accuracy attained…
Noiseless KRR achieves optimal rates and exhibits saturation effects.
problem Understanding optimal rates and saturation phenomena in noiseless kernel ridge regression.
method Comprehensive study of noiseless KRR, establishing minimax optimal rates and uncovering phenomena of extra-smoothness and saturation.
result Noiseless KRR achieves minimax optimal rates and exhibits saturation effects.
New proof classifies orbit closures in Hodge bundle.
problem Classifying -orbit closures in Hodge bundle.
method Using deformations of flat pairs of pants.
result Short proof of absolute period foliation classification.
Investigates optimal parameter allocation in Transformers for efficiency and expressivity.
problem Balancing expressivity and efficiency in Transformer model parameters.
method Mathematical analysis and theoretical characterization of attention heads and head dimensions.
result Later layers can operate more efficiently with reduced parameters due to saturation of softmax activations.
We define the representation ring of a saturated fusion system as the Grothendieck ring of the semiring of -stable representations, and study the dimension functions of -stable representations using the transfer map induced by the characteristic idempotent of . We find a…
For a stable marginally outer trapped surface (MOTS) in an axially symmetric spacetime with cosmological constant and with matter satisfying the dominant energy condition, we prove that the area and the angular momentum satisfy the inequality which is saturated pre…
In a previous article, we defined a very flexible notion of suborbifold and characterized those suborbifolds which can arise as the images of orbifold embeddings. In particular, suborbifolds are images of orbifold embeddings precisely when they are saturated and split. This article addresses the problem of orbifold str…