New method resolves density ratio estimation saturation issues.
problem Error saturation in density ratio estimation methods.
method Iterated regularization to improve kernel methods.
result Achieves fast error rates on regular learning problems.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “saturation error”
New method recovers signals from saturated data using linear loss and nonconvex penalties.
problem Signal recovery from saturated measurements with sign information loss.
method Linear loss and nonconvex penalties (e.g., minimax concave penalty, sorted ℓ1 norm).
result Estimation error is bounded and recovery performance improved.
We consider the reconstruction problem in compressed sensing in which the observations are recorded in a finite number of bits. They may thus contain quantization errors (from being rounded to the nearest representable value) and saturation errors (from being outside the range of representable values). Our formulation …
This paper presents the development of a hybrid learning system based on Support Vector Machines (SVM), Adaptive Neuro-Fuzzy Inference System (ANFIS) and domain knowledge to solve prediction problem. The proposed two-stage Domain Knowledge based Fuzzy Information System (DKFIS) improves the prediction accuracy attained…
The paper extends kernel ridge regression to product kernels and reveals new convergence behaviors.
problem Understanding kernel ridge regression in large dimensions with various kernels.
method Established a broad family of large dimensional kernels and derived convergence rates.
result Revealed new phenomena including minimax optimality, saturation effect, and multiple descent behavior.
Paper proves KRR saturation effect for smooth functions.
problem Kernel ridge regression fails to reach theoretical limits for smooth functions.
method Proof of conjectured saturation lower bound for KRR.
result Proved the conjectured saturation lower bound for KRR.
Investigates optimal parameter allocation in Transformers for efficiency and expressivity.
problem Balancing expressivity and efficiency in Transformer model parameters.
method Mathematical analysis and theoretical characterization of attention heads and head dimensions.
result Later layers can operate more efficiently with reduced parameters due to saturation of softmax activations.
On the exact minimization of saturated loss functions for robust regression and subspace estimationcs.LG
This paper deals with robust regression and subspace estimation and more precisely with the problem of minimizing a saturated loss function. In particular, we focus on computational complexity issues and show that an exact algorithm with polynomial time-complexity with respect to the number of data can be devised for r…
A new metric measures saturation of neural network layers.
problem Analyzing the quality of latent representations in neural networks.
method Layer Saturation metric based on spectral analysis.
result Saturation is related to generalization and predictive performance.
A method to analyze neural network performance by measuring layer saturation.
problem Understanding which layers contribute to network performance.
method Layer saturation method: restricts layer output to eigenspace of variance matrix.
result Layer saturation indicates which layers contribute to network performance.
Noiseless KRR achieves optimal rates and exhibits saturation effects.
problem Understanding optimal rates and saturation phenomena in noiseless kernel ridge regression.
method Comprehensive study of noiseless KRR, establishing minimax optimal rates and uncovering phenomena of extra-smoothness and saturation.
result Noiseless KRR achieves minimax optimal rates and exhibits saturation effects.
We extend the adaptive regression spline model by incorporating saturation, the natural requirement that a function extend as a constant outside a certain range. We fit saturating splines to data using a convex optimization problem over a space of measures, which we solve using an efficient algorithm based on the condi…
Mitigating Performance Saturation in Neural Marked Point Processes: Architectures and Loss Functionscs.LG
Neural marked point processes show saturation with complexity, leading to new simple architectures.
problem Performance saturation in neural marked point processes with complex architectures.
method Proposed GCHP with graph convolutional layers and likelihood ratio loss.
result GCHP reduces training time and improves model performance.
Kernel quadrature with DPPsstat.ML
Kernel quadrature uses DPPs for sampling with tight error bounds.
problem Efficiently sampling nodes for quadrature rules in RKHS.
method Nodes sampled from a truncated and saturated DPP kernel.
result Tighter quadrature error bounds using DPPs.
Optimal AFs minimize RFR test error and sensitivity.
problem Finding optimal AFs for RFR to minimize test error and sensitivity.
method Closed-form solution for AFs minimizing test error and sensitivity under different functional parsimony.
result Optimal AFs can be linear, saturated linear, or Hermite polynomial expressions.
Learning Shrinks the Hard Tail: Training-Dependent Inference Scaling in a Solvable Linear Modelcs.LG
Learning shrinks hard tail, improving inference performance.
problem Improving inference performance in neural networks.
method Latent Instance Difficulty (LID) model analyzing fine-tuning of neural networks.
result Training-dependent inference scaling, with growing with sample size before saturating.
The paper normalizes Poisson saturation of coregular submanifolds.
problem Normalizing the Poisson saturation of coregular submanifolds.
method Normal form construction and Poisson geometry analysis.
result Local Poisson saturation of coregular submanifolds is an embedded Poisson submanifold with a normal form.
A new recurrent unit alleviates vanishing gradients for long-term dependencies.
problem Vanishing gradients in recurrent neural networks make long-term dependencies hard to model.
method Proposes a new NRU architecture that avoids saturating activation functions and gates.
result Demonstrates superior performance across various tasks with and without long-term dependencies.
Accelerating Non-Negative and Bounded-Variable Linear Regression Algorithms with Safe Screeningcs.LG
The paper accelerates regression algorithms by identifying saturated coordinates.
problem Non-negative and bounded-variable linear regression problems.
method Safe screening technique to identify saturated coordinates.
result The approach provides theoretical guarantees for identifying saturated coordinates.
A homogeneously saturated equation for the time development of the price of a financial asset is presented and investigated for the pricing of European call options using noise that is distributed as a Student's t-distribution. In the limit that the saturation parameter of the equation equals zero, the standard model o…
33 curves on a 3-genus surface, all intersecting at most once.
problem Finding a saturated system of curves on a surface of genus 3.
method Constructing 33 essential curves pairwise non-homotopic and intersecting at most once.
result The constructed system is saturated, not properly contained in any other system.
Contextual PDA improves explanation of image classifications for saturated models.
problem Difficulty in explaining decisions of saturated classifiers.
method Proposes Contextual PDA, a faster method for explaining image classifications.
result Contextual PDA outperforms PDA in explaining image classifications of state-of-the-art deep networks.
Study reveals how depth of reasoning affects generalization in models.
problem Understanding scaling behavior of generalization with CoT depth.
method Theoretical model of CoT in linear regression using random matrix theory.
result Sharp phase transition between exponential and polynomial improvement, saturation, and overthinking.
This paper explores saturation effects in spectral algorithms over large dimensions.
problem Saturation effects in spectral algorithms over large dimensions.
method Improved minimax lower bound and gradient flow with early stopping strategy.
result Exact convergence rates of spectral algorithms in large dimensional settings.
Water saturation is an important property in reservoir engineering domain. Thus, satisfactory classification of water saturation from seismic attributes is beneficial for reservoir characterization. However, diverse and non-linear nature of subsurface attributes makes the classification task difficult. In this context,…
A recent paper suggests that Deep Neural Networks can be protected from gradient-based adversarial perturbations by driving the network activations into a highly saturated regime. Here we analyse such saturated networks and show that the attacks fail due to numerical limitations in the gradient computations. A simple s…
Paper develops an online learning algorithm for functional data models.
problem Recovering slope functions or predictors in functional data models.
method Online regularized learning algorithm in reproducing kernel Hilbert spaces with polynomially decaying step-size.
result Established fast convergence rates for estimation error without capacity assumption.
NS-GAN mode collapse due to sample weighting inversion, solved with MM-nsat.
problem Mode collapse in GANs due to sample weighting inversion.
method Preserves MM-GAN sample weighting while avoiding saturation by rescaling gradients.
result MM-nsat improves mode coverage, stability, and FID on MNIST and CIFAR-10.
Proposes QEP to mitigate quantization error propagation in layer-wise post-training quantization.
problem Growth of quantization errors across layers degrades performance, especially in low-bit regimes.
method Quantization Error Propagation (QEP) framework that explicitly propagates and compensates for quantization errors.
result QEP-enhanced layer-wise PTQ achieves substantially higher accuracy, especially in low-bit regimes.
A new method detects and compacts saturated entries in antisparse coding.
problem Efficiently solving antisparse coding problems with -norm penalties.
method Safe squeezing methodology to detect and compact saturated entries, reducing problem dimensionality.
result The method accelerates the computation of antisparse representation by detecting and compacting saturated entries.
Entrocraft addresses RL performance saturation in LLMs by customizing entropy curves.
problem Performance saturation in RL algorithms for LLMs.
method Entrocraft uses rejection sampling to bias advantage distributions for customized entropy schedules.
result Entrocraft significantly improves generalization, output diversity, and long-term training in 4B models.
Curiosity-Critic improves world model training by focusing on cumulative prediction error.
problem Training world models with intrinsic rewards that consider cumulative prediction error.
method Curiosity-Critic uses a surrogate reward based on the difference between current and asymptotic prediction errors, estimated online by a co-trained critic.
result Curiosity-Critic outperforms other methods in training speed and final world model accuracy.
Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updatescs.LG
New insights explain speedup saturation in distributed learning with large batches and delays.
problem Understanding and optimizing speedup in distributed learning with large batches and delays.
method Theoretical analysis of strongly convex, convex, and non-convex settings, considering data sparsity.
result Identification of a data-dependent parameter explaining speedup saturation in both batch size and gradient staleness.
The time development of the price of a financial asset is considered by constructing and solving Langevin equations for a homogeneously saturated model, and for comparison, for a standard model and for a logistic model. The homogeneously saturated model uses coupled rate equations for the money supply and for the price…
New theory explains GAN's high quality but low diversity.
problem Lack of theoretical justification for non-saturating GAN training.
method Showed non-saturating GAN training approximately minimizes a specific f-divergence.
result Non-saturating GAN training minimizes a particular f-divergence.
A general nonlinear logistic equation has been proposed to model long-time saturation in industrial growth. An integral solution of this equation has been derived for any arbitrary degree of nonlinearity. A time scale for the onset of nonlinear saturation in industrial growth can be estimated from an equipartition cond…
As one of standard approaches to train deep neural networks, dropout has been applied to regularize large models to avoid overfitting, and the improvement in performance by dropout has been explained as avoiding co-adaptation between nodes. However, when correlations between nodes are compared after training the networ…
We give a method to compute presentations of saturated cluster modular groups. Using this, we obtain finite presentations of the saturated cluster modular groups of finite mutation type and . We verify that the cluster modular groups of finite mutation type , , $\widetilde{E…
Common nonlinear activation functions used in neural networks can cause training difficulties due to the saturation behavior of the activation function, which may hide dependencies that are not visible to vanilla-SGD (using first order gradients only). Gating mechanisms that use softly saturating activation functions t…
New examples show deletion type admissible pairs can be rigid under rational saturation.
problem Rigidity of admissible pairs of rational homogeneous spaces of Picard number one.
method Application of Mok's general criterion for non-subdiagram type admissible pairs.
result Examples of deletion type admissible pairs are rigid under rational saturation.
Study network equilibria in saturated systems, revealing how small shocks can trigger major losses.
problem Understanding how small shocks can lead to major losses in financial networks and games.
method Derived explicit expressions for network equilibria, proved conditions for their uniqueness, and analyzed discontinuities.
result Bifurcation phenomenon in network equilibria, showing sensitivity to small shocks.
Study shows scaling up models doesn't always improve downstream tasks.
problem Understanding why scaling up models doesn't always improve downstream performance.
method Systematic study of 4800 experiments on various models, analyzing performance on 20 downstream tasks.
result Performance on downstream tasks saturates as model size increases, revealing a nonlinear relationship.
Paper explores how combining tail-averaging and minibatching improves SGD convergence.
problem Understanding and optimizing learning properties of SGD variants.
method Least squares learning in a nonparametric setting, focusing on multiple passes, mini-batching, and averaging.
result Tail averaging allows faster convergence rates than uniform averaging in nonparametric settings.
Soft-Radial Projection solves gradient saturation in constrained deep learning.
problem Gradient saturation in deep learning models when integrating hard constraints.
method Introduces Soft-Radial Projection, a differentiable layer that maps predictions onto constraint boundaries without rank-deficient Jacobians.
result Improves convergence and solution quality over state-of-the-art methods.
New methods show deep networks can compress info without saturating activations.
problem Understanding how neural networks generalize and compress information.
method Adaptive mutual information estimation techniques for neural networks.
result Compression occurs in networks with non-saturating activation functions.
Study reveals how model volume affects learning curves in machine learning.
problem Understanding the double descent risk phenomenon in machine learning.
method Investigates the role of model volume using MDL, Occam's Razor, and information geometry.
result Model volume can explain the double descent risk, suggesting better generalization with increased dimensionality.
The study analyzes spectral algorithms for kernel methods and derives generalization error.
problem Estimating generalization error of spectral algorithms for kernel methods.
method Considered spectral algorithms including KRR and GD, derived generalization error as a functional of learning profile.
result Showed the loss localizes on certain spectral scales and conjectured universality of the loss for noisy observations.
New formulations for Ricci flows without smoothness.
problem Characterize Ricci flows without smooth solutions.
method Weak formulations of super Ricci flows with saturation condition.
result Generalized formulations for singular settings.