arXiv research

Develops a new fuzzy model using QPs and ewl2 regularization to improve local region behavior.

We give a topological interpretation of the space of L2-harmonic forms on finite-volume manifolds with sufficiently pinched negative curvature. We give examples showing that this interpretation fails if the curvature is not sufficiently pinched and that our result is sharp with respect to the pinching constants. The me…

Batch Normalization is a commonly used trick to improve the training of deep neural networks. These neural networks use L2 regularization, also called weight decay, ostensibly to prevent overfitting. However, we show that L2 regularization has no regularizing effect when combined with normalization. Instead, regulariza…

Study on a metric on Hermitian metrics space, proving diffeomorphisms and completeness.

We investigate deep Bayesian neural networks with Gaussian weight priors and a class of ReLU-like nonlinearities. Bayesian neural networks with Gaussian priors are well known to induce an L2, "weight decay", regularization. Our results characterize a more intricate regularization effect at the level of the unit activat…

Regularization improves stability and consistency of sparse autoencoders.

Paper introduces stability in model averaging and proposes a L2-penalty method.

We give a fast oblivious L2-embedding of $A\in \mathbb{R}^{n x d}$ to $B\in \mathbb{R}^{r x d}$ satisfying $(1-\varepsilon)\|A x\|_2^2 \le \|B x\|_2^2 <= (1+\varepsilon) \|Ax\|_2^2.$ Our embedding dimension $r$ equals $d$, a constant independent of the distortion $\varepsilon$. We use as a black-box any L2-embedding $Π…

Ethereum upgrades increased TPS and lowered fees, with L2s surpassing Solana in 2029.

We examine some differential geometric approaches to finding approximate solutions to the continuous time nonlinear filtering problem. Our primary focus is a new projection method for the optimal filter infinite dimensional Stochastic Partial Differential Equation (SPDE), based on the direct L2 metric and on a family o…

Dynamic-weight AMMs outperform traditional CEX rebalancing in tokenized funds, especially on L2s.

Importance-weighted risk minimization is a key ingredient in many machine learning algorithms for causal inference, domain adaptation, class imbalance, and off-policy reinforcement learning. While the effect of importance weighting is well-characterized for low-capacity misspecified models, little is known about how it…

Distributional (or distribution-valued) data are a new type of data arising from several sources and are considered as realizations of distributional variables. A new set of fuzzy c-means algorithms for data described by distributional variables is proposed. The algorithms use the $L2$ Wasserstein distance between dist…

In this paper we introduce a projection method for the space of probability distributions based on the differential geometric approach to statistics. This method is based on a direct L2 metric as opposed to the usual Hellinger distance and the related Fisher Information metric. We explain how this apparatus can be used…

The problem of joint feature selection across a group of related tasks has applications in many areas including biomedical informatics and computer vision. We consider the l2,1-norm regularized regression model for joint feature selection from multiple tasks, which can be derived in the probabilistic framework by assum…

ESNs trained with Tikhonov least squares approximate ergodic dynamical systems in L2(μ) norm.

In this paper, we study Lichnerowicz type estimate for eigenvalues of drifting Laplacian operator and L1 and L2 energy for drifting heat equation on closed manifolds with weighted measure. In some sense, this study is about the eigenvalue estimate on Ricci solitons.

COS method convergence conditions expanded for heavy-tailed distributions.

The paper connects neural collapse and low-rank bias in networks with L2 regularization.

Forecast dam inflow using sea surface feature weights.

A standing conjecture in L2-cohomology is that every finite CW-complex X is of L2-determinant class. In this paper, we prove this whenever the fundamental group belongs to a large class of groups containing e.g. all extensions of residually finite groups with amenable quotients, all residually amenable groups and free …

In this paper, we propose a novel linear discriminant analysis criterion via the Bhattacharyya error bound estimation based on a novel L1-norm (L1BLDA) and L2-norm (L2BLDA). Both L1BLDA and L2BLDA maximize the between-class scatters which are measured by the weighted pairwise distances of class means and meanwhile mini…

Transfer learning through fine-tuning a pre-trained neural network with an extremely large dataset, such as ImageNet, can significantly accelerate training while the accuracy is frequently bottlenecked by the limited dataset size of the new target task. To solve the problem, some regularization methods, constraining th…

Extends online learning to metric spaces using exponential weights.

For a normal covering over a closed oriented topological manifold we give a proof of the L2-signature theorem with twisted coefficients, using Lipschitz structures and the Lipschitz signature operator introduced by Teleman. We also prove that the L-theory isomorphism conjecture as well as the C^*_max-version of the Bau…

Study extends holomorphic forms on noncompact Kahler manifolds.

Dropout is one of the key techniques to prevent the learning from overfitting. It is explained that dropout works as a kind of modified L2 regularization. Here, we shed light on the dropout from Bayesian standpoint. Bayesian interpretation enables us to optimize the dropout rate, which is beneficial for learning of wei…

We provide a proof for an inequality between volume and L2-Betti numbers of aspherical manifolds for which Gromov outlined a strategy based on general ideas of Connes. The implementation of that strategy involves measured equivalence relations, Gaboriau's theory of L2-Betti numbers of R-simplicial complexes, and other …

The main theme of this work is a unifying algorithm, \textbf{L}oop\textbf{L}ess \textbf{S}ARAH (L2S) for problems formulated as summation of $n$ individual loss functions. L2S broadens a recently developed variance reduction method known as SARAH. To find an $ε$-accurate solution, L2S enjoys a complexity of ${\cal O}\b…

Paper quantifies MEV on L2 networks, finding significant amounts on Polygon.

New method improves convergence of spatial filters in neural networks.

Several new properties of weighted Hilbert transform are obtained. If mu is zero, two Plancherel-like equations and the isotropic properties are derived. For mu is real number, a coerciveness is derived and two iterative sequences are constructed to find the inversion. The proposed iterative sequences are applicable to…

We find a convex model for traditional nonlinear regression under L2 loss.

We prove that L2-Boosting lacks a theoretical property which is central to the behaviour of l1-penalized methods such as basis pursuit and the Lasso: Whereas l1-penalized methods are guaranteed to recover the sparse parameter vector in a high-dimensional linear model under an appropriate restricted nullspace property, …

We introduce a covariance matrix estimator that both takes into account the heteroskedasticity of financial returns (by using an exponentially weighted moving average) and reduces the effective dimensionality of the estimation (and hence measurement noise) via techniques borrowed from random matrix theory. We calculate…

We present an Automatic Relevance Determination prior Bayesian Neural Network(BNN-ARD) weight l2-norm measure as a feature importance statistic for the model-x knockoff filter. We show on both simulated data and the Norwegian wind farm dataset that the proposed feature importance statistic yields statistically signific…

New L2 regularization improves softmax MAB performance.

Extends L2-norm LDA to 2D inputs using Bhattacharyya bound.

We examine the effect of the Group Lasso (gLasso) regularizer in selecting the salient nodes of Deep Neural Network (DNN) hidden layers by applying a DNN-HMM hybrid speech recognizer to TED Talks speech data. We test two types of gLasso regularization, one for outgoing weight vectors and another for incoming weight vec…

A new method for exponentially weighted moving models using approximations.

This work proposes a new algorithm for training a re-weighted L2 Support Vector Machine (SVM), inspired on the re-weighted Lasso algorithm of Candès et al. and on the equivalence between Lasso and SVM shown recently by Jaggi. In particular, the margin required for each training vector is set independently, defining a n…

Introduces Exponentially Weighted Signature for better path representation.

In this paper, we study the evolution of L2 p-forms under Ricci flow with bounded curvature on a complete non-compact or a compact Riemannian manifold. We show that under curvature pinching conditions on such a manifold, the L2 norm of a smooth p-form is non-increasing along the Ricci flow. The L^{\infty} norm is showe…

Adversarial training makes logistic regression weight loss landscapes sharper.

Study on L2-boosting behavior as learning rate approaches zero.

In this paper, we consider one dimensional (shallow) ReLU neural networks in which weights are chosen randomly and only the terminal layer is trained. First, we mathematically show that for such networks L2-regularized regression corresponds in function space to regularizing the estimate's second derivative for fairly …

This paper identifies a problem with the usual procedure for L2-regularization parameter estimation in a domain adaptation setting. In such a setting, there are differences between the distributions generating the training data (source domain) and the test data (target domain). The usual cross-validation procedure requ…

Improves deep learning performance on noisy datasets using inverse-variance weighting.