arXiv research

Paper quantifies MEV on L2 networks, finding significant amounts on Polygon.

The main theme of this work is a unifying algorithm, \textbf{L}oop\textbf{L}ess \textbf{S}ARAH (L2S) for problems formulated as summation of $n$ individual loss functions. L2S broadens a recently developed variance reduction method known as SARAH. To find an $ε$-accurate solution, L2S enjoys a complexity of ${\cal O}\b…

Batch Normalization is a commonly used trick to improve the training of deep neural networks. These neural networks use L2 regularization, also called weight decay, ostensibly to prevent overfitting. However, we show that L2 regularization has no regularizing effect when combined with normalization. Instead, regulariza…

Ethereum upgrades increased TPS and lowered fees, with L2s surpassing Solana in 2029.

In this paper, we analyze efficacy of the fast gradient sign method (FGSM) and the Carlini-Wagner's L2 (CW-L2) attack. We prove that, within a certain regime, the untargeted FGSM can fool any convolutional neural nets (CNNs) with ReLU activation; the targeted FGSM can mislead any CNNs with ReLU activation to classify a…

We present a novel view that unifies two frameworks that aim to solve sequential prediction problems: learning to search (L2S) and recurrent neural networks (RNN). We point out equivalences between elements of the two frameworks. By complementing what is missing from one framework comparing to the other, we introduce a…

ESNs trained with Tikhonov least squares approximate ergodic dynamical systems in L2(μ) norm.

A standing conjecture in L2-cohomology is that every finite CW-complex X is of L2-determinant class. In this paper, we prove this whenever the fundamental group belongs to a large class of groups containing e.g. all extensions of residually finite groups with amenable quotients, all residually amenable groups and free …

For a normal covering over a closed oriented topological manifold we give a proof of the L2-signature theorem with twisted coefficients, using Lipschitz structures and the Lipschitz signature operator introduced by Teleman. We also prove that the L-theory isomorphism conjecture as well as the C^*_max-version of the Bau…

Study extends holomorphic forms on noncompact Kahler manifolds.

Deep neural networks (DNNs) are known vulnerable to adversarial attacks. That is, adversarial examples, obtained by adding delicately crafted distortions onto original legal inputs, can mislead a DNN to classify them as any target labels. In a successful adversarial attack, the targeted mis-classification should be ach…

We provide a proof for an inequality between volume and L2-Betti numbers of aspherical manifolds for which Gromov outlined a strategy based on general ideas of Connes. The implementation of that strategy involves measured equivalence relations, Gaboriau's theory of L2-Betti numbers of R-simplicial complexes, and other …

We give a fast oblivious L2-embedding of $A\in \mathbb{R}^{n x d}$ to $B\in \mathbb{R}^{r x d}$ satisfying $(1-\varepsilon)\|A x\|_2^2 \le \|B x\|_2^2 <= (1+\varepsilon) \|Ax\|_2^2.$ Our embedding dimension $r$ equals $d$, a constant independent of the distortion $\varepsilon$. We use as a black-box any L2-embedding $Π…

We give a topological interpretation of the space of L2-harmonic forms on finite-volume manifolds with sufficiently pinched negative curvature. We give examples showing that this interpretation fails if the curvature is not sufficiently pinched and that our result is sharp with respect to the pinching constants. The me…

We find a convex model for traditional nonlinear regression under L2 loss.

We prove that L2-Boosting lacks a theoretical property which is central to the behaviour of l1-penalized methods such as basis pursuit and the Lasso: Whereas l1-penalized methods are guaranteed to recover the sparse parameter vector in a high-dimensional linear model under an appropriate restricted nullspace property, …

Study ablated data augmentation techniques and their mathematical equivalence to penalties.

Regularization plays an important role in generalization of deep neural networks, which are often prone to overfitting with their numerous parameters. L1 and L2 regularizers are common regularization tools in machine learning with their simplicity and effectiveness. However, we observe that imposing strong L1 or L2 reg…

New L2 regularization improves softmax MAB performance.

Paper investigates Lipschitz constants of self-attention modules in neural networks.

Robustness is an increasingly important property of machine learning models as they become more and more prevalent. We propose a defense against adversarial examples based on a k-nearest neighbor (kNN) on the intermediate activation of neural networks. Our scheme surpasses state-of-the-art defenses on MNIST and CIFAR-1…

In this paper, we study the evolution of L2 p-forms under Ricci flow with bounded curvature on a complete non-compact or a compact Riemannian manifold. We show that under curvature pinching conditions on such a manifold, the L2 norm of a smooth p-form is non-increasing along the Ricci flow. The L^{\infty} norm is showe…

Wide neural networks can benefit from multi-task learning in their infinite-width limit.

Transfer learning through fine-tuning a pre-trained neural network with an extremely large dataset, such as ImageNet, can significantly accelerate training while the accuracy is frequently bottlenecked by the limited dataset size of the new target task. To solve the problem, some regularization methods, constraining th…

Double descent risk in L2-regularized models explained and mitigated.

Regularization improves stability and consistency of sparse autoencoders.

Study on L2-boosting behavior as learning rate approaches zero.

We examine some differential geometric approaches to finding approximate solutions to the continuous time nonlinear filtering problem. Our primary focus is a new projection method for the optimal filter infinite dimensional Stochastic Partial Differential Equation (SPDE), based on the direct L2 metric and on a family o…

We present a generalization of the Cauchy/Lorentzian, Geman-McClure, Welsch/Leclerc, generalized Charbonnier, Charbonnier/pseudo-Huber/L1-L2, and L2 loss functions. By introducing robustness as a continuous parameter, our loss function allows algorithms built around robust loss minimization to be generalized, which imp…

A vanishing theorem for a convex cocompact hyperbolic manifold is established, which relates the L2 cohomology to the Hausdorff dimension of the limit set. The borderline case is shown to characterize the manifold completely.

In this article, we study the L2-transverse conformal Killing forms on complete foliated Riemannian manifolds and prove some vanishing theorems. Also, we study the same problems on Kahler foliations with a complete bundle-like metric.

Study shows pre-event L2 liquidity state predicts crypto futures liquidity better than event labels.

Study on a metric on Hermitian metrics space, proving diffeomorphisms and completeness.

In this paper, we establish various L2-estimates for the exterior differential operator on p-convex Riemannian manifolds in the sense of Harvey and Lawson. As geometric applications, we prove vanishing and finiteness results for the de Rham cohomology groups.

New characterizations of curvature operators for specific forms via L2-estimates.

Importance-weighted risk minimization is a key ingredient in many machine learning algorithms for causal inference, domain adaptation, class imbalance, and off-policy reinforcement learning. While the effect of importance weighting is well-characterized for low-capacity misspecified models, little is known about how it…

We study a policy gradient method with L2 regularization for MAB problems.

RTC-GTNLN model recovers traffic data from missing values and noise.

Study computability of real numbers from group properties.

For one-parameter degenerations of compact Kähler manifolds, we determine the asymptotic behavior of the first Chern form of the direct image of a Nakano semi-positive vector bundle twisted by the relative canonical bundle, when the direct image is equipped with the L2-metric.

In the previous papers \cite{L1, L2} the author constructed Mabuchi and Aubin-Yau functionals over any complex surfaces and three-folds, respectively. Using the method in \cite{L2}, we construct those functionals over any complex manifolds of the complex dimension bigger than or equal to 2.

Algorithm optimizes quantized isotonic regression with log-linear time updates.

MACER trains robust models without adversarial training, faster and more effective.

Paper proves L2 regression can learn k-juntas without distributional assumptions.

We show that the Novikov-Shubin invariant of an element of the integral group ring of the lamplighter group Z_2 \wr Z can be irrational. This disproves a conjecture of Lott and Lueck. Furthermore we show that every positive real number is equal to the Novikov-Shubin invariant of some element of the real group ring of Z…

Karen Uhlenbeck's compactness theorem for sequences of connections with L2 bounds on curvature applies only to connections on principal bundles with compact structure group. This article states and proves an extension of Uhlenbecks theorem that describes sequences of connections on principal PSL(2;C) bundles over compa…

Bayesian neural network improves feature selection and prediction.

Let G be a torsion free discrete group and let \bar{Q} denote the field of algebraic numbers in C. We prove that \bar{Q}[G] fulfills the Atiyah conjecture if G lies in a certain class of groups D, which contains in particular all groups which are residually torsion free elementary amenable or which are residually free.…