arXiv research

We find a convex model for traditional nonlinear regression under L2 loss.

Paper proves L2 regression can learn k-juntas without distributional assumptions.

We present a generalization of the Cauchy/Lorentzian, Geman-McClure, Welsch/Leclerc, generalized Charbonnier, Charbonnier/pseudo-Huber/L1-L2, and L2 loss functions. By introducing robustness as a continuous parameter, our loss function allows algorithms built around robust loss minimization to be generalized, which imp…

We study a family of sparse estimators defined as minimizers of some empirical Lipschitz loss function -- which include the hinge loss, the logistic loss and the quantile regression loss -- with a convex, sparse or group-sparse regularization. In particular, we consider the L1 norm on the coefficients, its sorted Slope…

The main theme of this work is a unifying algorithm, \textbf{L}oop\textbf{L}ess \textbf{S}ARAH (L2S) for problems formulated as summation of $n$ individual loss functions. L2S broadens a recently developed variance reduction method known as SARAH. To find an $ε$-accurate solution, L2S enjoys a complexity of ${\cal O}\b…

Improved sample efficiency in learning sparse Ising models.

Conjugate gradient (CG) methods are a class of important methods for solving linear equations and nonlinear optimization problems. In this paper, we propose a new stochastic CG algorithm with variance reduction and we prove its linear convergence with the Fletcher and Reeves method for strongly convex and smooth functi…

In this paper we study deep learning-based music source separation, and explore using an alternative loss to the standard spectrogram pixel-level L2 loss for model training. Our main contribution is in demonstrating that adding a high-level feature loss term, extracted from the spectrograms using a VGG net, can improve…

A standing conjecture in L2-cohomology is that every finite CW-complex X is of L2-determinant class. In this paper, we prove this whenever the fundamental group belongs to a large class of groups containing e.g. all extensions of residually finite groups with amenable quotients, all residually amenable groups and free …

Feature normalization prevents collapse in non-contrastive learning dynamics.

For a normal covering over a closed oriented topological manifold we give a proof of the L2-signature theorem with twisted coefficients, using Lipschitz structures and the Lipschitz signature operator introduced by Teleman. We also prove that the L-theory isomorphism conjecture as well as the C^*_max-version of the Bau…

Study extends holomorphic forms on noncompact Kahler manifolds.

We provide a proof for an inequality between volume and L2-Betti numbers of aspherical manifolds for which Gromov outlined a strategy based on general ideas of Connes. The implementation of that strategy involves measured equivalence relations, Gaboriau's theory of L2-Betti numbers of R-simplicial complexes, and other …

Paper quantifies MEV on L2 networks, finding significant amounts on Polygon.

We give a fast oblivious L2-embedding of $A\in \mathbb{R}^{n x d}$ to $B\in \mathbb{R}^{r x d}$ satisfying $(1-\varepsilon)\|A x\|_2^2 \le \|B x\|_2^2 <= (1+\varepsilon) \|Ax\|_2^2.$ Our embedding dimension $r$ equals $d$, a constant independent of the distortion $\varepsilon$. We use as a black-box any L2-embedding $Π…

We give a topological interpretation of the space of L2-harmonic forms on finite-volume manifolds with sufficiently pinched negative curvature. We give examples showing that this interpretation fails if the curvature is not sufficiently pinched and that our result is sharp with respect to the pinching constants. The me…

We prove that L2-Boosting lacks a theoretical property which is central to the behaviour of l1-penalized methods such as basis pursuit and the Lasso: Whereas l1-penalized methods are guaranteed to recover the sparse parameter vector in a high-dimensional linear model under an appropriate restricted nullspace property, …

New L2 regularization improves softmax MAB performance.

Adversarial training makes logistic regression weight loss landscapes sharper.

The use of machine-learning in neuroimaging offers new perspectives in early diagnosis and prognosis of brain diseases. Although such multivariate methods can capture complex relationships in the data, traditional approaches provide irregular (l2 penalty) or scattered (l1 penalty) predictive pattern with a very limited…

Recently, fully-connected and convolutional neural networks have been trained to achieve state-of-the-art performance on a wide variety of tasks such as speech recognition, image classification, natural language processing, and bioinformatics. For classification tasks, most of these "deep learning" models employ the so…

In this paper, we study the evolution of L2 p-forms under Ricci flow with bounded curvature on a complete non-compact or a compact Riemannian manifold. We show that under curvature pinching conditions on such a manifold, the L2 norm of a smooth p-form is non-increasing along the Ricci flow. The L^{\infty} norm is showe…

Gradient boosting improved with lassoed trees achieves faster convergence.

We propose SEARNN, a novel training algorithm for recurrent neural networks (RNNs) inspired by the "learning to search" (L2S) approach to structured prediction. RNNs have been widely successful in structured prediction applications such as machine translation or parsing, and are commonly trained using maximum likelihoo…

Dynamic-weight AMMs outperform traditional CEX rebalancing in tokenized funds, especially on L2s.

Study on L2-boosting behavior as learning rate approaches zero.

Transfer learning through fine-tuning a pre-trained neural network with an extremely large dataset, such as ImageNet, can significantly accelerate training while the accuracy is frequently bottlenecked by the limited dataset size of the new target task. To solve the problem, some regularization methods, constraining th…

Ethereum upgrades increased TPS and lowered fees, with L2s surpassing Solana in 2029.

We examine some differential geometric approaches to finding approximate solutions to the continuous time nonlinear filtering problem. Our primary focus is a new projection method for the optimal filter infinite dimensional Stochastic Partial Differential Equation (SPDE), based on the direct L2 metric and on a family o…

A vanishing theorem for a convex cocompact hyperbolic manifold is established, which relates the L2 cohomology to the Hausdorff dimension of the limit set. The borderline case is shown to characterize the manifold completely.

A new Branch-and-Bound solver tackles L0-penalized problems with flexible loss functions.

In this article, we study the L2-transverse conformal Killing forms on complete foliated Riemannian manifolds and prove some vanishing theorems. Also, we study the same problems on Kahler foliations with a complete bundle-like metric.

Study shows pre-event L2 liquidity state predicts crypto futures liquidity better than event labels.

Study on a metric on Hermitian metrics space, proving diffeomorphisms and completeness.

In this paper, we establish various L2-estimates for the exterior differential operator on p-convex Riemannian manifolds in the sense of Harvey and Lawson. As geometric applications, we prove vanishing and finiteness results for the de Rham cohomology groups.

New characterizations of curvature operators for specific forms via L2-estimates.

We study a policy gradient method with L2 regularization for MAB problems.

RTC-GTNLN model recovers traffic data from missing values and noise.

The performance of the state-of-the-art image segmentation methods heavily relies on the high-quality annotations, which are not easily affordable, particularly for medical data. To alleviate this limitation, in this study, we propose a weakly supervised image segmentation method based on a deep geodesic prior. We hypo…

Batch Normalization is a commonly used trick to improve the training of deep neural networks. These neural networks use L2 regularization, also called weight decay, ostensibly to prevent overfitting. However, we show that L2 regularization has no regularizing effect when combined with normalization. Instead, regulariza…

Study computability of real numbers from group properties.

For one-parameter degenerations of compact Kähler manifolds, we determine the asymptotic behavior of the first Chern form of the direct image of a Nakano semi-positive vector bundle twisted by the relative canonical bundle, when the direct image is equipped with the L2-metric.

In the previous papers \cite{L1, L2} the author constructed Mabuchi and Aubin-Yau functionals over any complex surfaces and three-folds, respectively. Using the method in \cite{L2}, we construct those functionals over any complex manifolds of the complex dimension bigger than or equal to 2.

Algorithm optimizes quantized isotonic regression with log-linear time updates.

In this paper, we analyze efficacy of the fast gradient sign method (FGSM) and the Carlini-Wagner's L2 (CW-L2) attack. We prove that, within a certain regime, the untargeted FGSM can fool any convolutional neural nets (CNNs) with ReLU activation; the targeted FGSM can mislead any CNNs with ReLU activation to classify a…

Superposition in autoencoders leads to loss in simple models.

We show that the Novikov-Shubin invariant of an element of the integral group ring of the lamplighter group Z_2 \wr Z can be irrational. This disproves a conjecture of Lott and Lueck. Furthermore we show that every positive real number is equal to the Novikov-Shubin invariant of some element of the real group ring of Z…

Karen Uhlenbeck's compactness theorem for sequences of connections with L2 bounds on curvature applies only to connections on principal bundles with compact structure group. This article states and proves an extension of Uhlenbecks theorem that describes sequences of connections on principal PSL(2;C) bundles over compa…