A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New algorithms reduce regret for convex bandits with small comparator norms.
New algorithms adapt to both gradient norms and comparator norms in online learning.
A new algorithm estimates mean adaptively to covariance, faster and more flexible than existing methods.
This work is substituted by the paper in arXiv:2011.14066. Stochastic gradient descent is the de facto algorithm for training deep neural networks (DNNs). Despite its popularity, it still requires fine tuning in order to achieve its best performance. This has led to the development of adaptive methods, that claim autom…
We prove that the norm version of the adaptive stochastic gradient method (AdaGrad-Norm) achieves a linear convergence rate for a subset of either strongly convex functions or non-convex functions that satisfy the Polyak Lojasiewicz (PL) inequality. The paper introduces the notion of Restricted Uniform Inequality of Gr…
Algorithm reduces regret in online learning with varying norms.
Proposes a new adaptive gradient method based on gradient differences.
We propose an adaptive optimization method for deep learning that dynamically adjusts batch size.
We consider the empirical risk minimization problem for linear supervised learning, with regularization by structured sparsity-inducing norms. These are defined as sums of Euclidean norms on certain subsets of variables, extending the usual -norm and the group -norm by allowing the subsets to overlap. T…
We develop a novel family of algorithms for the online learning setting with regret against any data sequence bounded by the empirical Rademacher complexity of that sequence. To develop a general theory of when this type of adaptive regret bound is achievable we establish a connection to the theory of decoupling inequa…
New methods improve online matrix optimization with reduced computational cost.
Kernel adaptive filters, a class of adaptive nonlinear time-series models, are known by their ability to learn expressive autoregressive patterns from sequential data. However, for trivial monotonic signals, they struggle to perform accurate predictions and at the same time keep computational complexity within desired …
We propose two sparsity-aware normalized subband adaptive filter (NSAF) algorithms by using the gradient descent method to minimize a combination of the original NSAF cost function and the l1-norm penalty function on the filter coefficients. This l1-norm penalty exploits the sparsity of a system in the coefficients upd…
New algorithm tackles high-dimensional contextual bandits without sparsity.
Adaptive algorithm AMSGrad converges for weakly convex constrained optimization problems.
New nonconvex regularizer speeds up low-rank matrix completion.
We analyze the computational limits of LoRA for transformer models using fine-grained complexity theory.
Using the -norm to regularize the estimation of the parameter vector of a linear model leads to an unstable estimator when covariates are highly correlated. In this paper, we introduce a new penalty function which takes into account the correlation of the design matrix to stabilize the estimation. This norm, ca…
New algorithm bounds MAB regret for unknown scale and magnitude of losses.
Improved online PCA algorithm learns from evolving norm of parameter vector.
In this paper, we present GASG21 (Grassmannian Adaptive Stochastic Gradient for norm minimization), an adaptive stochastic gradient algorithm to robustly recover the low-rank subspace from a large matrix. In the presence of column outliers, we reformulate the batch mode matrix norm minimization with…
This paper proposes a new method for estimating sparse precision matrices in the high dimensional setting. It has been popular to study fast computation and adaptive procedures for this problem. We propose a novel approach, called Sparse Column-wise Inverse Operator, to address these two issues. We analyze an adaptive …
AdaGrad-Norm achieves optimal convergence rates for non-convex objectives without tuning.
This paper explores adaptive methods in over-parameterized linear regression.
In this paper, we propose a novel linear discriminant analysis criterion via the Bhattacharyya error bound estimation based on a novel L1-norm (L1BLDA) and L2-norm (L2BLDA). Both L1BLDA and L2BLDA maximize the between-class scatters which are measured by the weighted pairwise distances of class means and meanwhile mini…
Theoretical framework for neural network compression using sparsity norms.
To recover a sparse signal from an underdetermined system, we often solve a constrained L1-norm minimization problem. In many cases, the signal sparsity and the recovery performance can be further improved by replacing the L1 norm with a "weighted" L1 norm. Without any prior information about nonzero elements of the si…
The paper proposes an efficient algorithm for solving Schatten- quasi-norm problems.
Efficient solver for nonconvex tensor regularization reduces computational cost.
New algorithm reduces regret from sqrt(T) to polylog(T) in stochastic contextual linear bandits.
We aim to design adaptive online learning algorithms that take advantage of any special structure that might be present in the learning task at hand, with as little manual tuning by the user as possible. A fundamental obstacle that comes up in the design of such adaptive algorithms is to calibrate a so-called step-size…
A l1-norm penalized orthogonal forward regression (l1-POFR) algorithm is proposed based on the concept of leaveone- out mean square error (LOOMSE). Firstly, a new l1-norm penalized cost function is defined in the constructed orthogonal space, and each orthogonal basis is associated with an individually tunable regulari…
In this paper, a new definition of tensor p-shrinkage nuclear norm (p-TNN) is proposed based on tensor singular value decomposition (t-SVD). In particular, it can be proved that p-TNN is a better approximation of the tensor average rank than the tensor nuclear norm when p < 1. Therefore, by employing the p-shrinkage nu…
Max-norm regularizer has been extensively studied in the last decade as it promotes an effective low-rank estimation for the underlying data. However, such max-norm regularized problems are typically formulated and solved in a batch manner, which prevents it from processing big data due to possible memory budget. In th…
New bounds adaptively control spectral complexity of trained Transformers.
The -nearest neighbour (-NN) classifier is one of the oldest and most important supervised learning algorithms for classifying datasets. Traditionally the Euclidean norm is used as the distance for the -NN classifier. In this thesis we investigate the use of alternative distances for the -NN classifier. We …
We show how to take any two parameter-free online learning algorithms with different regret guarantees and obtain a single algorithm whose regret is the minimum of the two base algorithms. Our method is embarrassingly simple: just add the iterates. This trick can generate efficient algorithms that adapt to many norms s…
Private algorithms adapt from public to private domains with minimal labeled data.
New algorithm improves tensor completion performance.
A new method for learning function parameters in operators using data-adaptive RKHS.
Proposes an algorithm for infinite-dimensional sparse learning in system identification.
Proposes a new regression method using -norms for non-Gaussian noise.
New algorithm improves online learning with reduced discretization.
Optimistic method adapted for faster convex-concave min-max problems.
New method certifies neural network function space norms from point evaluations.
We consider variational inequalities coming from monotone operators, a setting that includes convex minimization and convex-concave saddle-point problems. We assume an access to potentially noisy unbiased values of the monotone operators and assess convergence through a compatible gap function which corresponds to the …
This work proposes an adaptive trace lasso regularized L1-norm based graph cut method for dimensionality reduction of Hyperspectral images, called as `Trace Lasso-L1 Graph Cut' (TL-L1GC). The underlying idea of this method is to generate the optimal projection matrix by considering both the sparsity as well as the corr…
Novel bounds for deep MDA algorithms improve performance and efficiency.