arXiv research

Loose bounds found for least-norm interpolant in over-parameterized settings.

Proposes a new regression method using $L_p$-norms for non-Gaussian noise.

A new screening rule 'dynamic Sasvi' improves sparse optimization speed.

The paper analyzes the risk of a least squares estimator under a spike covariance model.

Derives integral formula for ReLU networks with limited weights.

We characterize the three-dimensional spaces admitting at least six or at least seven equidistant points. In particular, we show the existence of $C^\infty$ norms on $\R^3$ admitting six equidistant points, which refutes a conjecture of Lawlor and Morgan (1994, Pacific J. Math \textbf{166}, 55--83), and gives the exist…

Learning rates for least-squares regression are typically expressed in terms of $L_2$-norms. In this paper we extend these rates to norms stronger than the $L_2$-norm without requiring the regression function to be contained in the hypothesis space. In the special case of Sobolev reproducing kernel Hilbert spaces used …

Principal component analysis (PCA) is often used to reduce the dimension of data by selecting a few orthonormal vectors that explain most of the variance structure of the data. L1 PCA uses the L1 norm to measure error, whereas the conventional PCA uses the L2 norm. For the L1 PCA problem minimizing the fitting error of…

We prove statistical rates of convergence for kernel-based least squares regression from i.i.d. data using a conjugate gradient algorithm, where regularization against overfitting is obtained by early stopping. This method is related to Kernel Partial Least Squares, a regression method that combines supervised dimensio…

The $\ell$-1 norm based optimization is widely used in signal processing, especially in recent compressed sensing theory. This paper studies the solution path of the $\ell$-1 norm penalized least-square problem, whose constrained form is known as Least Absolute Shrinkage and Selection Operator (LASSO). A solution path …

ESNs trained with Tikhonov least squares approximate ergodic dynamical systems in L2(μ) norm.

A Bayesian approach termed BAyesian Least Squares Optimization with Nonnegative L1-norm constraint (BALSON) is proposed. The error distribution of data fitting is described by Gaussian likelihood. The parameter distribution is assumed to be a Dirichlet distribution. With the Bayes rule, searching for the optimal parame…

Support vector machines (SVMs) are an important tool in modern data analysis. Traditionally, support vector machines have been fitted via quadratic programming, either using purpose-built or off-the-shelf algorithms. We present an alternative approach to SVM fitting via the majorization--minimization (MM) paradigm. Alg…

In this paper we investigate panel regression models with interactive fixed effects. We propose two new estimation methods that are based on minimizing convex objective functions. The first method minimizes the sum of squared residuals with a nuclear (trace) norm regularization. The second method minimizes the nuclear …

Using the $\ell_1$-norm to regularize the estimation of the parameter vector of a linear model leads to an unstable estimator when covariates are highly correlated. In this paper, we introduce a new penalty function which takes into account the correlation of the design matrix to stabilize the estimation. This norm, ca…

We analyze low rank tensor completion (TC) using noisy measurements of a subset of the tensor. Assuming a rank-$r$, order-$d$, $N \times N \times \cdots \times N$ tensor where $r=O(1)$, the best sampling complexity that was achieved is $O(N^{\frac{d}{2}})$, which is obtained by solving a tensor nuclear-norm minimizatio…

Uniform convergence of interpolators proven for Gaussian data.

We study the total least squares (TLS) problem that generalizes least squares regression by allowing measurement errors in both dependent and independent variables. TLS is widely used in applied fields including computer vision, system identification and econometrics. The special case when all dependent and independent…

We consider the empirical risk minimization problem for linear supervised learning, with regularization by structured sparsity-inducing norms. These are defined as sums of Euclidean norms on certain subsets of variables, extending the usual $\ell_1$-norm and the group $\ell_1$-norm by allowing the subsets to overlap. T…

We obtain a Chern-Osserman type equality of a complete properly immersed surface in Euclidean space, provided the L^2-norm of the second fundamental form is finite. Also, by using a monotonicity formula, we prove that if the L^2-norm of mean curvature of a noncompact surface is finite, then it has at least quadratic ar…

This work presents a general framework for solving the low rank and/or sparse matrix minimization problems, which may involve multiple non-smooth terms. The Iteratively Reweighted Least Squares (IRLS) method is a fast solver, which smooths the objective function and minimizes it by alternately updating the variables an…

The paper explores why a specific type of predictor works well in noisy data.

The paper proves inequalities linking geometric norms and Thurston norms in hyperbolic 3-manifolds.

We provide a pointwise confidence bound for non-linear least-squares with fixed design.

Weight normalization and reparametrized gradient descent adaptively regularize weights and converge to minimum l2 norm solutions.

The real homology of a compact Riemannian manifold $M$ is naturally endowed with the stable norm. The stable norm on $H_1(M,\mathbb{R})$ arises from the Riemannian length functional by homogenization. It is difficult and interesting to decide which norms on the finite-dimensional vector space $H_1(M,\mathbb{R})$ are st…

The study analyzes robustness of estimators in linear models with adversarial errors.

Dual-sPLS improves feature selection and prediction in high-dimensional data.

Deep neural networks with adversarial training achieve sup-norm convergence for nonparametric regression.

A data filtering method for cluster analysis is proposed, based on minimizing a least squares function with a weighted $\ell_0$-norm penalty. To overcome the discontinuity of the objective function, smooth non-convex functions are employed to approximate the $\ell_0$-norm. The convergence of the global minimum points o…

The paper tackles estimation of hidden state LTI systems of unknown order.

Study pinches curvature under Laplacian G_2 flow, proving Weyl tensor norm blows up.

This paper explores using SSIM for better image generation in generative models.

Optimal multiscale learning of linear operators

Proposes a new method for joint sample and feature selection in multi-view data.

The paper analyzes error bounds and KL properties for noisy matrix recovery problems.

Self-focal points on ellipsoids of dimension 3 or higher are rare.

A typical approach in estimating the learning rate of a regularized learning scheme is to bound the approximation error by the sum of the sampling error, the hypothesis error and the regularization error. Using a reproducing kernel space that satisfies the linear representer theorem brings the advantage of discarding t…

We study randomized sketching methods for approximately solving least-squares problem with a general convex constraint. The quality of a least-squares approximation can be assessed in different ways: either in terms of the value of the quadratic objective function (cost approximation), or in terms of some distance meas…

New framework for private convex optimization in arbitrary norms.

Method improves SINDy for noisy nonlinear systems.

The paper improves bounds on topological complexity for certain manifolds.

We show that a regular isomorphism of profinite completion of the fundamental groups of two 3-manifolds $N_1$ and $N_2$ induces an isometry of the Thurston norms and a bijection between the fibered classes. We study to what extent does the profinite completion of knot groups distinguish knots and show that it distingui…

Analysis of non-asymptotic estimation error and structured statistical recovery based on norm regularized regression, such as Lasso, needs to consider four aspects: the norm, the loss function, the design matrix, and the noise model. This paper presents generalizations of such estimation error analysis on all four aspe…

The "double descent" risk curve was proposed to qualitatively describe the out-of-sample prediction accuracy of variably-parameterized machine learning models. This article provides a precise mathematical analysis for the shape of this curve in two simple data models with the least squares/least norm predictor. Specifi…

Stochastic optimization algorithms update models with cheap per-iteration costs sequentially, which makes them amenable for large-scale data analysis. Such algorithms have been widely studied for structured sparse models where the sparsity information is very specific, e.g., convex sparsity-inducing norms or $\ell^0$-n…

The paper tackles robust reinforcement learning with performance guarantees.

Bagging stabilizes linear interpolators, improving their generalization performance.