arXiv research

This paper is concerned with the squared F(robenius)-norm regularized factorization form for noisy low-rank matrix recovery problems. Under a suitable assumption on the restricted condition number of the Hessian for the loss function, we derive an error bound to the true matrix for the non-strict critical points with r…

Proposes a new regression method using $L_p$-norms for non-Gaussian noise.

A new screening rule 'dynamic Sasvi' improves sparse optimization speed.

Non-degeneracy of critical points proven for manifold's squared norm of second fundamental form.

Improved LDA with capped l_{2,1}-norm reduces outlier sensitivity.

We present a proof due to Duistermaat that the gradient flow of the norm squared of the moment map defines a deformation retract of the appropriate piece of the manifold onto the zero level set of the moment map. Duistermaat's proof is an adaptation of Lojasiewicz's argument for analytic functions to functions which ar…

Study classifies 3D self-shrinkers with constant second form norm.

Learning rates for least-squares regression are typically expressed in terms of $L_2$-norms. In this paper we extend these rates to norms stronger than the $L_2$-norm without requiring the regression function to be contained in the hypothesis space. In the special case of Sobolev reproducing kernel Hilbert spaces used …

Researchers classify special curved spheres in a complex space.

We study risk of the minimum norm linear least squares estimator in when the number of parameters $d$ depends on $n$, and $\frac{d}{n} \rightarrow \infty$. We assume that data has an underlying low rank structure by restricting ourselves to spike covariance matrices, where a fixed finite number of eigenvalues grow with…

In this paper we investigate panel regression models with interactive fixed effects. We propose two new estimation methods that are based on minimizing convex objective functions. The first method minimizes the sum of squared residuals with a nuclear (trace) norm regularization. The second method minimizes the nuclear …

The paper accelerates ISTA and FISTA algorithms for composite optimization problems.

Principal component analysis (PCA) is often used to reduce the dimension of data by selecting a few orthonormal vectors that explain most of the variance structure of the data. L1 PCA uses the L1 norm to measure error, whereas the conventional PCA uses the L2 norm. For the L1 PCA problem minimizing the fitting error of…

There is considered the problem of describing up to linear conformal equivalence those harmonic cubic homogeneous polynomials for which the squared-norm of the Hessian is a nonzero multiple of the quadratic form defining the Euclidean metric. Solutions are constructed in all dimensions and solutions are classified in d…

Support vector machines (SVMs) are an important tool in modern data analysis. Traditionally, support vector machines have been fitted via quadratic programming, either using purpose-built or off-the-shelf algorithms. We present an alternative approach to SVM fitting via the majorization--minimization (MM) paradigm. Alg…

This work presents a general framework for solving the low rank and/or sparse matrix minimization problems, which may involve multiple non-smooth terms. The Iteratively Reweighted Least Squares (IRLS) method is a fast solver, which smooths the objective function and minimizes it by alternately updating the variables an…

New algorithms adapt to both gradient norms and comparator norms in online learning.

We give a formal and complete characterization of the explicit regularizer induced by dropout in deep linear networks with squared loss. We show that (a) the explicit regularizer is composed of an $\ell_2$-path regularizer and other terms that are also re-scaling invariant, (b) the convex envelope of the induced regula…

We prove statistical rates of convergence for kernel-based least squares regression from i.i.d. data using a conjugate gradient algorithm, where regularization against overfitting is obtained by early stopping. This method is related to Kernel Partial Least Squares, a regression method that combines supervised dimensio…

Study properties of bi-warped product submanifolds in specific geometric spaces.

ESNs trained with Tikhonov least squares approximate ergodic dynamical systems in L2(μ) norm.

New findings on depth vs. width in neural networks, showing depth can improve learnability.

SNEPPPs use squared neural networks to efficiently model Poisson point processes.

A Bayesian approach termed BAyesian Least Squares Optimization with Nonnegative L1-norm constraint (BALSON) is proposed. The error distribution of data fitting is described by Gaussian likelihood. The parameter distribution is assumed to be a Dirichlet distribution. With the Bayes rule, searching for the optimal parame…

B. Y. Chen establish the relationship between the Ricci curvature and the squared mean curvature for submanifolds of Riemannian space form with arbitrary codimension. In this paper, we generalize the relationship between the Ricci curvature and the squared norm of mean curvature vector for submanifolds of Bochner Kahle…

It is our purpose to study complete self-shrinkers in Euclidean space. By introducing a generalized maximum principle for $\mathcal{L}$-operator, we give estimates on supremum and infimum of the squared norm of the second fundamental form of self-shrinkers without assumption on \emph{polynomial volume growth}, which is…

Study norm-squared of momentum map in infinite dimensions with applications to symplectic geometry.

This is the fourth article of our series. Here, we study weighted norm inequalities for the Riesz transform of the Laplace-Beltrami operator on Riemannian manifolds and of subelliptic sum of squares on Lie groups, under the doubling volume property and Gaussian upper bounds.

The study analyzes convergence rates for sparse pivotal estimators in high-dimensional regression.

Proves a pinching theorem for self-shrinkers of mean curvature flow.

Using the $\ell_1$-norm to regularize the estimation of the parameter vector of a linear model leads to an unstable estimator when covariates are highly correlated. In this paper, we introduce a new penalty function which takes into account the correlation of the design matrix to stabilize the estimation. This norm, ca…

We analyze low rank tensor completion (TC) using noisy measurements of a subset of the tensor. Assuming a rank-$r$, order-$d$, $N \times N \times \cdots \times N$ tensor where $r=O(1)$, the best sampling complexity that was achieved is $O(N^{\frac{d}{2}})$, which is obtained by solving a tensor nuclear-norm minimizatio…

The study classifies complete self-shrinkers in Euclidean space.

Improved sample complexity for ReLU networks with norm constraints.

We study the total least squares (TLS) problem that generalizes least squares regression by allowing measurement errors in both dependent and independent variables. TLS is widely used in applied fields including computer vision, system identification and econometrics. The special case when all dependent and independent…

We consider the question of what functions can be captured by ReLU networks with an unbounded number of units (infinite width), but where the overall network Euclidean norm (sum of squares of all weights in the system, except for an unregularized bias term for each unit) is bounded; or equivalently what is the minimal …

Paper develops efficient AltMin algorithm for SRPCP robust matrix recovery.

This study explains gradient flow dynamics in neural networks for small initialisation.

The study proves properties of self-shrinkers with bounded curvature.

The problem of low-rank approximation with convex constraints, which appears in data analysis, system identification, model order reduction, low-order controller design and low-complexity modelling is considered. Given a matrix, the objective is to find a low-rank approximation that meets rank and convex constraints, w…

We consider the empirical risk minimization problem for linear supervised learning, with regularization by structured sparsity-inducing norms. These are defined as sums of Euclidean norms on certain subsets of variables, extending the usual $\ell_1$-norm and the group $\ell_1$-norm by allowing the subsets to overlap. T…

The $\ell$-1 norm based optimization is widely used in signal processing, especially in recent compressed sensing theory. This paper studies the solution path of the $\ell$-1 norm penalized least-square problem, whose constrained form is known as Least Absolute Shrinkage and Selection Operator (LASSO). A solution path …

We study a regularizer which is defined as a parameterized infimum of quadratics, and which we call the box-norm. We show that the k-support norm, a regularizer proposed by [Argyriou et al, 2012] for sparse vector prediction problems, belongs to this family, and the box-norm can be generated as a perturbation of the fo…

Study spectral distribution of twisted Laplacian on high genus hyperbolic surfaces.

New bounds for SA with arbitrary norm contractions and Markovian noise.

We show that the scalar curvature of a steady gradient Ricci soliton satisfying that the ratio between the square norm of the Ricci tensor and the square of the scalar curvature is bounded by one half, is boundend from below by the hyperbolic secant of one half the distance function from a fixed point.

The study computes Bergman kernels and point process asymptotics on Kähler manifolds.

We show that if a compact Kaehler manifold $M$ of non-negative Ricci curvature admits closed Fedosov star product then the reduced Lie algebra of holomorphic vector fields on $M$ is reductive. This comes in pair with the obstruction previously found by La Fuente-Gravy. More generally we consider the squared norm of Cah…