arXiv research

Global convergence for robust regression problems via IRLS with enhancements.

Proposes a non-convex optimization method for a parsimonious weighted naive Bayes classifier.

NAPP-ERM improves ERM with differential privacy guarantees by iteratively achieving target regularization and delivering strong convexity.

A new algorithm solves signed Fréchet regression on manifolds with bounded curvature.

Study sharp inequalities for perimeter functionals in capillarity and convex cones.

Strong geodesic convex function and strong monotone vector field of order $m$ on Riemannian manifolds have been established. A characterization of strong geodesic convex function of order $m$ for the continuously differentiable functions has been discussed. The relation between the solution of a new variational inequal…

We introduce a flexible family of fairness regularizers for (linear and logistic) regression problems. These regularizers all enjoy convexity, permitting fast optimization, and they span the rang from notions of group fairness to strong individual fairness. By varying the weight on the fairness regularizer, we can comp…

FastAdaBelief improves convergence rate of AdaBelief by exploiting strong convexity.

This paper shows how to learn variational inequalities fast with strong monotonicity.

Estimate collapsibility of causal effects in CPDAGs via strong d-convex hulls.

SAGA is a fast incremental gradient method on the finite sum problem and its effectiveness has been tested on a vast of applications. In this paper, we analyze SAGA on a class of non-strongly convex and non-convex statistical problem such as Lasso, group Lasso, Logistic regression with $\ell_1$ regularization, linear r…

Harmonic functions on compact symmetric spaces exhibit strong convexity properties.

We propose a projected semi-stochastic gradient descent method with mini-batch for improving both the theoretical complexity and practical performance of the general stochastic gradient descent method (SGD). We are able to prove linear convergence under weak strong convexity assumption. This requires no strong convexit…

Improved kernel quadrature with convex weights using subsampling.

Optimal weight windows are found by projecting the origin onto a convex polytope.

New weight initialisation for ICNNs accelerates learning and improves generalization.

Optimal control in changing systems without strong convexity assumptions.

New guarantees for Group LASSO in sparse convex optimization.

Epoch-GDA achieves optimal convergence rate for SCSC min-max problems.

New inequalities for convex hypersurfaces in various spaces.

New algorithm solves saddle point problems in Banach spaces.

Positive weights improve kernel quadrature's accuracy.

Range penalization enhances statistical accuracy and resource efficiency in federated learning.

The paper proves ML estimators are strongly consistent for identifying edge weights in BAR models.

The paper explores inequalities for strongly-convex sets in weighted Riemannian manifolds.

We show that an infinite dimensional Lie group in Milnor's sense has the strong Trotter property if it is locally $μ$-convex. This is a continuity condition imposed on the Lie group multiplication that generalizes the triangle inequality for locally convex vector spaces, and is equivalent to $C^0$-continuity of the evo…

The versatility of exponential families, along with their attendant convexity properties, make them a popular and effective statistical model. A central issue is learning these models in high-dimensions, such as when there is some sparsity pattern of the optimal parameter. This work characterizes a certain strong conve…

This paper analyzes SGD with increasingly weighted averaging for optimization and generalization.

The paper proves new inequalities for convex hypersurfaces in hyperbolic and spherical spaces.

We propose a DC proximal Newton algorithm for solving nonconvex regularized sparse learning problems in high dimensions. Our proposed algorithm integrates the proximal Newton algorithm with multi-stage convex relaxation based on the difference of convex (DC) programming, and enjoys both strong computational and statist…

This work studies the strong duality of non-convex matrix factorization problems: we show that under certain dual conditions, these problems and its dual have the same optimum. This has been well understood for convex optimization, but little was known for non-convex problems. We propose a novel analytical framework an…

Sparsity inducing regularization is an important part for learning over-complete visual representations. Despite the popularity of $\ell_1$ regularization, in this paper, we investigate the usage of non-convex regularizations in this problem. Our contribution consists of three parts. First, we propose the leaky capped …

New findings on strong convexity in triangulations of convex polygons.

We obtain an improved finite-sample guarantee on the linear convergence of stochastic gradient descent for smooth and strongly convex objectives, improving from a quadratic dependence on the conditioning $(L/μ)^2$ (where $L$ is a bound on the smoothness and $μ$ on the strong convexity) to a linear dependence on $L/μ$. …

We consider empirical risk minimization of linear predictors with convex loss functions. Such problems can be reformulated as convex-concave saddle point problems, and thus are well suitable for primal-dual first-order algorithms. However, primal-dual algorithms often require explicit strongly convex regularization in …

New weighted surface area measures for convex bodies with applications.

The Adam algorithm has become extremely popular for large-scale machine learning. Under convexity condition, it has been proved to enjoy a data-dependant $O(\sqrt{T})$ regret bound where $T$ is the time horizon. However, whether strong convexity can be utilized to further improve the performance remains an open problem…

The paper extends geometric inequalities for nearly spherical sets in various space forms.

The paper proves new inequalities in hyperbolic space using Euclidean methods.

Convex optimization refines neural network training, improving model performance and reducing hyperparameter sensitivity.

We present a novel approach for constrained Bayesian inference. Unlike current methods, our approach does not require convexity of the constraint set. We reduce the constrained variational inference to a parametric optimization over the feasible set of densities and propose a general recipe for such problems. We apply …

Stochastic approximation (SA) is a classical approach for stochastic convex optimization. Previous studies have demonstrated that the convergence rate of SA can be improved by introducing either smoothness or strong convexity condition. In this paper, we make use of smoothness and strong convexity simultaneously to boo…

SVRG and its variants are among the state of art optimization algorithms for large scale machine learning problems. It is well known that SVRG converges linearly when the objective function is strongly convex. However this setup can be restrictive, and does not include several important formulations such as Lasso, grou…

The paper proves inequalities for star-shaped and $F$-mean convex hypersurfaces in $\mathbb{R}^{n+1}$.

Stochastic gradient algorithms estimate the gradient based on only one or a few samples and enjoy low computational cost per iteration. They have been widely used in large-scale optimization problems. However, stochastic gradient algorithms are usually slow to converge and achieve sub-linear convergence rates, due to t…

Study shows robust method for estimating density ratios even with heavy contamination.

Optimizes sample weights for representative data averages.

RR algorithm improves convergence rate without strong convexity assumptions.