arXiv research

Study on consistency of ML methods for moving objects in non-stationary environments.

Stabilized SVRG finds second-order stationary points efficiently in nonconvex optimization.

This paper tackles the computational complexity of finding approximate stationary points in non-convex optimization.

New method finds stationary points in bilevel optimization problems.

Improved DP optimization for nonconvex, nonsmooth objectives with reduced sample complexity.

Novel RL-based NPG improves multi-objective NAS efficiency and performance.

This work ensures policy gradient methods converge to global optima for certain control problems.

New algorithm optimizes multi-objective outcomes in uncertain environments.

Building models, or maps, of robot environments is a highly active research area; however, most existing techniques construct unstructured maps and assume static environments. In this paper, we present an algorithm for learning object models of non-stationary objects found in office-type environments. Our algorithm exp…

Study proves convergence of subgradients for optimal transport-based objectives.

We study parameter estimation and asymptotic inference for sparse nonlinear regression. More specifically, we assume the data are given by $y = f( x^\top β^* ) + ε$, where $f$ is nonlinear. To recover $β^*$, we propose an $\ell_1$-regularized least-squares estimator. Unlike classical linear regression, the correspondin…

New method solves stochastic optimization problems with random models.

Proposes new stochastic algorithms for multi-objective optimization.

New algorithm improves fairness and robustness in federated learning.

New algorithms optimize non-smooth, non-convex objectives with improved complexity.

The paper tackles finding stationary points in stochastic convex optimization problems.

Given a heterogeneous time-series sample, the objective is to find points in time (called change points) where the probability distribution generating the data has changed. The data are assumed to have been generated by arbitrary unknown stationary ergodic distributions. No modelling, independence or mixing assumptions…

New method finds points for approximating distributions faster.

MAML optimizes shared priors for subtasks in a nonconvex meta-objective.

New methods solve statistical problems for stationary time series.

The thesis presents a new perspective on high-dimensional optimization.

Novel methods for accelerating optimization in complex bilevel and minimax problems.

Cautious Weight Decay modifies weight decay for better optimization.

Improved algorithm finds second-order stationary points in non-convex optimization.

ConvNP improves SP prediction with translation equivariance and coherent samples.

We give a simple proof that the Frank-Wolfe algorithm obtains a stationary point at a rate of $O(1/\sqrt{t})$ on non-convex objectives with a Lipschitz continuous gradient. Our analysis is affine invariant and is the first, to the best of our knowledge, giving a similar rate to what was already proven for projected gra…

New infinite family of geodesic nets on Euclidean plane discovered.

Novel Bayesian approach for non-stationary linear contextual bandits.

Stochastic optimization naturally arises in machine learning. Efficient algorithms with provable guarantees, however, are still largely missing, when the objective function is nonconvex and the data points are dependent. This paper studies this fundamental challenge through a streaming PCA problem for stationary time s…

Paper solves non-convex min-max games using gradient descent-ascent.

New algorithm finds approximate stationary points in non-convex optimization.

Paper develops sparse learning for heavy-tailed time series with locally stationary dynamics.

New theory for area of Legendrian surfaces, proving smoothness and variational results.

This paper tackles objective inconsistency in federated optimization with heterogeneous clients.

Bayesian optimization improves with nonstationary covariance functions.

New algorithm solves structured nonconvex-nonconcave min-max problems.

HA-SME models SGD dynamics with Hessian info for better escaping behaviors.

SUSTAIN algorithm tackles stochastic bilevel optimization with near-optimal complexity.

Learning new representations of input observations in machine learning is often tackled using a factorization of the data. For many such problems, including sparse coding and matrix completion, learning these factorizations can be difficult, in terms of efficiency and to guarantee that the solution is a global minimum.…

Hamiltonian stationary Lagrangian submanifolds (HSLAG) are a natural generalization of special Lagrangian manifolds (SLAG). The latter only make sense on Calabi-Yau manifolds whereas the former are defined for any almost Kähler manifold. Special Lagrangians, and, more specificaly, fibrations by special Lagrangians play…

We propose a general theory for studying the \xl{landscape} of nonconvex \xl{optimization} with underlying symmetric structures \tz{for a class of machine learning problems (e.g., low-rank matrix factorization, phase retrieval, and deep linear neural networks)}. In specific, we characterize the locations of stationary …

The alternating gradient descent (AGD) is a simple but popular algorithm which has been applied to problems in optimization, machine learning, data ming, and signal processing, etc. The algorithm updates two blocks of variables in an alternating manner, in which a gradient step is taken on one block, while keeping the …

Improved complexity for smooth nonconvex optimization using quasi-Newton methods.

In this paper, we consider efficient differentially private empirical risk minimization from the viewpoint of optimization algorithms. For strongly convex and smooth objectives, we prove that gradient descent with output perturbation not only achieves nearly optimal utility, but also significantly improves the running …

Study non-stationary bandits with resource constraints.

We provide novel theoretical results regarding local optima of regularized $M$-estimators, allowing for nonconvexity in both loss and penalty functions. Under restricted strong convexity on the loss and suitable regularity conditions on the penalty, we prove that \emph{any stationary point} of the composite objective f…

A new method solves optimization problems over Stiefel manifold.

DILATE improves deep time series forecasting for non-stationary signals.