Random convex analysis tackles problems in random environments.
problem Dealing with problems in random environments like conditional convex risk measures.
method Developing random convex analysis over random locally convex modules, establishing inferior limit behavior, continuity, subdifferentiability, and approximating ε-subdifferentials.
result Established relationships among subdifferentiability, Gâteaux-differentiability, and Fréchet-differentiability for proper L 0 L^0 L 0 -convex functions. Generalizes smoothness conditions for optimization methods.
problem Optimization under non-uniform smoothness conditions.
method Develops a new analysis technique for bounding gradients.
result Obtains convergence rates for gradient descent and Nesterov's method.
First-order methods tackle g-convex optimization on Hadamard manifolds.
problem Geodesically convex optimization on nonlinear metric spaces.
method Iteration complexity analysis for first-order algorithms.
result Upper bounds for global complexity of g-convex optimization.
Prototypal analysis improves archetypal analysis by penalizing distant prototypes, making it more robust and interpretable.
problem Sensitivity to outliers and non-locality in archetypal analysis limit its applicability as a learning tool.
method Prototypal analysis finds prototypes through convex combination of data points, penalizing distant prototypes.
result Prototypal analysis is more robust and interpretable than archetypal analysis.
Unified analysis of stochastic gradient methods for convex and smooth optimization.
problem Minimizing composite convex and smooth functions.
method Unified convergence analysis of various stochastic gradient methods.
result Unified convergence rates for a variety of methods including proximal SGD, variance reduced methods, quantization, and coordinate descent.
Survey of convex optimization for community detection in networks.
problem Community detection in networks.
method Convex optimization techniques and their theoretical analysis.
result Advantages of convex community detection, including robustness, consistency, and adaptivity.
Near-convex archetypal analysis improves interpretability and fitting error in NMF.
problem High data fitting error in traditional archetypal analysis.
method Introduces near-convex archetypal analysis (NCAA) that combines AA and NMF.
result NCAA achieves lower data fitting error than state-of-the-art methods.
Paper improves stability analysis of SGD for various loss functions and data distributions.
problem Improving stability analysis of SGD for non-convex loss functions and data distributions.
method Analyzes stability of SGD for convex and non-convex loss functions, and improves data-dependent bounds.
result Improved stability bounds for non-convex loss functions and convex regularized loss functions.
A new non-convex method improves robust PCA with features.
problem Robust Principal Component Analysis with prior feature information.
method A novel non-convex optimization approach for decomposition.
result Exact recovery guarantees with low computational complexity.
Classifies geodetically convex sets and functions on Heisenberg group.
problem Characterizing geodetically convex sets and functions in the Heisenberg group.
method Classification through mathematical analysis.
result Geodetically convex sets and functions defined on Heisenberg group H n {\mathbb H}^n H n classified. New insights into using momentum for non-convex optimization.
problem Improving training of non-convex models like deep neural networks.
method Developed a Lyapunov analysis of SGD with momentum using stochastic primal averaging.
result Precise conditions under which SGD+M outperforms SGD and optimal hyper-parameter schedules.
We develop a convex relaxation method for analyzing neural network generalization.
problem Analyzing the generalization of parallel positively homogeneous networks.
method Linking non-convex ERM to a convex optimization problem over prediction functions.
result Achieved generalization bounds with almost linear sample complexity in network width.
Unified analysis improves SAM for non-convex optimization.
problem Improving generalization in machine learning models.
method Sharpness-aware minimization (SAM) and Unified SAM.
result Unified SAM provides convergence guarantees under relaxed assumptions.
Novel analysis of neural networks using geometric algebra and convex optimization.
problem Understanding the inner workings of deep neural networks.
method Geometric (Clifford) algebra and convex optimization.
result Optimal weights are given by the wedge product of training samples.
New techniques solve robust principal component analysis problems.
problem Robust Principal Component Analysis (RPCA) problems.
method Dual smoothing and level set techniques in convex optimization.
result Numerous theoretical and practical improvements for RPCA.
Develops a provable convex tensor clustering method.
problem Cluster analysis of tensors, especially in high dimensions.
method Provably convex formulation of tensor co-clustering.
result Non-asymptotic error bound revealing 'blessing of dimensionality'.
SGHMC uses noise to find global minima in non-convex learning.
problem Finding global minima in non-convex optimization problems.
method SGHMC with momentum and Gaussian noise for non-asymptotic convergence.
result Non-asymptotic convergence analysis for non-convex optimization.
We consider the closely related problems of bandit convex optimization with two-point feedback, and zero-order stochastic convex optimization with two function evaluations per round. We provide a simple algorithm and analysis which is optimal for convex Lipschitz functions. This improves on \cite{dujww13}, which only p…
Simple analysis for fast rates in empirical minimization with concave losses and convex regularization.
problem Fast rates in empirical minimization with concave losses and convex regularization.
method Simple analysis using covering number and concentration inequality.
result First result of fast rates with high probability for exponential concave empirical risk minimization.
Study shows how to control jump-diffusion processes with stable feedback controls in reinforcement learning.
problem Control jump-diffusion processes with unknown coefficients in reinforcement learning.
method Lipschitz continuous optimal feedback controls, stability analysis of forward-backward SDEs, least-squares algorithm.
result Achieves O ( N ln N ) O(\sqrt{N\ln N}) O ( N ln N ) regret for linear-convex learning problems with jumps. Bounds on chemical reaction network relaxation rates using convex analysis.
problem Understanding relaxation dynamics in chemical reaction networks.
method Convex analysis, generalized gradient flows, singular values of stoichiometric matrix.
result Bounds on Kullback-Leibler divergence to equilibrium for CRNs.
Set-functions appear in many areas of computer science and applied mathematics, such as machine learning, computer vision, operations research or electrical networks. Among these set-functions, submodular functions play an important role, similar to convex functions on vector spaces. In this tutorial, the theory of sub…
Paper analyzes convergence of proximal algorithm in metric spaces without geodesic convexity.
problem Analyzing convergence of proximal algorithm in general metric spaces.
method Analysis of the Wasserstein proximal algorithm without geodesic convexity assumption.
result Establishes unbiased and linear convergence rate for proximal algorithm under natural Wasserstein inequality.
Paper introduces a continuous convexity measure for compact sets.
problem Lack of continuity in existing convexity measures.
method Enriched axioms with continuity hypothesis in Hausdorff's sense.
result Theoretical grounding and continuous convexity measure construction.
Generalizes PCA to maximize any convex function of components.
problem Finding a principal vector that maximizes a convex function of components.
method Gradient ascent algorithm for solving the generalized PCA problem; fixed points of neural networks for kernel version.
result Solutions can be obtained as fixed points of simple neural networks.
Gradient method achieves linear convergence for saddle point problems without strong convexity.
problem Solving saddle point problems with non-strongly convex functions.
method Primal-dual gradient method with a novel analysis technique.
result Linear convergence achieved without strong convexity of f f f . The paper finds new inequalities for convex polygons.
problem Finding precise inequalities for convex polygons.
method Analytic isoperimetric inequalities based on Schur convex functions, followed by Bonnesen-style and inverse Bonnesen-style inequalities.
result Sharp discrete isoperimetric inequalities for planar convex polygons.
Paper improves robust PCA for noisy, outlier, and missing data.
problem Robust PCA with noise, outliers, and missing data.
method Bridging convex and nonconvex optimization.
result Near-optimal statistical accuracy for robust PCA.
Study iterative regularization for linear models with convex bias, improving robust sparse recovery.
problem Improving robust sparse recovery with iterative regularization for linear models.
method Primal-dual gradient approach, analyzing convergence in presence of noise, combining regularization and optimization.
result Theoretical results show state-of-the-art performances with computational speed-ups.
Unified analysis for shuffling-type gradient methods in optimization.
problem Optimization of finite-sum problems using shuffling strategies.
method Unified convergence analysis for various shuffling methods.
result Improved convergence rates for nonconvex problems and matching rates for convex problems.
Paper constructs L 2 L^2 L 2 estimates for flat vector bundles and generalizes Prékopa's theorem.
problem Constructing L 2 L^2 L 2 estimates for flat vector bundles. method Using Hörmander's L 2 L^2 L 2 -estimate for the operator d d d on a flat vector bundle over a p p p -convex Riemannian manifold. result Generalizes Prékopa's theorem in convex analysis.
New method tackles non-convex optimization problems using variance reduction.
problem Non-convex composite optimization problems.
method Variance-reduced proximal stochastic gradient descent (prox-SVRG and prox-SAGA).
result Converges to a stationary point within O(1/ε) iterations.
New analysis shows D-SGD can generalize well regardless of graph connectivity.
problem Improving generalization of D-SGD in decentralized settings.
method Algorithmic stability analysis and optimization-dependent generalization bounds.
result D-SGD can achieve generalization bounds similar to classical SGD, independent of graph connectivity.
Paper improves Heavy-ball method convergence in convex settings.
problem Convergence analysis of Heavy-ball method in convex optimization.
method Improved convergence complexity results for Heavy-ball method with constant step size.
result First non-ergodic O(1/k) rate result for coercive objective functions.
Develops a Riemannian archetypal analysis for interpretable non-linear data.
problem Limited performance of classical archetypal analysis on non-linear data.
method Riemannian geometry for data-driven pullback, geodesic convex combinations, convex relaxation followed by non-convex refinement.
result Combines interpretability of classical archetypal analysis with expressive power of modern non-linear models.
The subdifferential of convex functions of the singular spectrum of real matrices has been widely studied in matrix analysis, optimization and automatic control theory. Convex analysis and optimization over spaces of tensors is now gaining much interest due to its potential applications to signal processing, statistics…
This paper tackles multilayer graph clustering via convex layer aggregation.
problem Challenges in clustering multilayer graphs and combining information from each layer.
method Theoretical framework for multilayer spectral graph clustering via convex layer aggregation.
result Establishes a critical value on the noise level for reliable cluster separation.
Unified analysis of multi-attribute graph learning with non-convex penalties.
problem Graph inference from multi-attribute data.
method Penalized log-likelihood objective function with ADMM and local linear approximation.
result Local consistency in support recovery and precision matrix estimation for non-convex penalties.
New binary AA methods improve on existing techniques.
problem Binary data limitations in AA methods.
method Proposed two optimization frameworks for binary AA.
result Superior performance on synthetic and real binary data.
New method improves tensor completion and robust PCA using non-convex tensor rank and sparsity measures.
problem Challenging tensor rank minimization in machine learning.
method Proposes a non-convex tensor rank surrogate function and sparsity measure, using concavity for optimization.
result Demonstrates improved accuracy and efficiency in tensor completion and robust PCA.
The study proves curvature rigidity for convex polytopes.
problem Proving curvature rigidity for convex polytopes.
method Using Fredholm theory for Dirac operators and a theorem of Fefferman and Phong.
result Scalar curvature rigidity theorem for convex polytopes proved.
SGLD helps escape local minima in non-convex learning problems.
problem Non-convex optimization in machine learning.
method Stochastic Gradient Langevin Dynamics with Gaussian noise.
result Finite-time guarantees for SGLD to find approximate minimizers.
ProxSkip achieves linear speedup in distributed non-convex optimization.
problem Achieving linear speedup in distributed non-convex optimization.
method Unified convergence analysis for stochastic non-convex, convex, and strongly convex problems.
result ProxSkip achieves linear speedup in the number of nodes under stochastic gradients.
Study AFPP of unions of convex digital disks in 2D.
problem Conditions for AFPP of union of convex disks in digital plane.
method Use results from [6] to analyze AFPP.
result Conditions for AFPP of union of convex disks.
Optimal algorithms for online convex optimization with random order.
problem Online convex optimization with random order and non-convex loss functions.
method Stochastic gradient descent and algorithmic stability analysis.
result Achieves optimal bounds and significantly outperforms previous methods.
A new algorithm speeds up convex clustering.
problem Optimizing clustering with convex optimization and avoiding local minima.
method Smoothing proximal gradient algorithm (Sproga) for convex clustering.
result Sproga is faster and uses less memory than existing methods.
The paper proves consistency of archetypal analysis for multivariate data.
problem Finding optimal archetype points for multivariate data.
method Uses convex polytope to summarize data, proving consistency under specific distribution assumptions.
result Archetype points converge to optimal solution under certain conditions.
This paper addresses challenges in distance metric learning by promoting orthogonality and providing theoretical guarantees.
problem Challenges in distance metric learning, including non-convex optimization, lack of theoretical understanding, and generalization issues.
method Develops convex relaxations of non-convex problems, provides theoretical analysis on orthogonality, and offers a direct link to generalization performance.
result Convex methods promote balancedness, compactness, and generalization more effectively and efficiently.