Develops optimal low-dimensional approximations to high-dimensional SDEs.
problem Approximating solutions to high-dimensional SDEs in a low-dimensional space.
method Introduces Ito-vector and Ito-jet projections for optimal approximation.
result Optimal projection filters yield better approximations than Stratonovich projection.
New projection techniques reduce the frequency of projections in solving LCPs.
problem Solving linearly constrained problems efficiently with reduced projection frequency.
method Delayed projection technique to call a projection less frequently.
result Theoretical and practical improvements in convergence rates and efficiency.
We define two new notions of projection of a stochastic differential equation (SDE) onto a submanifold: the Ito-vector and Ito-jet projections. This allows one to systematically develop low dimensional approximations to high dimensional SDEs using differential geometric techniques. The approach generalizes the notion o…
A dissertation on scalable projection-free optimization methods.
problem Efficient optimization algorithms for large-scale machine learning problems.
method Study of Frank-Wolfe variants and their extensions to distributed and derivative-free settings.
result Development of 1-SFW and QFW, achieving state-of-the-art complexity and efficiency.
Python package for projecting onto quadratic hypersurfaces.
problem Projections onto non-cylindrical central quadratic hypersurfaces.
method User-friendly Python package with documentation.
result Efficiently projects points onto quadratic hypersurfaces.
New framework solves low-rank optimization problems to certifiable optimality.
problem Low-rank optimization problems with certifiable solutions.
method Mixed-Projection Conic Optimization framework using symmetric projection matrices and outer-approximation algorithms.
result Solves low-rank problems to certifiable optimality, outperforming existing methods.
Optimizes reinsurance and investment strategies to minimize ruin probability.
problem Optimizing reinsurance and investment strategies to minimize ruin probability.
method Stochastic projected gradient method based on Malliavin calculus.
result Effectiveness of the proposed method demonstrated through numerical experiments.
New algorithm reduces adaptive regret without projections.
problem Computational expense of projections in online convex optimization.
method Lazy gradient-based algorithm with set-membership computations.
result Near-optimal adaptive regret bounds for general convex functions.
A new projection method for convex optimization reduces computation costs.
problem Efficiently projecting points into convex sets for deep learning.
method Interpolation-based projection for cheaper computation.
result The proposed method converges for linear and convex constraints.
Efficiently solves heterogeneous QPs by reducing variables using instance-specific projections.
problem Solving high-dimensional quadratic programming problems efficiently.
method Data-driven framework with a graph neural network generating projections tailored to each QP instance.
result Produces high-quality solutions with reduced computation time, outperforming existing methods.
We consider stochastic strongly convex optimization with a complex inequality constraint. This complex inequality constraint may lead to computationally expensive projections in algorithmic iterations of the stochastic gradient descent~(SGD) methods. To reduce the computation costs pertaining to the projections, we pro…
Paper tackles efficient SGD methods for constrained bilevel optimization.
problem Stochastic bilevel optimization with equality constraints.
method Alternating implicit projected SGD and its variants.
result Achieves sample complexity matching state-of-the-art for unconstrained problems.
Improves point-cloud reconstruction by optimizing projections with self-attention.
problem Inefficient and non-metric projection methods for sliced Wasserstein distances.
method Proposes distributional sliced Wasserstein distance with self-attention for permutation-invariant and metric optimization.
result Self-attention amortized distributional projection optimization achieves better performance in point-cloud reconstruction.
This paper focuses on convex constrained optimization problems, where the solution is subject to a convex inequality constraint. In particular, we aim at challenging problems for which both projection into the constrained domain and a linear optimization under the inequality constraint are time-consuming, which render …
Efficiently projects points onto polytopes, especially useful in web-scale applications.
problem Efficiently projecting points onto polytopes in large-scale applications.
method Developed a vertex-oriented incremental algorithm for polytope projection, tailored for simplex and unit-box cut polytopes.
result Majority of projections lie on vertices of polytopes, leading to significant performance improvements.
A new method optimizes projection directions for sliced Wasserstein distances.
problem Finding informative projecting directions for sliced Wasserstein distances is computationally expensive.
method Amortized projection optimization to predict directions efficiently.
result Proposed amortized models improve generative modeling performance.
Proof of convergence for multi-objective optimization using inverse reinforcement learning.
problem Proving convergence in multi-objective optimization problems.
method Wasserstein inverse reinforcement learning with projective subgradient method and gradient descent.
result Convergence of inverse reinforcement learning for multi-objective optimization.
Soft-Radial Projection solves gradient saturation in constrained deep learning.
problem Gradient saturation in deep learning models when integrating hard constraints.
method Introduces Soft-Radial Projection, a differentiable layer that maps predictions onto constraint boundaries without rank-deficient Jacobians.
result Improves convergence and solution quality over state-of-the-art methods.
Paper tackles online learning on curved spaces without projections.
problem Online learning on Riemannian manifolds with computational constraints.
method Develops projection-free algorithms for geodesically convex optimization.
result Achieves sub-linear regret guarantees in online geodesically convex optimization.
Optimal projections enhance Naive Bayes classification.
problem Improving Naive Bayes classification accuracy.
method Projection pursuit to find optimal linear projections.
result The approach substantially outperforms other models.
In many online learning problems the computational bottleneck for gradient-based methods is the projection operation. For this reason, in many problems the most efficient algorithms are based on the Frank-Wolfe method, which replaces projections by linear optimization. In the general case, however, online projection-fr…
Optimal inequalities found between Riemannian and Hilbert metrics in convex projective domains.
problem Finding optimal bounds between Riemannian and Hilbert metrics in convex projective domains.
method Optimal control techniques applied to Riemannian metrics induced by centro-affine hypersurface immersions.
result Optimal inequalities between Riemannian and Hilbert metrics for a class of convex projective domains.
Consider convex optimization problems subject to a large number of constraints. We focus on stochastic problems in which the objective takes the form of expected values and the feasible set is the intersection of a large number of convex sets. We propose a class of algorithms that perform both stochastic gradient desce…
Introduces PIT-plot for prioritizing projects based on their impact.
problem Optimizing R&D investments in project portfolios.
method Develops a new tool (PIT-plot) focusing on project impact rather than project properties.
result Identifies projects with the largest impact for risk mitigation or value-adding.
Paper proposes PPMM for fast estimation of large-scale OTM.
problem Estimation of large-scale optimal transport maps (OTM) is challenging due to the curse of dimensionality.
method Combines projection pursuit regression and sufficient dimension reduction to adaptively select projection directions.
result PPMM consistently estimates the most informative projection direction and weakly converges to the target OTM.
Optimizes energy of mappings from complex projective spaces.
problem Finding energy-minimizing mappings between complex projective spaces and Riemannian manifolds.
method Establishes optimal lower bounds for energy functionals and characterizes optimal mappings.
result Optimal lower bounds for energy functionals are characterized for mappings from real and complex projective spaces.
Develop a framework for barycentric projections of optimal transport plans on Riemannian manifolds.
problem Optimal transport couplings are probabilistic objects, while many learning pipelines require deterministic maps.
method Develop a framework for barycentric projections of transport couplings on Riemannian manifolds.
result The intrinsic projection maps each source point to the conditional Fréchet mean of its destination law and is shown to be the best deterministic representative under squared geodesic loss.
New framework reduces private mean estimation error with optimal efficiency.
problem Locally private mean estimation of high-dimensional vectors.
method ProjUnit framework: random projections, normalization, and optimal algorithm execution in lower dimensions.
result Optimal error up to a 1+o(1)-factor with computational efficiency and low communication complexity.
Paper presents efficient computation of robust Wasserstein distance using Riemannian optimization.
problem Intractability of optimizing Projection Robust Wasserstein (PRW) distance due to non-convexity and non-smoothness.
method Riemannian optimization to efficiently compute PRW/Wasserstein Projection Pursuit (WPP) distance.
result The original formulation of PRW/WPP can be efficiently computed in practice, providing better behavior than its convex relaxation.
Conditional gradients constitute a class of projection-free first-order algorithms for smooth convex optimization. As such, they are frequently used in solving smooth convex optimization problems over polytopes, for which the computational cost of orthogonal projections would be prohibitive. However, they do not enjoy …
New algorithms optimize actions under time-varying constraints without projecting.
problem Optimizing actions under time-varying constraints without projecting.
method Projection-free algorithms using linear optimization oracle.
result Guaranteed ildeO(T3/4) regret and O(T7/8) constraints violation. We propose and study kernel conjugate gradient methods (KCGM) with random projections for least-squares regression over a separable Hilbert space. Considering two types of random projections generated by randomized sketches and Nyström subsampling, we prove optimal statistical results with respect to variants of norms …
CASP improves portfolio optimization by considering asset covariance.
problem Infeasibility in cardinality-constrained portfolio optimization.
method CASP uses volatility-normalized selection and covariance-aware projection.
result CASP-Basic delivers lower portfolio variance than standard Euclidean repair.
The paper develops a method for optimal projection selection in high-dimensional classification.
problem High-dimensional classification with latent variable structure.
method Formulates a latent-variable model and proposes a computationally efficient classifier.
result Explicit rates of convergence for excess risk of the proposed classifier are derived and shown to be optimal.
Study efficient algorithms for nonconvex optimization with state-dependent Markov data.
problem Stochastic optimization with Markovian data and state-dependent transition kernels.
method Projection-based and projection-free algorithms for constrained nonconvex problems.
result The number of oracle calls to achieve an ε-stationary point is O(1/ε2.5). Improved neural network training in low-dimensional random bases.
problem Inefficient optimization in large-scale neural networks.
method Re-draw random subspace at each training step, apply independent projections to different network parts.
result Significantly better optimization performance and efficiency.
Project fair estimators while maintaining accuracy.
problem Making estimators fair without sacrificing accuracy.
method Optimal transport tools to find closest fair estimator.
result Efficiently constructs fair estimators with quantified cost.
s-OTDD compares datasets efficiently without training, robust to class variations.
problem Efficiently compare datasets without training or class variations.
method Moment Transform Projection (MTP) and sliced optimal transport.
result s-OTDD correlates with optimal transport and transfer learning performance.
This paper focuses on projection-free methods for solving smooth Online Convex Optimization (OCO) problems. Existing projection-free methods either achieve suboptimal regret bounds or have high per-iteration computational costs. To fill this gap, two efficient projection-free online methods called ORGFW and MORGFW are …
We revisit the challenge of designing online algorithms for the bandit convex optimization problem (BCO) which are also scalable to high dimensional problems. Hence, we consider algorithms that are \textit{projection-free}, i.e., based on the conditional gradient method whose only access to the feasible decision set, i…
The paper explores properties of projections and gradient methods in hyperbolic space forms.
problem Optimization problems in hyperbolic space forms.
method Intrinsic κ-projection and gradient projection methods.
result Every accumulation point of the sequence generated by the gradient projection method is a stationary point.
Paper optimizes approximating high-dimensional diffusions by independent coordinates.
problem Optimizing approximations of high-dimensional diffusions by independent coordinates.
method Introduces independent projection as optimal for two criteria.
result Independent projection is optimal for two criteria related to entropy and convergence.
A major source of risk in project management is inaccurate forecasts of project costs, demand, and other impacts. The paper presents a promising new approach to mitigating such risk, based on theories of decision making under uncertainty which won the 2002 Nobel prize in economics. First, the paper documents inaccuracy…
Paper bounds subspace estimator error from noisy projections.
problem Estimating subspaces from noisy data.
method Derives perturbation bound on optimal subspace estimator.
result Fundamental result with implications in matrix completion and clustering.
New framework for decentralized optimization of upper-linearizable functions with improved regret and complexity.
problem Decentralized optimization of upper-linearizable functions with general constraints.
method Decentralized projection-free optimization with upper-linearizable function framework.
result Regret of O(T1−θ/2) with communication complexity of O(Tθ) and linear optimization calls of O(T2θ). This work improves understanding of projection robust optimal transport distances.
problem Understanding the behavior of minimum Wasserstein estimators in high-dimensional and misspecified models.
method Adopting projection robust (PR) optimal transport, establishing statistical properties, proposing IPRW distance, and providing asymptotic guarantees.
result Established fundamental statistical properties and proposed new distances that outperform Wasserstein distances empirically.
Optimizes differentially private kernel learning with random projection.
problem Privacy-preserving learning algorithms with optimal performance.
method Differentially private kernel ERM algorithm based on random projection in reproducing kernel Hilbert space.
result Achieves minimax-optimal excess risk rates for various loss functions.
Large sectors of the recent optimization literature focused in the last decade on the development of optimal stochastic first order schemes for constrained convex models under progressively relaxed assumptions. Stochastic proximal point is an iterative scheme born from the adaptation of proximal point algorithm to nois…