New algorithm reduces adaptive regret without projections.
problem Computational expense of projections in online convex optimization.
method Lazy gradient-based algorithm with set-membership computations.
result Near-optimal adaptive regret bounds for general convex functions.
Three new efficient algorithms project vectors onto weighted l1 ball.
problem Sparse system identification and feature selection.
method Projected gradient descent algorithms with linear or highly competitive quadratic worst case complexities.
result Efficient tools for machine learning methods like compress sensing and feature selection.
A novel PP algorithm using GMMs and GAs for detecting informative structures.
problem Detecting informative structures in multivariate datasets.
method Gaussian mixture models (GMMs) and Genetic Algorithms (GAs) for optimal projection.
result The approach effectively detects informative structures in multivariate datasets.
Efficiently projects points onto polytopes, especially useful in web-scale applications.
problem Efficiently projecting points onto polytopes in large-scale applications.
method Developed a vertex-oriented incremental algorithm for polytope projection, tailored for simplex and unit-box cut polytopes.
result Majority of projections lie on vertices of polytopes, leading to significant performance improvements.
New algorithm reduces online learning iterations by a factor of T^2/3.
problem Efficiency in online learning with smooth cost functions.
method Follow-the-Perturbed-Leader method using online primal-dual framework.
result Guaranteed T^2/3 regret for general online convex optimization.
New projection techniques reduce the frequency of projections in solving LCPs.
problem Solving linearly constrained problems efficiently with reduced projection frequency.
method Delayed projection technique to call a projection less frequently.
result Theoretical and practical improvements in convergence rates and efficiency.
Consider convex optimization problems subject to a large number of constraints. We focus on stochastic problems in which the objective takes the form of expected values and the feasible set is the intersection of a large number of convex sets. We propose a class of algorithms that perform both stochastic gradient desce…
EIM algorithm maximizes information projection for multi-modal data modeling.
problem Challenging task of modeling highly multi-modal data.
method Expected Information Maximization (EIM) algorithm using variational upper bound.
result EIM algorithm efficiently optimizes the I-projection for Gaussian mixtures models.
Paper solves robust multi-dimensional scaling with accelerated projections.
problem Localize point locations from noisy pairwise distances.
method Alternating projections with tangent space acceleration.
result Linear convergence of reconstructed points to original points.
An algorithm determines knot colorability and determinants from petal projections.
problem Determining knot colorability and determinants from petal projections.
method Algorithm based on petal projections and permutations.
result Determinants of all prime knots with crossing number less than 10 computed.
Cooper and Long generalised Epstein and Penner's Euclidean cell decomposition of cusped hyperbolic manifolds of finite volume to non-compact strictly convex projective manifolds of finite volume. We show that Weeks' algorithm to compute this decomposition for a hyperbolic surface generalises to strictly convex projecti…
In this paper, we propose a new fast and robust recursive algorithm for near-separable nonnegative matrix factorization, a particular nonnegative blind source separation problem. This algorithm, which we refer to as the successive nonnegative projection algorithm (SNPA), is closely related to the popular successive pro…
A faster Wasserstein k-means algorithm for histogram data reduces computation and maintains clustering quality.
problem Efficiently clustering histogram data with reduced computation time.
method Sparse simplex projection to reduce data samples, centroids, and ground cost matrix, dynamically removing lower-valued samples.
result Significant reduction in computational complexity without compromising clustering quality.
We present a novel algorithm for deciding whether a given planar curve is an image of a given spatial curve, obtained by a central or a parallel projection with unknown parameters. The motivation comes from the problem of establishing a correspondence between an object and an image, taken by a camera with unknown posit…
The paper introduces DP algorithms using random projections and sign random projections for improved privacy in machine learning.
problem Improving differential privacy in machine learning applications.
method Developed algorithms based on random projections and sign random projections, focusing on individual differential privacy (iDP) and standard differential privacy (DP).
result DP-SignOPORP and iDP-SignRP achieve superior performance in differential privacy, especially for small epsilon values.
Diverse projection ensembles improve distributional reinforcement learning.
problem Learning the distribution of returns in reinforcement learning.
method Combining multiple projection methods to improve model diversity and exploration.
result Diverse projection ensembles lead to significant performance improvements in exploration tasks.
Solves large-scale metric constrained problems using Project and Forget algorithm.
problem Finding consistent metric representations for large dissimilarity datasets.
method Active set algorithm with Bregman projections, converges to global optimal solution.
result Algorithm efficiently solves metric constrained problems with exponentially many constraints.
Improved regret bounds for scalable bandit convex optimization.
problem Designing online algorithms for high-dimensional bandit convex optimization.
method Projection-free algorithms using a linear optimization oracle.
result First algorithm with O(T3/4) expected regret in O(T) calls. Random projections have been applied in many machine learning algorithms. However, whether margin is preserved after random projection is non-trivial and not well studied. In this paper we analyse margin distortion after random projection, and give the conditions of margin preservation for binary classification problem…
New framework reduces private mean estimation error with optimal efficiency.
problem Locally private mean estimation of high-dimensional vectors.
method ProjUnit framework: random projections, normalization, and optimal algorithm execution in lower dimensions.
result Optimal error up to a 1+o(1)-factor with computational efficiency and low communication complexity.
The paper addresses the k-tangle enumeration problem. We introduce a notion of cascade diagram for k-tangle projections. An effective enumeration algorithm for projections is proposed based on cascade representation. Tangles projections with up to 12 crossings are tabulated. We provide also pictures of alternating …
Product models of low dimensional experts are a powerful way to avoid the curse of dimensionality. We present the ``under-complete product of experts' (UPoE), where each expert models a one dimensional projection of the data. The UPoE is fully tractable and may be interpreted as a parametric probabilistic model for pro…
New algorithm uses random projections for robust, sparse data classification.
problem Improving robustness and sparsity in data classification.
method Randomly projects data into a high-dimensional space, truncates small entries, and applies a cap operation.
result The method enhances classification accuracy with minimal loss, especially in noisy conditions.
We provide an elementary proof of a simple, efficient algorithm for computing the Euclidean projection of a point onto the probability simplex. We also show an application in Laplacian K-modes clustering.
This paper focuses on convex constrained optimization problems, where the solution is subject to a convex inequality constraint. In particular, we aim at challenging problems for which both projection into the constrained domain and a linear optimization under the inequality constraint are time-consuming, which render …
A new framework for dimension reduction using ensemble of random projections.
problem High-dimensional regression problems with limited data.
method Aggregating an ensemble of carefully chosen random projections, retaining based on empirical performance, and selecting singular vectors.
result The proposed method stabilizes error as the number of projection groups increases.
New algorithm improves sparse-view tomography without needing ground-truth data.
problem Poor image reconstructions with sparse projections and non-uniform sensors.
method Unsupervised deep learning with CNN and STN modules.
result Significantly outperforms filtered backprojection in sparse-view scenarios.
A new faster neural network training method using backprojection.
problem Training feedforward neural networks more efficiently.
method Projection and reconstruction at each layer to force projected data and reconstructed labels to be similar.
result The proposed method is faster than backpropagation and gives insights into networks.
New framework solves low-rank optimization problems to certifiable optimality.
problem Low-rank optimization problems with certifiable solutions.
method Mixed-Projection Conic Optimization framework using symmetric projection matrices and outer-approximation algorithms.
result Solves low-rank problems to certifiable optimality, outperforming existing methods.
A new projection method for convex optimization reduces computation costs.
problem Efficiently projecting points into convex sets for deep learning.
method Interpolation-based projection for cheaper computation.
result The proposed method converges for linear and convex constraints.
Paper analyzes LPSA algorithm for constrained optimization, revealing phase transitions and bias-variance trade-offs.
problem Optimization problems with linear constraints.
method Loopless projection stochastic approximation (LPSA) with jump diffusion approximation.
result LPSA trajectories converge to SDEs, revealing asymptotic behaviors and phase transitions.
Python package for projecting onto quadratic hypersurfaces.
problem Projections onto non-cylindrical central quadratic hypersurfaces.
method User-friendly Python package with documentation.
result Efficiently projects points onto quadratic hypersurfaces.
A dissertation on scalable projection-free optimization methods.
problem Efficient optimization algorithms for large-scale machine learning problems.
method Study of Frank-Wolfe variants and their extensions to distributed and derivative-free settings.
result Development of 1-SFW and QFW, achieving state-of-the-art complexity and efficiency.
Tukey median performance analyzed under TV corruptions.
problem Performance analysis of Tukey median under TV corruptions.
method Analysis of Tukey median and projection algorithm under TV corruptions.
result Breakdown point reduced to 1/4 under TV corruptions, compared to 1/3 under Huber's model.
Enhances projection pursuit tree classifier with visual diagnostics for better multi-class classification.
problem Rigidity of original algorithm limits performance in complex high-dimensional classification problems.
method Allowing more splits and flexible class groupings in projection pursuit computation, and developing visual diagnostics.
result Demonstrates enhanced classifier performs as intended through interactive visual diagnostics.
The paper links knot properties in projective space to their fundamental groups.
problem Understanding the fundamental group of knots in projective space.
method Relates knot properties to the fundamental group of the space minus the knot.
result Provides a simple algorithm to find generators and relations for the fundamental group.
We present a number of conditions which are necessary for an n-dimensional projective structure (M,[nabla]) to include the Levi-Civita connection nabla of some metric on M. We provide an algorithm, which effectively checks if a Levi-Civita connection is in the projective class and, in the positive, which finds this con…
The Whitney embedding theorem gives an upper bound on the smallest embedding dimension of a manifold. If a data set lies on a manifold, a random projection into this reduced dimension will retain the manifold structure. Here we present an algorithm to find a projection that distorts the data as little as possible.
Inference in general Ising models is difficult, due to high treewidth making tree-based algorithms intractable. Moreover, when interactions are strong, Gibbs sampling may take exponential time to converge to the stationary distribution. We present an algorithm to project Ising model parameters onto a parameter set that…
In this paper, we study a family of non-convex and possibly non-smooth inf-projection minimization problems, where the target objective function is equal to minimization of a joint function over another variable. This problem include difference of convex (DC) functions and a family of bi-convex functions as special cas…
New algorithm solves complex optimization problems without needing projections.
problem Optimizing nested functions under convex constraints with noisy evaluations.
method Projection-free conditional gradient-type algorithm for smooth stochastic multi-level composition optimization.
result The algorithm achieves ε-stationary solutions with complexity bounds independent of ε and T. Study efficient algorithms for nonconvex optimization with state-dependent Markov data.
problem Stochastic optimization with Markovian data and state-dependent transition kernels.
method Projection-based and projection-free algorithms for constrained nonconvex problems.
result The number of oracle calls to achieve an ε-stationary point is O(1/ε2.5). We extend the mixtures of Gaussians (MOG) model to the projected mixture of Gaussians (PMOG) model. In the PMOG model, we assume that q dimensional input data points z_i are projected by a q dimensional vector w into 1-D variables u_i. The projected variables u_i are assumed to follow a 1-D MOG model. In the PMOG model…
Improves deep learning with less labeled data using unsupervised projection.
problem Lack of labeled data for deep learning models.
method Modified unsupervised discriminant projection as a regularization term for semi-supervised learning.
result Proposes an algorithm that enhances classification performance with minimal labeled data.
In this paper, we consider the problem of learning high-dimensional tensor regression problems with low-rank structure. One of the core challenges associated with learning high-dimensional models is computation since the underlying optimization problems are often non-convex. While convex relaxations could lead to polyn…
New algorithms optimize actions under time-varying constraints without projecting.
problem Optimizing actions under time-varying constraints without projecting.
method Projection-free algorithms using linear optimization oracle.
result Guaranteed ildeO(T3/4) regret and O(T7/8) constraints violation. Paper proposes PRWB and RPRWB for Wasserstein barycenters.
problem Numerical challenges in computing Wasserstein barycenters.
method Projection robust Wasserstein barycenter (PRWB) and relaxed PRWB (RPRWB).
result RPRWB improves clustering performance on real text datasets.
Recent advances suggest that encoding images through Symmetric Positive Definite (SPD) matrices and then interpreting such matrices as points on Riemannian manifolds can lead to increased classification performance. Taking into account manifold geometry is typically done via (1) embedding the manifolds in tangent space…