Efficiently completes low-rank matrices with nearly linear time complexity.
problem Completing low-rank matrices from a few observed entries.
method Robust alternating minimization framework with approximate updates.
result Achieves nearly linear time complexity in matrix completion.
We consider the problem of solving mixed random linear equations with k components. This is the noiseless setting of mixed linear regression. The goal is to estimate multiple linear models from mixed samples in the case where the labels (which sample corresponds to which model) are not observed. We give a tractable a…
We consider the problem of sparse coding, where each sample consists of a sparse linear combination of a set of dictionary atoms, and the task is to learn both the dictionary elements and the mixing coefficients. Alternating minimization is a popular heuristic for sparse coding, where the dictionary and the coefficient…
AM converges super-linearly for solving mixed linear regression problems.
problem Learning linear regressors from unlabeled observations in multiple linear regression models.
method Alternating Minimization (AM) algorithm, which alternates between label estimation and regression solving.
result AM converges super-linearly in certain parameter regimes, requiring only O(log log(1/ε)) iterations to achieve an error of ε.
Alternating minimization represents a widely applicable and empirically successful approach for finding low-rank matrices that best fit the given data. For example, for the problem of low-rank matrix completion, this method is believed to be one of the most accurate and efficient, and formed a major component of the wi…
A new method reduces sample complexity for meta-learning.
problem Efficiently learn new tasks with minimal data.
method Alternating minimization method (MLLAM) for linear regression tasks.
result MLLAM achieves nearly-optimal estimation error with Ω(logd) samples per task. New algorithm trains ReLU networks via alternating minimization.
problem Training deep neural networks with ReLU activations.
method Alternating minimization of activation patterns and weight updates.
result Proves linear convergence for recovering true parameters.
A meta-learning approach improves the performance of alternating minimization for non-convex optimization problems.
problem Optimizing non-convex problems with multiple variables using alternating minimization.
method Meta-learning based alternating minimization (MLAM) to replace handcrafted updating rules.
result The proposed MLAM method outperforms traditional AM-based methods in various non-convex optimization problems.
Paper analyzes agnostic learning of mixed linear regression without generative models.
problem Learning mixed linear regression without assuming stochastic generation.
method Expectation Maximization (EM) and Alternating Minimization (AM) algorithms.
result AM and EM algorithms converge to population loss minimizers under standard conditions.
New tensor completion method converges linearly and is highly practical.
problem Recovering low-rank tensors from sparse observations.
method Adapted alternating minimization to tensor setting.
result Linear convergence even with highly correlated factors.
We consider the following problem: for which classes of finite groups, and in particular finite simple groups, does the minimal dimension of a faithful, smooth action on a homology sphere coincide with the minimal dimension of a faithful, linear action on a sphere? We prove that the two minimal dimensions coincide for …
Alternating Minimization is a widely used and empirically successful heuristic for matrix completion and related low-rank optimization problems. Theoretical guarantees for Alternating Minimization have been hard to come by and are still poorly understood. This is in part because the heuristic is iterative and non-conve…
New algorithm improves convergence of dictionary learning models.
problem Learning rich image features from data using structured unitary sparsifying operators.
method Alternating minimization for structured unitary sparsifying operator learning with convergence analysis.
result The algorithm converges to the underlying sparsifying model of the data under mild assumptions.
Paper proposes an efficient algorithm for clustering with sparse feature selection.
problem Estimating labels and sparse weights in unsupervised clustering.
method Alternating minimization of Frobenius norm criterion with K-sparse algorithm.
result Significantly improves clustering results on single-cell RNA sequencing datasets.
IRM fails to improve over standard methods in complex settings.
problem Learning invariant features for out-of-distribution generalization.
method Analysis of Invariant Risk Minimization (IRM) and related approaches under a general model.
result IRM can fail catastrophically in non-linear settings, even when test data are similar to training distribution.
New algorithms solve large-scale rank minimization problems efficiently.
problem Large-scale rank minimization problems.
method Define and apply bi-trace and tri-trace norms to rank minimization problems; design efficient linearized alternating minimization algorithms.
result Proved algorithms converge to critical points; provide RSC and MC error bounds.
Paper tackles low-rank matrix recovery with KL property and DC reformulation.
problem Low-rank matrix recovery with coarse rank estimation.
method Adds ℓ2,0-norm and balanced terms to factorized loss function; establishes KL property and DC reformulations. result Establishes KL property of exponent 1/2 for the composite function and its global minimizers. Paper formulates mutual information optimal control for discrete-time systems.
problem Optimal control of discrete-time linear systems with mutual information.
method Formulates MIOCP as an extension of MEOCP, derives optimal policy and prior, proposes alternating minimization algorithm.
result Proposes an alternating minimization algorithm for MIOCP.
Paper models non-linear dynamics from time series data.
problem Modeling non-linear dynamical systems from time series data.
method Introduces latent state modeling and a novel alternating minimization algorithm.
result LaNoLem achieves competitive performance in dynamics estimation and prediction.
Phase retrieval problems involve solving linear equations, but with missing sign (or phase, for complex numbers) information. More than four decades after it was first proposed, the seminal error reduction algorithm of (Gerchberg and Saxton 1972) and (Fienup 1982) is still the popular choice for solving many variants o…
New method improves representation learning from non-i.i.d. and non-isotropic data.
problem Improving representation learning from non-i.i.d. and non-isotropic data.
method Introducing a new adaptation of alternating minimization-descent scheme to reduce noise scaling.
result Established linear convergence to optimal representation with noise scaling down with total source data size.
New methods solve non-Lipschitz smooth problems with guaranteed convergence.
problem Non-Lipschitz smooth problems in machine learning and signal processing.
method Bregman-divergence based algorithms for relatively smooth problems.
result Guaranteed convergence to second-order stationary points for any relatively smooth problem.
Paper tackles MLR prediction error without assuming realizable models.
problem Prediction error in mixture of linear regressions without realizable assumptions.
method Developed algorithms for list-decoding MLR predictions and minimized empirical risk.
result Alternating minimization algorithm finds best fit lines in non-realizable settings.
Develops Frank-Wolfe Augmented Lagrangian for convex optimization.
problem Minimizing functions over intersections of convex sets.
method Frank-Wolfe Augmented Lagrangian (FW-AL) method.
result Sublinear convergence rate for general convex compact sets, linear for polytopes.
Proposes an alternative invariance penalty to address domain generalization issues.
problem Addressing domain generalization problems by finding invariant representations.
method Revisits the Gramian matrix of the data representation to propose an alternative invariance penalty.
result The proposed approach guarantees recovery of an invariant representation under mild conditions.
A new portfolio optimization model minimizes maximum drawdown, offering faster and more robust solutions.
problem Optimizing portfolios during financial distress, especially during crises.
method Linearization of Markowitz model based on maximum drawdown, with a Mixed-Integer Linear Programming variation.
result 200 times faster solving time with a more profitable and robust solution.
New algorithm improves learning efficiency in multi-task contextual bandits.
problem Improving learning efficiency in multi-task contextual bandits.
method Alternating projected gradient descent (GD) and minimization estimator for low-rank feature matrix recovery.
result Proved regret bound for multi-task learning algorithm.
This paper finds all prime alternating knots with minimal warping degree two.
problem Finding knots with minimal warping degree.
method Examined all prime alternating knots and determined those with minimal warping degree two.
result All prime alternating knots with minimal warping degree two were identified.
Proposes a new PCA method that balances Euclidean and angle distances.
problem PCA's loss minimization often uses Euclidean distance, but angle distance is more critical in some fields.
method Introduces a method with constraints to unify Euclidean and angle distances, solving the nonconvex optimization problem with an alternating linearized minimization approach.
result Demonstrates the effectiveness and advantages of the new method over state-of-the-art clustering methods on synthetic and real-world datasets.
Paper tackles robust control policy learning for uncertain systems.
problem Learning control policies for an unknown linear dynamical system with quadratic cost.
method Convex optimization method balancing exploitation and exploration.
result Minimizes worst-case cost by reducing uncertainty in model parameters.
Alternative proof and extension of curvature estimates for minimal immersions.
problem Curvature estimates and Bernstein-type theorems for minimal immersions.
method Iteration method à la De Giorgi, ε-regularity theorem, Caccioppoli inequalities.
result Extension of Schoen--Simon--Yau and Schoen--Simon theorems to 6-dimensional stable minimal immersions.
Max-linear regression problem solved with convex programming.
problem Estimating parameters in max-linear regression models.
method Formulated and analyzed a scalable convex program called anchored regression (AR).
result AR provides high probability recovery of parameters with a sample complexity of k4p. Proposes a convex method for high-dimensional sparse sliced inverse regression.
problem Difficulty in interpreting results and variability in high-dimensional settings.
method Convex formulation and linearized alternating direction methods of multiplier algorithm.
result Upper bound on the subspace distance between estimated and true subspaces.
Study dynamics of alternating minimization for bilinear regression under large system limits.
problem Understanding the time evolution of alternating minimization for bilinear regression.
method Replica method applied to a multi-temperature glassy system.
result Dynamics of alternating minimization can be described by a two-dimensional discrete stochastic process.
New method for better initializations in variational Bayes for deep models.
problem Effective initializations for stochastic variational inference in deep models.
method Layer-wise initialization strategy based on Bayesian linear models.
result Faster and better convergence compared to alternatives.
Paper develops a method to approximate Markov chains with fewer states.
problem Identifying the state aggregation structure of Markov chains with fewer states.
method Proposes a convex optimization problem with a nonnegative factorization approach.
result The method likely converges to the global solution and outperforms existing methods.
AGF explains feature learning in neural networks through alternating steps.
problem Understanding what features neural networks learn and how they learn them.
method AGF is an algorithmic framework that approximates the dynamics of feature learning in two-layer networks.
result AGF provides a unified framework to understand feature learning in neural networks, matching experimental results across various architectures.
TSSM splits neural networks for parallel training with minimal accuracy loss.
problem Accuracy degradation in parallel training of deep neural networks.
method TSSM reformulates alternating minimization to achieve parallelism with minimal accuracy loss.
result TSSM achieves significant speedup without accuracy loss on multiple datasets.
We analyze the problem of learning a single user's preferences in an active learning setting, sequentially and adaptively querying the user over a finite time horizon. Learning is conducted via choice-based queries, where the user selects her preferred option among a small subset of offered alternatives. These queries …
In this paper, we consider the problem of minimizing the sum of two convex functions subject to linear linking constraints. The classical alternating direction type methods usually assume that the two convex functions have relatively easy proximal mappings. However, many problems arising from statistics, image processi…
The paper provides guarantees for an alternating minimization algorithm in dictionary learning.
problem Dictionary learning problem of factorizing samples into a basis and sparse vectors.
method Alternating minimization procedure switching between ℓ1 minimization and gradient descent. result Local convergence guarantees for the alternating minimization algorithm under a new matrix infinity norm condition.
Paper analyzes ADMM convergence for nonconvex Gaussian phase retrieval.
problem Nonconvex optimization in Gaussian phase retrieval.
method Block coordinate descent as ADMM with dual variable fixed.
result Block coordinate descent converges linearly to global minimizer.
Analyzes alternating minimization for nonconvex sets in high-dimensional statistics.
problem Optimizing loss functions over nonconvex sets in high-dimensional statistics.
method Local concavity coefficients for nonconvex sets, alternating minimization, inexact algorithms.
result Reveals distinctions between alternating and non-alternating methods, provides convergence conditions.
Mixed linear regression involves the recovery of two (or more) unknown vectors from unlabeled linear measurements; that is, where each sample comes from exactly one of the vectors, but we do not know which one. It is a classic problem, and the natural and empirically most popular approach to its solution has been the E…
Study of linear classifiers in infinite imbalance scenarios.
problem Behavior of linear discriminant functions in extreme imbalance conditions.
method Analysis of linear classifiers under infinite imbalance, focusing on weight function properties and limit behavior.
result Limiting coefficient vectors reflect robustness or conservatism, optimizing against worst-case alternatives.
Paper proposes iterative trimmed loss minimization for learning from corrupted data.
problem Learning from corrupted training data.
method Iterative trimmed loss minimization, alternating between selecting and retraining samples.
result Recovery of ground truth with linear convergence rate in generalized linear models.
This is the third paper in a series devoted to enumerating the prime alternating knots and links. This paper establishes a method for enumerating the prime alternating links. It is shown that one may choose any prime alternating link diagram of a given minimal crossing size and by applications of just two operators (T …
Minimal grid diagrams for 12-crossing prime knots identified.
problem Identifying minimal grid diagrams for prime knots.
method Listed minimal grid diagrams for 12-crossing prime knots.
result Provided a list of minimal grid diagrams for 12-crossing prime knots.