New method solves nonsmooth low-rank matrix optimization problems efficiently.
problem Nonsmooth and low-rank matrix optimization problems in statistics and machine learning.
method Low-rank Extragradient Method with warm-start initialization.
result The extragradient method converges to an optimal solution with rate O(1/t) and requires only two low-rank SVDs per iteration. Framework benchmarks optimizers on multiple criteria.
problem Benchmarking optimizers across diverse test functions.
method Union-free generic depth function for partial orders/rankings.
result Identifies central and outlying rankings of optimizers.
New framework solves low-rank optimization problems to certifiable optimality.
problem Low-rank optimization problems with certifiable solutions.
method Mixed-Projection Conic Optimization framework using symmetric projection matrices and outer-approximation algorithms.
result Solves low-rank problems to certifiable optimality, outperforming existing methods.
Paper extends multivariate rank tests for robust subspace detection.
problem Testing distributional similarity in multivariate data.
method Soft and subspace robust multivariate rank tests based on entropy regularized optimal transport.
result Trade-off between detection power and false alarm rate via projections.
Bayesian model ranks treatments in multi-response experiments.
problem Identifying the best treatment among competing ideal properties.
method Bayesian approach with Markov Chain Monte Carlo algorithm.
result Reliable inference of treatment ranks in practice.
Rank-based Bayesian Optimization improves molecule selection in chemical systems.
problem Optimizing chemical compounds using traditional regression models.
method Introducing Rank-based Bayesian Optimization (RBO) using ranking models.
result RBO outperforms regression-based BO, especially for rough landscapes and activity cliffs.
Optimizes tensor rank selection for neural network compression.
problem Finding optimal tensor rank for regression models.
method Analyzes population expressions for training-testing discrepancy under Gaussian design.
result Optimal rank minimizes prediction error and aligns with cross-validation.
A new Riemannian framework optimizes LoRA for faster convergence and better performance.
problem Optimizing low-rank adapters in neural networks to improve convergence and performance.
method Integrates Riemannion optimizer, LoRA initialization, and efficient implementation for geometrically treating low-rank adapters.
result Consistent and noticeable improvements in convergence speed and final task performance over standard LoRA and its modifications.
New method solves matrix completion problems to certifiable optimality.
problem Certifying optimality in low-rank matrix completion.
method Disjunctive branch-and-bound scheme for convex relaxation.
result Decreases optimality gap by two orders of magnitude.
QS-BO optimizes functions using only rank-based feedback.
problem Optimizing expensive functions with unreliable or unavailable metric values.
method Quantile-scaling pipeline to convert ranks into Gaussian targets.
result QS-BO consistently achieves lower objective values and is statistically significant.
Algorithm ensures fair ranking by minority groups alongside majority groups.
problem Ensuring fair ranking of items from minority groups alongside majority groups.
method Optimal transport-based regularizer for individual fairness and efficient optimization algorithm.
result Certifiably individually fair LTR models are achieved.
New approach to convex hulls for low-rank problems.
problem Characterizing convex hulls for low-rank sets.
method Matrix perspective function and orthogonal projection matrices.
result Strong relaxations for various low-rank problems.
We consider the problem of constructing a reduced-rank regression model whose coefficient parameter is represented as a singular value decomposition with sparse singular vectors. The traditional estimation procedure for the coefficient parameter often fails when the true rank of the parameter is high. To overcome this …
A new kernel for ranked data tackles computational challenges.
problem Complex geometric structure and partial rankings make existing algorithms infeasible for real-world applications.
method Derives a graph cut kernel that combines submodular optimization and kernel-based methods.
result The graph cut kernel efficiently handles large-scale ranked data.
RCPO uses ranked choice modeling for better LLM alignment.
problem Pairwise preference optimization limits LLM alignment.
method Unified framework combining preference optimization and ranked choice modeling.
result RCPO outperforms competitive baselines in LLM alignment.
The optimal ranking score between precision and recall is rarely F1 and can be found using specific methods.
problem Finding a meaningful and optimal compromise between precision and recall scores.
method Established a shortest path between precision- and recall-induced rankings, framed the problem as an optimization problem, and provided theoretical tools to find the optimal β.
result F1 and its skew-insensitive version are not optimal tradeoffs between precision and recall scores.
Hashing, or learning binary embeddings of data, is frequently used in nearest neighbor retrieval. In this paper, we develop learning to rank formulations for hashing, aimed at directly optimizing ranking-based evaluation metrics such as Average Precision (AP) and Normalized Discounted Cumulative Gain (NDCG). We first o…
Partial convexification improves tractability of low-rank spectral optimization problems.
problem Minimizing linear objectives subject to matrix inequalities and low-rank constraints.
method Partial convexification of the domain set, deriving rank bounds, and developing a column generation algorithm.
result The partial convexification LSOP-R is equivalent to the original LSOP under certain conditions and yields high-quality solutions.
Paper uses optimal transport-based statistics for change point detection.
problem Change point detection in multivariate data.
method Soft rank energy and entropically regularized optimal transport.
result Soft rank energy performs better in real datasets with strong continuity and convergence properties.
We consider the problem of maximizing an unknown function over a compact and convex set using as few observations as possible. We observe that the optimization of the function essentially relies on learning the induced bipartite ranking rule of f. Based on this idea, we relate global optimization to bipartite ranking w…
Optimal privacy-preserving ranking from noisy comparisons.
problem Protecting individual privacy in ranking from noisy comparisons.
method Differentially private ranking algorithms under edge and individual differential privacy.
result Achieved minimax optimal rates of convergence under privacy constraints.
Efficiently reduces rank of non-negative matrices with quadratic time complexity.
problem Efficiently reducing the rank of non-negative matrices.
method Formulated rank reduction as a mean-field approximation using a log-linear model.
result Optimal solution for minimizing KL divergence can be computed in closed form.
This paper concerns a fundamental class of convex matrix optimization problems. It presents the first algorithm that uses optimal storage and provably computes a low-rank approximation of a solution. In particular, when all solutions have low rank, the algorithm converges to a solution. This algorithm, SketchyCGM, modi…
Equivalent formulations for low-rank matrix optimization are proven.
problem Low-rank matrix optimization with rank constraints.
method Established geometric landscape connections between manifold and factorization formulations.
result Equivalence between manifold and factorization formulations at FOSPs, SOSPs, and strict saddles.
Physics-inspired methods optimize SVD compression of LLMs.
problem Efficiently compressing large language models (LLMs) using SVD.
method FermiGrad for globally optimal rank selection and PivGa for lossless compression.
result Global optimization of SVD ranks and lossless compression of low-rank factors.
Ranking models are typically designed to provide rankings that optimize some measure of immediate utility to the users. As a result, they have been unable to anticipate an increasing number of undesirable long-term consequences of their proposed rankings, from fueling the spread of misinformation and increasing polariz…
Uplift modeling has effectively been used in fields such as marketing and customer retention, to target those customers that are most likely to respond due to the campaign or treatment. Uplift models produce uplift scores which are then used to essentially create a ranking. We instead investigate to learn to rank direc…
Convex optimization method recovers low-rank matrices from rank-one projections efficiently.
problem Recovering low-rank matrices from limited rank-one projections.
method Unlifted convex optimization with subgradient method.
result The estimator succeeds with high probability if the number of measurements exceeds r2(d1+d2) up to logarithmic factors. This work studies low-rank approximation of a positive semidefinite matrix from partial entries via nonconvex optimization. We characterized how well local-minimum based low-rank factorization approximates a fixed positive semidefinite matrix without any assumptions on the rank-matching, the condition number or eigensp…
AdaRL improves robust RL by adaptively adjusting policy complexity.
problem Handling epistemic uncertainty in environment dynamics.
method Bi-level optimization framework with adaptive rank adjustment.
result AdaRL outperforms existing methods on MuJoCo benchmarks.
We introduce a new framework for optimal transport using Schatten-p regularization to recover low-rank structures.
problem Optimal transport problems with low-rank structure recovery.
method Schatten-p norm regularization to promote low-rank structure in transport maps and plans.
result Unified convex programs for low-rank structure recovery with theoretical guarantees and efficient algorithms.
This paper tackles ranking-based performance normalization for optimization algorithms.
problem Ranking optimization algorithms across diverse numerical scales disrupts performance comparisons.
method Introduces absolute ranking and a sampling-based computational method to address numerical scale variation.
result Provides a more robust framework for assessing performance across multiple algorithms and problems.
StochasticRank optimizes ranking metrics efficiently and guarantees global convergence.
problem Optimizing discrete ranking metrics due to their ill-posed nature.
method Stochastic smoothing, gradient estimate, debiasing, and Stochastic Gradient Langevin Boosting.
result Global convergence and superior performance on ranking datasets.
New algorithm ranks players from partial comparisons with optimal rate.
problem Ranking players from partial pairwise comparisons.
method Divide-and-conquer approach, local MLE within groups.
result Optimal ranking algorithm with minimax rate.
Optimization problems with rank constraints appear in many diverse fields such as control, machine learning and image analysis. Since the rank constraint is non-convex, these problems are often approximately solved via convex relaxations. Nuclear norm regularization is the prevailing convexifying technique for dealing …
Truncated Singular Value Decomposition (SVD) calculates the closest rank-k approximation of a given input matrix. Selecting the appropriate rank k defines a critical model order choice in most applications of SVD. To obtain a principled cut-off criterion for the spectrum, we convert the underlying optimization prob…
Unified framework for statistical inference of low-rank tensors.
problem Statistical inference for tensors in high-dimensional data.
method Unified framework using debiasing and tangent space projection.
result Achieves asymptotic normality and minimax-optimal confidence intervals.
Robust PCA is a widely used statistical procedure to recover a underlying low-rank matrix with grossly corrupted observations. This work considers the problem of robust PCA as a nonconvex optimization problem on the manifold of low-rank matrices, and proposes two algorithms (for two versions of retractions) based on ma…
GeLoRA optimizes LoRA fine-tuning by dynamically adjusting ranks based on intrinsic dimensionality.
problem Efficient fine-tuning of large language models with limited computational resources.
method GeLoRA computes intrinsic dimensionality to adaptively select LoRA ranks, balancing expressivity and efficiency.
result GeLoRA consistently outperforms recent baselines within the same parameter budget on multiple tasks.
Paper introduces MPES for top-k ranking BO with preferential observations.
problem Handling top-k ranking and tie/indifference observations in Bayesian optimization.
method Designs a surrogate model and introduces MPES acquisition function.
result MPES outperforms existing acquisition functions in handling preferential observations.
New model enhances SPIM for solving low-rank combinatorial optimization and statistical learning problems.
problem Solving large-scale combinatorial optimization problems efficiently.
method Proposed a new computing model for SPIM that can handle low-rank interaction matrices.
result Demonstrated efficient learning, classification, and sampling of MNIST images using the model.
Item recommendation is the task of predicting a personalized ranking on a set of items (e.g. websites, movies, products). In this paper, we investigate the most common scenario with implicit feedback (e.g. clicks, purchases). There are many methods for item recommendation from implicit feedback like matrix factorizatio…
Optimizes ranking of top-k players from partial comparison data.
problem Identifying the top-k players from incomplete pairwise comparisons.
method Maximum Likelihood Estimator (MLE) and Spectral Method.
result MLE achieves optimal partial and exact recovery, while Spectral Method is sub-optimal.
We study the problem of approximate ranking from observations of pairwise interactions. The goal is to estimate the underlying ranks of n objects from data through interactions of comparison or collaboration. Under a general framework of approximate ranking models, we characterize the exact optimal statistical error …
We study the problem of learning to rank from multiple information sources. Though multi-view learning and learning to rank have been studied extensively leading to a wide range of applications, multi-view learning to rank as a synergy of both topics has received little attention. The aim of the paper is to propose a c…
PLUMAGE improves large model training efficiency and stability.
problem Accelerator memory and networking constraints during large model training.
method Probabilistic Low rank Unbiased Minimum Variance Gradient Estimator (PLUMAGE) that resolves bias and variance issues.
result PLUMAGE reduces training loss by 28% on average across the GLUE benchmark.
Optimizes wide low-rank neural networks for reduced parameters and cost.
problem Reducing the number of learnable parameters in wide neural networks.
method Analyzed edge-of-chaos dynamics and derived formulae for optimal weight and bias variances.
result Optimal weight and bias variances for low-rank networks follow from multiplicative scaling.
Ranking problems, also known as preference learning problems, define a widely spread class of statistical learning problems with many applications, including fraud detection, document ranking, medicine, credit risk screening, image ranking or media memorability. In this article, we systematically review different types…