Improved machine learning with reduced tensor rank constraints and dropout.
problem Efficiently approximating large tensors in machine learning.
method Tree tensor networks with CP rank constraints and tensor dropout.
result Low-rank TTN classifier achieves 90.3% accuracy in Fashion-MNIST.
We analyze the structure of covariance matrices under graph constraints.
problem Analyzing the structure of covariance matrices under graph constraints.
method We explore the algebraic structure of the solution space of convex optimization problem Constrained Minimum Trace Factor Analysis (CMTFA) under a latent star topology.
result CMTFA can have either a rank 1 or a rank n-1 solution, with conditions for both.
The problem of low-rank approximation with convex constraints, which appears in data analysis, system identification, model order reduction, low-order controller design and low-complexity modelling is considered. Given a matrix, the objective is to find a low-rank approximation that meets rank and convex constraints, w…
New algorithm tackles low-rank constraints in optimal transport problems.
problem Optimal transport problems with low-rank constraints.
method Explicit factorization of low-rank couplings as a product of sub-coupling factors linked by a common marginal.
result Stationary convergence of the algorithm proved.
Most learning methods with rank or sparsity constraints use convex relaxations, which lead to optimization with the nuclear norm or the ℓ1-norm. However, several important learning applications cannot benefit from this approach as they feature these convex norms as constraints in addition to the non-convex rank a…
We consider the problem of noisy 1-bit matrix completion under an exact rank constraint on the true underlying matrix M∗. Instead of observing a subset of the noisy continuous-valued entries of a matrix M∗, we observe a subset of noisy 1-bit (or binary) measurements generated according to a probabilistic model. W…
This paper proposes a mechanism to produce equivalent Lipschitz surrogates for zero-norm and rank optimization problems by means of the global exact penalty for their equivalent mathematical programs with an equilibrium constraint (MPECs). Specifically, we reformulate these combinatorial problems as equivalent MPECs by…
Minimizing the rank of a matrix subject to constraints is a challenging problem that arises in many applications in control theory, machine learning, and discrete geometry. This class of optimization problems, known as rank minimization, is NP-HARD, and for most practical problems there are no efficient algorithms that…
Paper improves MVSC using tensor low-rank modeling.
problem Improving multi-view spectral clustering.
method Structured tensor low-rank norm for MVSC optimization.
result Proposed method outperforms state-of-the-art methods.
Extends RRR to capture nonlinear interactions in multi-response regression.
problem Complex relationships in real-world data cannot be adequately modeled by linear interactions.
method Introduces Higher Order Reduced Rank Regression (HORRR) using tensor representations and Tucker decomposition.
result HORRR can capture nonlinear interactions in multi-response regression.
Algorithm samples fair rankings to ensure individual fairness while maintaining group fairness.
problem Fair ranking tasks with group fairness constraints and uncertainty in item utilities.
method Efficient algorithm that samples rankings from an individually-fair distribution ensuring group fairness.
result Expected utility of output ranking is at least α times optimal fair solution, where α depends on utilities and constraints.
New algorithm improves deep learning models' robustness without sacrificing accuracy.
problem Low-rank methods compromise model robustness against adversarial perturbations.
method Robust low-rank training via approximate orthonormal constraints.
result Ensures well-conditioning and better adversarial robustness without sacrificing model accuracy.
Paper develops compact formulations for optimization problems with rank-one convex functions and indicator variables.
problem Optimization problems involving rank-one convex functions with support constraints.
method Perspective reformulation techniques to exploit conic structure and establish convex hull results.
result Systematic perspective formulations for convex hull descriptions of sets with nonlinear separable or non-separable objective functions and combinatorial constraints.
We analyze the local convergence of proximal splitting algorithms to solve optimization problems that are convex besides a rank constraint. For this, we show conditions under which the proximal operator of a function involving the rank constraint is locally identical to the proximal operator of its convex envelope, hen…
Introduces nondecreasing rank for matrices and tensors, developing methods and applications.
problem Finding low-rank approximations for matrices and tensors with monotonic constraints.
method Developed a variant of hierarchical alternating least squares algorithm for finding low ND rank approximations.
result Low ND rank factorizations can be found and interpreted for real-world datasets.
Paper proposes a new algorithm for graph learning with covariance constraints.
problem Graphical models and factor analysis not jointly leveraged in graph learning processes.
method Penalized maximum likelihood estimation of an elliptical distribution with Riemannian optimization.
result Effectiveness of the proposed approach demonstrated on real-world data sets.
Recovery of low-rank matrices has recently seen significant activity in many areas of science and engineering, motivated by recent theoretical results for exact reconstruction guarantees and interesting practical applications. A number of methods have been developed for this recovery problem. However, a principled meth…
Optimization problems with rank constraints appear in many diverse fields such as control, machine learning and image analysis. Since the rank constraint is non-convex, these problems are often approximately solved via convex relaxations. Nuclear norm regularization is the prevailing convexifying technique for dealing …
Many applications require recovering a matrix of minimal rank within an affine constraint set, with matrix completion a notable special case. Because the problem is NP-hard in general, it is common to replace the matrix rank with the nuclear norm, which acts as a convenient convex surrogate. While elegant theoretical c…
Proposes a new method to optimize treatment allocation with budget constraints.
problem Optimizing treatment allocation with budget constraints in marketing.
method Learning to rank for profit maximization.
result Learning to rank can maximize the area under a policy's incremental profit curve.
Classifies links with small Khovanov homology ranks.
problem Classifying links with specific ranks in Khovanov homology.
method Previous results combined with new classifications.
result All links with ranks ≤ 8 and three-component links with ranks ≤ 12 are classified.
Study consumption-investment problem in markets with rank-based returns.
problem Consumption-investment problem in markets with rank-based returns.
method Derives an HJB equation with Neumann boundary conditions for the value function and proves a corresponding verification theorem.
result Explicit solutions for unconstrained, open market constraints, and fully invested cases.
We construct a sequence of primitive-stable representations of free groups into PSL(2,C) whose ranks go to infinity, but whose images are discrete with quotient manifolds that converge geometrically to a knot complement. In particular this implies that the rank and geometry of the image of a primitive-stable representa…
New approach to convex hulls for low-rank problems.
problem Characterizing convex hulls for low-rank sets.
method Matrix perspective function and orthogonal projection matrices.
result Strong relaxations for various low-rank problems.
Proposes a new model for image restoration combining deep learning and total variation.
problem Restoring images from limited data with low-rank constraints insufficient.
method Regularized Deep Matrix Factorized (RDMF) model using deep neural network's low-rank bias and total variation.
result Outperforms state-of-the-art models in image restoration from few observations.
The paper tackles fairness in scoring functions for binary classification.
problem Fairness in scoring functions for binary classification tasks.
method Introduces ROC-based fairness constraints and learning algorithms.
result Generalization bounds and practical learning algorithms for fair scoring functions.
Proposes a new method for rank-consistent ordinal regression without weight-sharing constraints.
problem Ordinal response variables in real-world prediction problems are often ignored by conventional classification losses.
method CORN framework using conditional training sets and the chain rule for conditional probability distributions.
result Improves performance substantially compared to the CORAL reference approach without weight-sharing restrictions.
Paper introduces GAMs for interpretable learning-to-rank models.
problem Need for transparent ranking models in legal or policy scenarios.
method Developed generalized additive models (GAMs) for ranking tasks using neural networks.
result Neural ranking GAMs achieve better performance than traditional GAMs while maintaining interpretability.
New framework for fair ranking with noisy protected attributes.
problem Errors in socially-salient attributes undermine fairness guarantees.
method Modeling perturbations in protected attributes and incorporating probabilistic information.
result Framework provides provable guarantees on fairness and utility.
New metrics improve landing algorithms for orthogonality constraints.
problem Optimizing landing algorithms with orthogonality constraints.
method Proposed a family of metrics over full-rank matrices to enhance landing algorithms.
result Natural extension of β-metric improves landing performance.
Improved unsupervised probing for ranking tasks using Contrast-Consistent Ranking.
problem Improving self-consistency in language model rankings.
method Adapting Contrast-Consistent Search (CCS) to Contrast-Consistent Ranking (CCR) for ranking tasks.
result CCR probing outperforms prompting techniques across different models and datasets.
Low-rank inducing unitarily invariant norms have been introduced to convexify problems with low-rank/sparsity constraint. They are the convex envelope of a unitary invariant norm and the indicator function of an upper bounding rank constraint. The most well-known member of this family is the so-called nuclear norm. To …
Develops TOFU for tensor bandits with low-rank structure.
problem Linear bandit models fail to capture high-dimensional, low-rank tensor structures.
method Develops TOFU, a tensor bandit algorithm that estimates low-dimensional subspaces and uses norm constraints.
result Improves regret bound by a multiplicative factor that grows exponentially in system order.
Multi-view spectral clustering, which aims at yielding an agreement or consensus data objects grouping across multi-views with their graph laplacian matrices, is a fundamental clustering problem. Among the existing methods, Low-Rank Representation (LRR) based method is quite superior in terms of its effectiveness, intu…
New ranking system balances fairness and user utility.
problem Achieving group fairness in ranking systems.
method Formulated a minimax game between a ranking player and an adversary.
result Better utility for highly fair rankings.
This paper proposes robust matrix variate regression models with rank constraints and vector regularization.
problem High dimensional and noisy matrix-valued predictors in regression models.
method Rank constraint, vector regularization, alternating projected gradient descent algorithm.
result The proposed method achieves the minimax rate of estimation errors.
A new algorithm solves constrained optimization problems with stochastic gradients.
problem Nonlinear equality constrained optimization with rank-deficient Jacobians.
method Step decomposition strategy combining normal and tangential steps.
result Convergence guarantees in rank-deficient Jacobian cases.
This work investigates implicit bias in multiclass separable data using a novel geometry-aware optimizer.
problem Understanding implicit bias in overparameterized models on multiclass separable data.
method Introduces NucGD, a geometry-aware optimizer enforcing low-rank structures through nuclear norm constraints.
result NucGD enables scalable training and characterizes the impact of stochastic optimization dynamics.
Algorithm learns fair ranking from biased data.
problem Unfair ranking policies from biased implicit feedback.
method Policy-gradient approach with amortized fairness constraints.
result Efficient algorithm FULTR learns fair policies.
DNN-based cross-modal retrieval has become a research hotspot, by which users can search results across various modalities like image and text. However, existing methods mainly focus on the pairwise correlation and reconstruction error of labeled data. They ignore the semantically similar and dissimilar constraints bet…
Motivated by an application in computational biology, we consider low-rank matrix factorization with {0,1}-constraints on one of the factors and optionally convex constraints on the second one. In addition to the non-convexity shared with other matrix factorization schemes, our problem is further complicated by a c…
PLUMAGE improves large model training efficiency and stability.
problem Accelerator memory and networking constraints during large model training.
method Probabilistic Low rank Unbiased Minimum Variance Gradient Estimator (PLUMAGE) that resolves bias and variance issues.
result PLUMAGE reduces training loss by 28% on average across the GLUE benchmark.
We study the projected gradient descent method on low-rank matrix problems with a strongly convex objective. We use the Burer-Monteiro factorization approach to implicitly enforce low-rankness; such factorization introduces non-convexity in the objective. We focus on constraint sets that include both positive semi-defi…
New framework solves low-rank optimization problems to certifiable optimality.
problem Low-rank optimization problems with certifiable solutions.
method Mixed-Projection Conic Optimization framework using symmetric projection matrices and outer-approximation algorithms.
result Solves low-rank problems to certifiable optimality, outperforming existing methods.
Analyzes learning dynamics of RNNs under locality constraints.
problem Understanding learning dynamics in RNNs with locality constraints.
method Dynamical systems theory applied to data-aligned linear RNNs.
result RFLO solutions are restricted to low-rank perturbations of initial parameters.
Study algebraic invariants from lightning self-attention models.
problem Understanding polynomial coefficients of self-attention mechanisms.
method Identify algebraic invariants using polynomial coefficients and coordinate geometry.
result Found linear and nonlinear families of algebraic invariants.
We consider whether algorithmic choices in over-parameterized linear matrix factorization introduce implicit regularization. We focus on noiseless matrix sensing over rank-r positive semi-definite (PSD) matrices in Rn×n, with a sensing mechanism that satisfies restricted isometry properties (RIP)…
Proposes a low-rank bilinear pooling model for link prediction in knowledge graphs.
problem Link prediction in incomplete knowledge graphs.
method Factorized bilinear pooling model with Tucker decomposition constraints.
result Efficient and parameter-efficient model with low-rank approximation.