arXiv research

DM2L tackles missing labels in multi-label learning by modeling local and global rank structures.

Develops methods to estimate high rank tensors from noisy data.

Sparse sampling method for tensor factorization and completion of high rank tensors.

New bound explains why high-rank neural nets generalize well.

Recent advances in matrix completion enable data imputation in full-rank matrices by exploiting low dimensional (nonlinear) latent structure. In this paper, we develop a new model for high rank matrix completion (HRMC), together with batch and online methods to fit the model and out-of-sample extension to complete new …

We show that given an estimate $\widehat{A}$ that is close to a general high-rank positive semi-definite (PSD) matrix $A$ in spectral norm (i.e., $\|\widehat{A}-A\|_2 \leq δ$), the simple truncated SVD of $\widehat{A}$ produces a multiplicative approximation of $A$ in Frobenius norm. This observation leads to many inte…

We investigate how to train kernel approximation methods that generalize well under a memory budget. Building on recent theoretical work, we define a measure of kernel approximation error which we find to be more predictive of the empirical generalization performance of kernel approximation methods than conventional me…

Given a graphical model (GM), computing its partition function is the most essential inference task, but it is computationally intractable in general. To address the issue, iterative approximation algorithms exploring certain local structure/consistency of GM have been investigated as popular choices in practice. Howev…

Paper introduces HRPCFD for efficient training of stochastic processes.

We study the variety of actions of a fixed (Chevalley) group on arbitrary geodesic, Gromov hyperbolic spaces. In high rank we obtain a complete classification. In rank one, we obtain some partial results and give a conjectural picture.

Flora uses random projections to achieve high-rank updates with low memory usage.

We consider a generalization of low-rank matrix completion to the case where the data belongs to an algebraic variety, i.e. each data point is a solution to a system of polynomial equations. In this case the original matrix is possibly high-rank, but it becomes low-rank after mapping each column to a higher dimensional…

New saddle network architectures preserve convex-concave geometry in optimization problems.

A method for learning complex functions from data with reduced memory usage.

We uncover a large and significant low-minus-high rank effect for commodities across two centuries. There is nothing anomalous about this anomaly, nor is it clear how it can be arbitraged away. Using nonparametric econometric methods, we demonstrate that such a rank effect is a necessary consequence of a stationary rel…

New method solves nonsmooth low-rank matrix optimization problems efficiently.

Given a compact Riemannian spin manifold with positive scalar curvature, we find a family of connections $\nabla^{A_t}$ for $t\in[0,1]$ on a trivial vector bundle of sufficiently high rank, such that the first eigenvalue of the twisted Dirac operator $D_{A_t}$ is nonzero and becomes arbitrarily small as $t\to1$. Howeve…

New method clusters incomplete data by fusing subspaces.

New method recovers matrices with nonlinear structures using optimization on Grassmann manifold.

New method estimates tensors from noisy data with missing entries.

Improves tensor networks for classifying medical images.

Study mapping class group actions on cohomology of configuration spaces.

DoRA improves adaptation efficiency for large models by factoring norms and fusing kernels.

Flow-SSN improves segmentation efficiency and accuracy.

Classifies GL(2,R)-invariant subvarieties in complex geometry.

The Frank-Wolfe (FW) algorithm has been widely used in solving nuclear norm constrained problems, since it does not require projections. However, FW often yields high rank intermediate iterates, which can be very expensive in time and space costs for large problems. To address this issue, we propose a rank-drop method …

We prove the holomorphic rigidity conjecture of Teichmüller space which loosely speaking states that the action of the mapping class group uniquely determines the Teichmüller space as a complex manifold. The method of proof is through harmonic maps. We prove that the singular set of a harmonic map from a smooth $n$-dim…

We introduce a Noise-based prior Learning (NoL) approach for training neural networks that are intrinsically robust to adversarial attacks. We find that the implicit generative modeling of random noise with the same loss function used during posterior maximization, improves a model's understanding of the data manifold …

Unified bounds for neural networks incorporating physical laws.

Robust principal component analysis (RPCA) can recover low-rank matrices when they are corrupted by sparse noises. In practice, many matrices are, however, of high-rank and hence cannot be recovered by RPCA. We propose a novel method called robust kernel principal component analysis (RKPCA) to decompose a partially cor…

This paper studies how to sketch element-wise functions of low-rank matrices. Formally, given low-rank matrix A = [Aij] and scalar non-linear function f, we aim for finding an approximated low-rank representation of the (possibly high-rank) matrix [f(Aij)]. To this end, we propose an efficient sketching-based algorithm…

This paper develops new methods to recover the missing entries of a high-rank or even full-rank matrix when the intrinsic dimension of the data is low compared to the ambient dimension. Specifically, we assume that the columns of a matrix are generated by polynomials acting on a low-dimensional intrinsic variable, and …

Develops a new nonparametric trace regression model for high-dimensional data.

Generalized linear models with nonlinear feature transformations are widely used for large-scale regression and classification problems with sparse inputs. Memorization of feature interactions through a wide set of cross-product feature transformations are effective and interpretable, while generalization requires more…

This review synthesizes missing data imputation across diverse fields.

Black-box neural network models are widely used in industry and science, yet are hard to understand and interpret. Recently, the attention mechanism was introduced, offering insights into the inner workings of neural language models. This paper explores the use of attention-based neural networks mechanism for estimatin…

Anomaly detectors are often used to produce a ranked list of statistical anomalies, which are examined by human analysts in order to extract the actual anomalies of interest. Unfortunately, in realworld applications, this process can be exceedingly difficult for the analyst since a large fraction of high-ranking anomal…

This paper considers the problem of completing a matrix with many missing entries under the assumption that the columns of the matrix belong to a union of multiple low-rank subspaces. This generalizes the standard low-rank matrix completion problem to situations in which the matrix rank can be quite high or even full r…

Rank aggregation based on pairwise comparisons over a set of items has a wide range of applications. Although considerable research has been devoted to the development of rank aggregation algorithms, one basic question is how to efficiently collect a large amount of high-quality pairwise comparisons for the ranking pur…

This paper proposes a probabilistic imputation method with uncertainty quantification.

SGD can jump from high rank minima to low rank minima in DLNs, but not back.

The paper explores how kernel eigenalignments affect generalization in KRR.

We revisit the use of Stochastic Gradient Descent (SGD) for solving convex optimization problems that serve as highly popular convex relaxations for many important low-rank matrix recovery problems such as \textit{matrix completion}, \textit{phase retrieval}, and more. The computational limitation of applying SGD to so…

Improved tensor rank learning for CPD models using a generalized hyperbolic prior.

The study of hierarchy in networks of the human brain has been of significant interest among the researchers as numerous studies have pointed out towards a functional hierarchical organization of the human brain. This paper provides a novel method for the extraction of hierarchical connectivity components in the human …

VLM judges rank well but score poorly; task difficulty and annotation quality affect interval width.

We consider the related tasks of matrix completion and matrix approximation from missing data and propose adaptive sampling procedures for both problems. We show that adaptive sampling allows one to eliminate standard incoherence assumptions on the matrix row space that are necessary for passive sampling procedures. Fo…

We present supersymmetric, curved space, quantum mechanical models based on deformations of a parabolic subalgebra of osp(2p+2|Q). The dynamics are governed by a spinning particle action whose internal coordinates are Lorentz vectors labeled by the fundamental representation of osp(2p|Q). The states of the theory are t…