arXiv research

CoreFlow models matrix-valued distributions efficiently, preserving shared low-rank structure.

Projection-cost preservation is a low-rank approximation guarantee which ensures that the cost of any rank-$k$ projection can be preserved using a smaller sketch of the original data matrix. We present a general structural result outlining four sufficient conditions to achieve projection-cost preservation. These condit…

SLTR model preserves tensor structure and reduces prediction time costs.

Low-rank approximation is an effective model compression technique to not only reduce parameter storage requirements, but to also reduce computations. For convolutional neural networks (CNNs), however, well-known low-rank approximation methods, such as Tucker or CP decomposition, result in degraded model accuracy becau…

The paper establishes theoretical foundations for low-rank knowledge distillation in LLMs.

New method improves tensor completion by selectively preserving important elements.

Multi-view spectral clustering, which aims at yielding an agreement or consensus data objects grouping across multi-views with their graph laplacian matrices, is a fundamental clustering problem. Among the existing methods, Low-Rank Representation (LRR) based method is quite superior in terms of its effectiveness, intu…

Sign-RIP improves robust low-rank matrix recovery by preserving norms even with corrupted measurements.

IMPACT optimizes LLM compression by focusing on activation importance, reducing model size up to 55.4%.

LR-EDNN reduces PDE solver complexity by limiting network weights to low-rank subspace.

Proposes MvLPE for better multi-view representation learning.

This paper tackles fitting multilevel low rank matrices by addressing three problems.

Proposes a faster Isomap algorithm by reducing eigenvalue decomposition complexity.

This paper describes a suite of algorithms for constructing low-rank approximations of an input matrix from a random linear image of the matrix, called a sketch. These methods can preserve structural properties of the input matrix, such as positive-semidefiniteness, and they can produce approximations with a user-speci…

What learning algorithms can be run directly on compressively-sensed data? In this work, we consider the question of accurately and efficiently computing low-rank matrix or tensor factorizations given data compressed via random projections. We examine the approach of first performing factorization in the compressed dom…

Asynchronous federated modeling improves spatial data sharing without centralizing raw data.

Efficient algorithm for Hadamard decomposition of matrices.

Tensor decomposition on big data has attracted significant attention recently. Among the most popular methods is a class of algorithms that leverages compression in order to reduce the size of the tensor and potentially parallelize computations. A fundamental requirement for such methods to work properly is that the lo…

LaLoRA prevents forgetting in LoRA fine-tuning.

We simplify SSL by approximating redundant structural components with low-rank factorization.

Feature selection methods are widely used in order to solve the 'curse of dimensionality' problem. Many proposed feature selection frameworks, treat all data points equally; neglecting their different representation power and importance. In this paper, we propose an unsupervised hypergraph feature selection method via …

Proposes a low-rank bilinear pooling model for link prediction in knowledge graphs.

Faster GW alignment for incomparable point clouds via low-rank couplings.

A new method classifies color images using quaternion algebra.

Paper proposes CLAIR for efficient LLM fine-tuning across clients.

The objective in extreme multi-label learning is to train a classifier that can automatically tag a novel data point with the most relevant subset of labels from an extremely large label set. Embedding based approaches make training and prediction tractable by assuming that the training label matrix is low-rank and hen…

A federated model learns shared archetypes from heterogeneous clients in continual learning.

Tensorized random projections reduce high-dimensional tensor size efficiently.

SRCA reduces high-dimensional data to lower dimensions while preserving geometric structures.

GAME improves matrix completion by considering subgroup-specific latent structures.

Low-rank framework for task-specific LLM ranking from sparse comparisons.

Meta learns low-rank covariance factors for better uncertainty estimation.

A fast method learns plasma collision kernels from simulations, improving kinetic models.

Paper proposes a method to learn structure from data similarities.

GLSKF improves tensor completion by capturing both global and local variations.

Paper proposes a novel MTL framework for personalized modeling of diverse individuals.

In an era of ubiquitous large-scale streaming data, the availability of data far exceeds the capacity of expert human analysts. In many settings, such data is either discarded or stored unprocessed in datacenters. This paper proposes a method of online data thinning, in which large-scale streaming datasets are winnowed…

A new method for filling in missing traffic data improves accuracy over existing techniques.

The low displacement rank (LDR) framework for structured matrices represents a matrix through two displacement operators and a low-rank residual. Existing use of LDR matrices in deep learning has applied fixed displacement operators encoding forms of shift invariance akin to convolutions. We introduce a class of LDR ma…

Study robust recovery of low-rank matrices from corrupted measurements without rank prior.

The paper tackles transfer learning for growing matrix representations, improving estimation accuracy.

Dimensionality reduction is a main step in the learning process which plays an essential role in many applications. The most popular methods in this field like SVD, PCA, and LDA, only can be applied to data with vector format. This means that for higher order data like matrices or more generally tensors, data should be…

Efficiently completes low-rank matrices with nearly linear time complexity.

We propose tensorial neural networks (TNNs), a generalization of existing neural networks by extending tensor operations on low order operands to those on high order ones. The problem of parameter learning is challenging, as it corresponds to hierarchical nonlinear tensor decomposition. We propose to solve the learning…

New preconditioners speed up SGD on Lie groups.

DMF improves POI recommendation privacy and efficiency.

New method clusters multi-view data by squeezing hybrid knowledge.

Data similarity is a key concept in many data-driven applications. Many algorithms are sensitive to similarity measures. To tackle this fundamental problem, automatically learning of similarity information from data via self-expression has been developed and successfully applied in various models, such as low-rank repr…