Estimates low-rank distributional matrices from incomplete samples.
problem Matrix completion for distributional entries with limited observed data.
method Kernel mean embeddings, Tucker rank, functional unfolding operators.
result Effective estimator for distributional matrix completion established.
Study the distribution for low-rank matrix learning, improving inference methods.
problem Lack of understanding of underlying probability distributions in low-rank matrix learning.
method Analyze the distribution f(X)∝e−λ∥X∥∗, using differential geometry to design an improved MCMC algorithm and learn penalty parameter λ. result Improved MCMC algorithm and penalty parameter learning for low-rank Bayesian inference.
Novel method for efficient low-rank matrix estimation and bandit algorithms.
problem Low-rank matrix estimation and bandit problems.
method LowPopArt method for low-rank matrix estimation and novel experimental design criterion.
result Improved recovery guarantees and regret bounds for low-rank bandit algorithms.
We study the convergence of a variant of distributed gradient descent (DGD) on a distributed low-rank matrix approximation problem wherein some optimization variables are used for consensus (as in classical DGD) and some optimization variables appear only locally at a single node in the network. We term the resulting a…
Paper develops inference methods for low-rank tensors without debiasing.
problem Statistical inference for low-rank tensor models.
method Two-iteration alternating minimization for asymptotic distribution.
result Asymptotic distributions and confidence regions for singular subspaces.
We consider the problem of learning a high-dimensional but low-rank matrix from a large-scale dataset distributed over several machines, where low-rankness is enforced by a convex trace norm constraint. We propose DFW-Trace, a distributed Frank-Wolfe algorithm which leverages the low-rank structure of its updates to ac…
New method reduces summary points for datasets while maintaining quality.
problem Thinning datasets to reduce summary points while maintaining quality.
method Low-rank analysis of sub-Gaussian thinning.
result Guarantees high-quality compression for any distribution and kernel.
Bayesian model improves image completion accuracy by automatically learning low rank structure.
problem Improving image completion accuracy with limited data and avoiding overfitting.
method Developed a Bayesian low rank tensor ring model with multiplicative interaction and Student-T distribution for sparse core factors.
result The proposed method outperforms state-of-the-art image completion techniques, especially in recovery accuracy.
Characterizes uncertainty in low-rank matrix completion with noisy data.
problem Uncertainty quantification in low-rank matrix completion with heterogeneous sub-exponential noise.
method Characterizes the distribution of estimated matrix entries under low-rank estimators with heterogeneous sub-exponential noise.
result Explicit formulas for the distribution of estimated matrix entries under Poisson and Binary noise.
The problem of low rank matrix completion is considered in this paper. To exploit the underlying low-rank structure of the data matrix, we propose a hierarchical Gaussian prior model, where columns of the low-rank matrix are assumed to follow a Gaussian distribution with zero mean and a common precision matrix, and a W…
CoreFlow models matrix-valued distributions efficiently, preserving shared low-rank structure.
problem Learning matrix-valued distributions from high-dimensional and incomplete data.
method Low-rank flow model that learns shared row/column subspaces and trains a normalizing flow on the core.
result CoreFlow improves generation quality in few-sample regimes and remains competitive in data-rich settings.
Low-rank MPPCA improves importance sampling in high dimensions.
problem Estimating full-rank GMM covariance matrices in high dimensions is numerically unstable.
method Use MPPCA mixtures as low-rank proposals for importance sampling in high-dimensional spaces.
result Consistent gains in sample efficiency and quality of failure distribution characterization.
Estimates joint probability distribution from 1-way marginals using low-rank tensors and random projections.
problem Nonparametric estimation of joint probability mass function (PMF) from limited data.
method Low-rank tensor decomposition and random projections to link data to PMF estimation.
result Estimates joint density from 1-way marginals using transformed space and novel algorithm.
New method for robust PCA with exponential family distributions.
problem Recovering low-rank structure from data matrices with outliers.
method Alternating Direction Method of Multipliers for eextRPCA. result Demonstrated effectiveness in steel sheet defect detection and crime activity monitoring.
New method for robust matrix completion with mixed data types.
problem Recovering a structured low rank matrix with mixed data types.
method Proposes a computationally feasible statistical approach with strong recovery guarantees for mixed data types.
result Strong recovery guarantees for low rank matrix completion with mixed data types.
Paper studies model stealing for low-rank language models.
problem Model stealing threatens proprietary models' security and data privacy.
method Theoretical study of model stealing for Hidden Markov Models (HMMs) and low-rank language models using the conditional query model.
result Efficient algorithm for learning any low-rank distribution in the conditional query model.
Paper proposes a new algorithm for graph learning with covariance constraints.
problem Graphical models and factor analysis not jointly leveraged in graph learning processes.
method Penalized maximum likelihood estimation of an elliptical distribution with Riemannian optimization.
result Effectiveness of the proposed approach demonstrated on real-world data sets.
LoCo-RLHF models diverse human feedback with contextual information.
problem Heterogeneous human feedback from diverse contexts and preferences.
method Low-rank contextual preference model, PRS policy.
result LoCo-RLHF achieves tighter sub-optimality gap than existing methods.
New algorithm for weighted low rank approximation with provable guarantees.
problem Weighted low rank approximation (WLRA) is computationally hard.
method Reweights the low rank solution using the weight matrix itself.
result Provably optimal approximation guarantees for WLRA.
We study the problem of learning a distribution from samples, when the underlying distribution is a mixture of product distributions over discrete domains. This problem is motivated by several practical applications such as crowd-sourcing, recommendation systems, and learning Boolean functions. The existing solutions e…
Weight Decay induces low-rank weight matrices in neural networks, improving generalization.
problem Improving generalization in neural networks.
method Training ReLU NN with Weight Decay and Stochastic Gradient Descent.
result The weight matrix of a trained NN is approximately rank-two.
Meta learns low-rank covariance factors for better uncertainty estimation.
problem Sub-optimal covariance matrices in multi-task settings.
method Meta learns diagonal or diagonal plus low-rank factors using an attentive set encoder.
result Efficiently constructed task-specific covariance matrices improve uncertainty estimation.
Paper finds a lower bound for estimating low-rank matrices in logistic regression.
problem Estimating low-rank coefficient matrices in logistic regression.
method Derives a minimax lower bound on the risk.
result The bound depends on matrix dimensions, rank, and sample size.
In this work, we present a novel robust distributed beamforming (RDB) approach based on low-rank and cross-correlation techniques. The proposed RDB approach mitigates the effects of channel errors in wireless networks equipped with relays based on the exploitation of the cross-correlation between the received data from…
Compact parameterization improves Bayesian neural network performance.
problem Improving performance of Bayesian neural networks using variational methods.
method Restricting variational distribution to a k-tied Normal distribution with low-rank factorization.
result Compact parameterization improves signal-to-noise ratio and convergence speed.
Low-rank matrix regression refers to the instances of recovering a low-rank matrix based on specially designed measurements and the corresponding noisy outcomes. In the last decade, numerous statistical methodologies have been developed for efficiently recovering the unknown low-rank matrices. However, in some applicat…
New method for initializing low-rank neural networks improves performance.
problem Training low-rank neural networks efficiently and accurately.
method Inspired by function approximation, proposes a novel low-rank initialization framework.
result Demonstrates significant gap between spectral and low-rank initialization approaches.
Method estimates joint probability density from samples using low-rank decomposition and random projections.
problem Estimating joint probability density from limited samples.
method Low-rank tensor decomposition, dictionaries, and Radon transforms.
result Algorithm outperforms previous methods in estimating synthetic probability densities.
Efficiently reduces tensor ranks using mean-field approximation.
problem Low-rank approximation of non-negative tensors.
method Mean-field approximation of tensor rank reduction.
result Our algorithm achieves faster and competitive tensor rank reduction.
We give an algorithm for completing an order-m symmetric low-rank tensor from its multilinear entries in time roughly proportional to the number of tensor entries. We apply our tensor completion algorithm to the problem of learning mixtures of product distributions over the hypercube, obtaining new algorithmic result…
New algorithm learns low-rank matrices with linear number of samples.
problem Learning low-rank matrices efficiently in latent-variable applications.
method Proposed algorithm that uses linear number of samples in high dimension.
result Learning kimesk, rank-r, matrices requires $Ω( rac{kr}{ε^2})$ samples. TFB simplifies Bayesian LLM uncertainty estimation without extra training.
problem Estimating uncertainty in LLM responses remains challenging.
method Training-Free Bayesianization (TFB) that transforms low-rank adapters into Bayesian ones without additional training.
result TFB achieves superior uncertainty estimation and generalization compared to existing methods.
We study the spherical cap packing problem with a probabilistic approach. Such probabilistic considerations result in an asymptotic sharp universal uniform bound on the maximal inner product between any set of unit vectors and a stochastically independent uniformly distributed unit vector. When the set of unit vectors …
The study examines denoising and noisy-input regression under distribution shift, revealing double descent behavior and insights for data augmentation.
problem Understanding denoising in machine learning, especially under noisy inputs and distribution shift.
method Theoretical analysis of supervised denoising and noisy-input regression, considering low-rank data and proportional regime.
result The test error exhibits double descent under general distribution shift, indicating that overfitting the noise can be benign, tempered, or catastrophic.
Paper proposes recycling model updates in federated learning by exploiting low-rank gradient subspaces.
problem Large parameter transmissions in federated learning.
method Look-back Gradient Multiplier (LBGM) algorithm exploiting low-rank property of gradient subspaces.
result LBGM reduces communication overhead with minimal performance loss.
Efficiently samples complex distributions using tensor train format.
problem Sampling from high-dimensional complex probability densities efficiently.
method Integrates tensor train format with backward stochastic differential equations (BSDEs) for fast, robust, and accurate sampling.
result Improved efficiency in sampling from challenging target distributions.
This paper extends the Gaussian process interpretation of deep networks to more varied weight distributions.
problem Understanding the impact of different weight initialization schemes on deep learning dynamics.
method Extending the Gaussian process interpretation to PSEUDO-IID weight distributions, including sparse and low-rank networks.
result PSEUDO-IID initialized networks are effectively equivalent up to variance, enabling tractable posterior distributions.
There has recently been considerable interest in completing a low-rank matrix or tensor given only a small fraction (or few linear combinations) of its entries. Related approaches have found considerable success in the area of recommender systems, under machine learning. From a statistical estimation point of view, the…
New algorithm for low-rank optimal transport with improved interpretability and efficiency.
problem Quadratic scaling of optimal transport coupling matrix for massive datasets.
method Factor Relaxation with Latent Coupling (FRLC) algorithm.
result Superior performance on diverse applications including graph clustering and spatial transcriptomics.
This paper presents a Bayesian method for estimating the rank of a low-rank tensor model of joint PMF.
problem Estimating the rank of a low-rank tensor model of joint PMF from observed data.
method Bayesian framework for estimating low-rank components and rank simultaneously, using variational inference.
result Automatic rank detection and improved estimation accuracy compared to cross-validation methods.
TSRGA scales multivariate linear regression for feature-distributed data.
problem Multivariate linear regression for feature-distributed data with high dimensions and many computing nodes.
method Two-stage relaxed greedy algorithm (TSRGA) for multivariate linear regression.
result TSRGA is highly scalable and can yield low-rank coefficient estimates.
PowerGossip compresses model differences for decentralized deep learning with low-rank linear compressors.
problem Communication bottleneck in decentralized deep learning models.
method Low-rank linear compressors applied on model differences using power iteration steps.
result Asymptotically independent of network and compression, faster convergence, and comparable performance to tuned compression algorithms.
In statistical connectomics, the quantitative study of brain networks, estimating the mean of a population of graphs based on a sample is a core problem. Often, this problem is especially difficult because the sample or cohort size is relatively small, sometimes even a single subject. While using the element-wise sampl…
This work explores efficient reinforcement learning with density features in low-rank MDPs.
problem Efficient reinforcement learning with density features in low-rank MDPs.
method Proposes algorithms for off-policy estimation and online construction of exploratory data distributions.
result Demonstrates sample-efficient learning with density features in low-rank MDPs, overcoming technical challenges.
In this paper, we develop a novel procedure for low-rank tensor regression, namely \emph{\underline{I}mportance \underline{S}ketching \underline{L}ow-rank \underline{E}stimation for \underline{T}ensors} (ISLET). The central idea behind ISLET is \emph{importance sketching}, i.e., carefully designed sketches based on bot…
We study gradient compression methods to alleviate the communication bottleneck in data-parallel distributed optimization. Despite the significant attention received, current compression schemes either do not scale well or fail to achieve the target test accuracy. We propose a new low-rank gradient compressor based on …
Paper tackles matrix estimation under arbitrary noise, achieving minimax optimality.
problem Noisy low-rank-plus-sparse matrix recovery under arbitrary dependence.
method Incoherent-constrained least-square estimator, novel energy spreading result.
result Achieves minimax optimality in estimating structured Markov transition kernels.
Proposes a tensor Laplacian-based method for better subspace clustering of non-uniformly distributed data.
problem LRR's inability to handle non-uniform data distribution and local information loss.
method Tensor Laplacian Regularized Low-Rank Representation (TLRR) using hypergraph model and tensor Laplacian algorithm.
result Higher accuracy and precision in subspace clustering compared to state-of-the-art methods.