We theoretically and experimentally investigate tensor-based regression and classification. Our focus is regularization with various tensor norms, including the overlapped trace norm, the latent trace norm, and the scaled latent trace norm. We first give dual optimization methods using the alternating direction method …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “Latent Norms”
A new metric assesses latent variable models using data and model moments.
problem Difficulty in assessing the quality of unsupervised learning models.
method A moment-matching metric using matrix norms to compare data and model moments.
result The proposed metric is faster and has less variance than alternative methods.
In this paper, we propose an unifying view of several recently proposed structured sparsity-inducing norms. We consider the situation of a model simultaneously (a) penalized by a set- function de ned on the support of the unknown parameter vector which represents prior knowledge on supports, and (b) regularized in Lp-n…
Estimates latent norms and Gram matrices for graphs on Euclidean balls.
problem Estimating latent points and their relationships in graphs on Euclidean balls.
method Estimates latent norms and Gram matrices using observed graph data.
result Graphs on Euclidean balls can have power-law degree distributions.
We study a norm for structured sparsity which leads to sparse linear predictors whose supports are unions of prede ned overlapping groups of variables. We call the obtained formulation latent group Lasso, since it is based on applying the usual group Lasso penalty on a set of latent variables. A detailed analysis of th…
We discuss structured Schatten norms for tensor decomposition that includes two recently proposed norms ("overlapped" and "latent") for convex-optimization-based tensor decomposition, and connect tensor decomposition with wider literature on structured sparsity. Based on the properties of the structured Schatten norms,…
We propose a set of convex low rank inducing norms for a coupled matrices and tensors (hereafter coupled tensors), which shares information between matrices and tensors through common modes. More specifically, we propose a mixture of the overlapped trace norm and the latent norms with the matrix trace norm, and then, w…
GAME improves matrix completion by considering subgroup-specific latent structures.
problem Heterogeneous data with overlapping categories, smoothing away subgroup-specific variation.
method Group-Aware Matrix Estimation (GAME) with overlapping nuclear-norm penalties.
result GAME outperforms global low-rank estimators in structured missingness regimes.
Neural Processes Mixed-Effect Models for Deep Normative Modeling of Clinical Neuroimaging Datastat.ML
Normative modeling has recently been introduced as a promising approach for modeling variation of neuroimaging measures across individuals in order to derive biomarkers of psychiatric disorders. Current implementations rely on Gaussian process regression, which provides coherent estimates of uncertainty needed for the …
One of the popular approaches for low-rank tensor completion is to use the latent trace norm regularization. However, most existing works in this direction learn a sparse combination of tensors. In this work, we fill this gap by proposing a variant of the latent trace norm that helps in learning a non-sparse combinatio…
Convex optimization method infers latent structure in random dot product graphs.
problem Inferring latent probability matrix of random dot product graphs.
method Conic programming with nuclear norm regularization.
result Asymptotic consistency of probability estimates and recovery of latent structure.
Task shift from classification to regression is possible in overparameterized linear models with limited additional data.
problem Transferability of latent knowledge from classification to regression in overparameterized linear models.
method Investigation of task shift in overparameterized linear regression, zero-shot and few-shot cases, with a focus on minimum-norm interpolation.
result Minimum-norm interpolators can transfer latent knowledge from classification to regression with limited additional data.
Correlation matrices play a key role in many multivariate methods (e.g., graphical model estimation and factor analysis). The current state-of-the-art in estimating large correlation matrices focuses on the use of Pearson's sample correlation matrix. Although Pearson's sample correlation matrix enjoys various good prop…
Unified framework for coupled tensor completion improves recovery accuracy.
problem Improving recovery accuracy in coupled tensor completion.
method Unified framework using tensor ring (TR) decomposition with shared latent factors and novel optimization model.
result The proposed method achieves superior recovery accuracy on real-world data compared to state-of-the-art methods.
Study spectral properties of sparse random graphs to recover latent vectors.
problem Recovering latent vectors in sparse random geometric graphs.
method Analyzes spectral concentration and uses orthogonal polynomial expansions, decoupling, and matrix concentration.
result Sharpens spectral norm bounds and proves exact recovery for Gaussian mixture models.
Rank minimization has attracted a lot of attention due to its robustness in data recovery. To overcome the computational difficulty, rank is often replaced with nuclear norm. For several rank minimization problems, such a replacement has been theoretically proven to be valid, i.e., the solution to nuclear norm minimiza…
Discovering the latent structure from many observed variables is an important yet challenging learning task. Existing approaches for discovering latent structures often require the unknown number of hidden states as an input. In this paper, we propose a quartet based approach which is \emph{agnostic} to this number. Th…
Extracting latent low-dimensional structure from high-dimensional data is of paramount importance in timely inference tasks encountered with `Big Data' analytics. However, increasingly noisy, heterogeneous, and incomplete datasets as well as the need for {\em real-time} processing of streaming data pose major challenge…
FSGD uses latent factors to scale SGD for high-dimensional learning.
problem Scalable optimization in high-dimensional machine learning.
method Factor-Augmented SGD (FSGD) that operates on streaming data.
result Established theoretical framework for latent factor estimation error in SGD.
For graphs generated from stochastic blockmodels, adjacency spectral embedding is asymptotically consistent. Further, adjacency spectral embedding composed with universally consistent classifiers is universally consistent to achieve the Bayes error. However when the graph contains private or sensitive information, trea…
New model for multiplex networks learns shared structure.
problem Complex network structures not well modeled by existing methods.
method Latent space model for multiplex networks with shared structure.
result Model recovers latent positions with sufficient separation.
Extends random dot product graph model to handle multiple graphs.
problem Modeling and analyzing multiple graphs with shared nodes.
method Jointly embed adjacency matrices into a latent space.
result Node representations converge to latent positions with Gaussian error.
Classification of sparsely labeled spatio-temporal data through semi-supervised adversarial learningstat.ML
In recent years, Generative Adversarial Networks (GAN) have emerged as a powerful method for learning the mapping from noisy latent spaces to realistic data samples in high-dimensional space. So far, the development and application of GANs have been predominantly focused on spatial data such as images. In this project,…
Exploration of Large Networks with Covariates via Fast and Universal Latent Space Model Fittingstat.ME
Latent space models are effective tools for statistical modeling and exploration of network data. These models can effectively model real world network characteristics such as degree heterogeneity, transitivity, homophily, etc. Due to their close connection to generalized linear models, it is also natural to incorporat…
Unified framework for estimating high-dimensional conditional factor models.
problem Estimating high-dimensional conditional latent factor models with practical limitations.
method Constrained nuclear norm regularization and cross-validation for parameter selection.
result Imposing homogeneity improves model predictability, with new method outperforming alternatives.
Suppose a given observation matrix can be decomposed as the sum of a low-rank matrix and a sparse matrix (outliers), and the goal is to recover these individual components from the observed sum. Such additive decompositions have applications in a variety of numerical problems including system identification, latent var…
Model infers latent variables in sparse coding models using Langevin dynamics.
problem Sampling posterior distribution in sparse coding models.
method Langevin dynamics for inference and simultaneous learning of parameters.
result Langevin dynamics efficiently sample from 'L0 sparse' posterior distribution.
In tensor completion tasks, the traditional low-rank tensor decomposition models suffer from the laborious model selection problem due to their high model sensitivity. In particular, for tensor ring (TR) decomposition, the number of model possibilities grows exponentially with the tensor order, which makes it rather ch…
ALCORE tensor decomposition reduces computational cost for sparse count data.
problem Efficiently decompose sparse count data matrices.
method Probabilistic Tucker decomposition with -norm constraint.
result ALCORE achieves similar results to full Tucker decomposition at a fraction of the cost.
We study the problem of inverting a deep generative model with ReLU activations. Inversion corresponds to finding a latent code vector that explains observed measurements as much as possible. In most prior works this is performed by attempting to solve a non-convex optimization problem involving the generator. In this …
New framework optimizes for 'value' rather than engagement.
problem Gap between engagement signals and desired notion of 'value'.
method Measurement theory framework, latent variable model, qualitative evaluation.
result Operationalizes and optimizes for a desired notion of 'value'.
This paper improves sampling from complex distributions using Langevin dynamics.
problem Pathological behaviors in normalizing flows for complex distributions.
method A Metropolis adjusted Langevin algorithm (MALA) to sample in the latent space.
result The method preserves tractability of the likelihood and works with any pre-trained NF network.
New defense method inspired by encryption improves visual classification accuracy.
problem Conventional defenses reduce accuracy and are defeated by obfuscated gradients.
method Block-wise pixel shuffling with secret key for training and test images.
result Achieves high accuracy (91.55%) on clean images and (89.66%) on adversarial examples.
ODBAE detects complex phenotypes in biological data.
problem Challenges in identifying complex phenotypes from high-dimensional biological data.
method ODBAE (Outlier Detection using Balanced Autoencoders) identifies influential and high leverage points in latent relationships among multiple physiological parameters.
result ODBAE reveals novel metabolism-related genes and uncovers coordinated abnormalities across metabolic indicators.
This study analyzes prediction risk for PCR method in latent factor regression models.
problem Prediction risk analysis in latent factor regression models.
method Adaptive PCR method with risk bounds established under factor regression model.
result Unified framework for analyzing various linear prediction methods under factor regression.
Adversarial examples are typically constructed by perturbing an existing data point within a small matrix norm, and current defense methods are focused on guarding against this type of attack. In this paper, we propose unrestricted adversarial examples, a new threat model where the attackers are not restricted to small…
Proves EM algorithm guarantees for hierarchical imitation learning.
problem Learning hierarchical policies from expert demonstrations.
method Converts hierarchical imitation learning to latent variable model inference, analyzes EM algorithm.
result Proves EM algorithm converges to true parameter under regular conditions.
Primal-Dual Wasserstein GANstat.ML
We introduce Primal-Dual Wasserstein GAN, a new learning algorithm for building latent variable models of the data distribution based on the primal and the dual formulations of the optimal transport (OT) problem. We utilize the primal formulation to learn a flexible inference mechanism and to create an optimal approxim…
VJE learns latent representations without contrastive learning, providing probabilistic semantics.
problem Learning latent representations without contrastive signals.
method VJE maximizes a symmetric conditional evidence lower bound (ELBO) on paired encoder embeddings, using a Student-t distribution on a polar representation.
result VJE outperforms standard non-contrastive baselines in ImageNet-1K, CIFAR-10/100, and STL-10.
Proposes a new method for joint sample and feature selection in multi-view data.
problem Cannot detect latent subsets of samples and remove outliers.
method Weighted Sparse Partial Least Squares (-wsPLS) method for joint sample and feature selection.
result Developed globally convergent algorithm and iterative algorithms for multi-view data fusion.
Low-rank tensor completion recovers missing entries based on different tensor decompositions. Due to its outstanding performance in exploiting some higher-order data structure, low rank tensor ring has been applied in tensor completion. To further deal with its sensitivity to sparse component as it does in tensor princ…
No GANs can learn disconnected manifolds precisely.
problem Learning disconnected manifolds is challenging due to unimodal latent distributions.
method Formalized a no free lunch theorem and derived a rejection sampling method.
result Upper bound on the precision of the targeted disconnected manifold distribution.
We propose a general framework for reduced-rank modeling of matrix-valued data. By applying a generalized nuclear norm penalty we can directly model low-dimensional latent variables associated with rows and columns. Our framework flexibly incorporates row and column features, smoothing kernels, and other sources of sid…
Abstract compares two norms in holomorphic quadratic differentials.
problem Comparing two norms in holomorphic quadratic differentials.
method Comparison between Avila-Gouëzel-Yoccoz norm and Teichmüller norm.
result Comparison of two norms in holomorphic quadratic differentials.
This paper explores the computational hardness of generating latent vectors for generative models.
problem Computational hardness of generating latent vectors for generative models.
method Established lower bounds for exact and approximate model inversion under strong exponential time hypothesis (SETH) and exponential time hypothesis (ETH).
result Lower bounds for computational complexity of exact and approximate model inversion.
We introduce a new family of matrix norms, the "local max" norms, generalizing existing methods such as the max norm, the trace norm (nuclear norm), and the weighted or smoothed weighted trace norms, which have been extensively used in the literature as regularizers for matrix reconstruction problems. We show that this…
We introduce twisted Alexander norms of a compact connected orientable 3-manifold with first Betti number bigger than one generalizing norms of McMullen and Turaev. We show that twisted Alexander norms give lower bounds on the Thurston norm of a 3-manifold. Using these we completely determine the Thurston norm of many …
We study a regularizer which is defined as a parameterized infimum of quadratics, and which we call the box-norm. We show that the k-support norm, a regularizer proposed by [Argyriou et al, 2012] for sparse vector prediction problems, belongs to this family, and the box-norm can be generated as a perturbation of the fo…