New theory for partial disentanglement from sparse graphs.
problem Disentangling latent factors from sparse causal graphs.
method Generalization of disentanglement theory to any graph, using consistency equivalence.
result Partial disentanglement captures expected factor entanglement based on graph structure.
New method identifies causal variables from partially observed data.
problem Learning from unpaired observations with instance-dependent partial observability.
method Proposes two methods enforcing sparsity in the inferred representation.
result Establishes two identifiability results for linear and piecewise linear mixing functions.
New methods generalize nonlinear ICA beyond structural sparsity.
problem Identify true latent sources from nonlinear mixtures without structural sparsity assumptions.
method Propose identifiability results for undercomplete, partial sparsity, and flexible grouping structures.
result Prove identifiability in general settings of undercompleteness, partial sparsity, and flexible grouping structures.
The paper explores partial identifiability in nonnegative matrix factorization under specific conditions.
problem Identifying specific columns of the matrices in nonnegative matrix factorization.
method Mathematical rigor and geometric interpretation to analyze partial identifiability of columns in nonnegative matrix factorization.
result The partial uniqueness of a single column of C or S can be guaranteed under certain sparsity and algebraic conditions. Develops a regression model for partially observed dynamic tensor data.
problem Characterizing the relationship between dynamic tensor data and external covariates when data is only partially observed.
method Introduces low-rank, sparsity, and fusion structures on the regression coefficient tensor, and uses a loss function projected over observed entries. Developed an efficient non-convex alternating updating algorithm.
result Derived finite-sample error bounds for the estimator.
Paper introduces structured sparsity estimators for Generalized Linear Models.
problem Estimating structured sparsity in GLMs with debiased estimators.
method Extends Stucky and van de Geer's results to GLMs with structured sparsity.
result Proves oracle inequalities for structured sparsity estimators in GLMs.
Blind source separation (BSS) is a very popular technique to analyze multichannel data. In this context, the data are modeled as the linear combination of sources to be retrieved. For that purpose, standard BSS methods all rely on some discrimination principle, whether it is statistical independence or morphological di…
Automated PDE discovery from multiple noisy experiments.
problem Inherent variability in experiments makes single experiment inference unreliable.
method Randomised adaptive group Lasso sparsity estimator in deep learning framework.
result More generalizable PDEs found from multiple datasets.
Paper tackles causal inference with partially labeled data, introducing robust methods.
problem Challenges in causal inference due to partially labeled datasets and potential bias.
method Decaying missing-at-random framework and BRSS estimator for doubly robust causal inference.
result Established asymptotic normality of BRSS estimator under decaying labeling propensity scores.
This paper improves system identification by reducing sample complexity for high-dimensional linear dynamical systems.
problem High sample complexity for learning partially observed linear dynamical systems in high dimensions.
method Introduces an ℓ1-regularized estimation method that reduces sample complexity from linear to logarithmic with system dimension. result Markov parameters can be learned with logarithmic number of samples relative to system dimension, improving sample complexity.
Inverse problems and regularization theory is a central theme in contemporary signal processing, where the goal is to reconstruct an unknown signal from partial indirect, and possibly noisy, measurements of it. A now standard method for recovering the unknown signal is to solve a convex optimization problem that enforc…
This work introduces adversarial sparsity to measure robustness beyond adversarial accuracy.
problem Evaluating robustness to adversarial attacks beyond just accuracy.
method Adversarial sparsity, which quantifies the difficulty of finding perturbations.
result Sparsity provides valuable insights into neural networks and suggests improvements in robustness.
Cross-domain collaborative filtering (CF) aims to alleviate data sparsity in single-domain CF by leveraging knowledge transferred from related domains. Many traditional methods focus on enriching compared neighborhood relations in CF directly to address the sparsity problem. In this paper, we propose superhighway const…
New method for disentangling latent factors with sparse dependencies.
problem Disentangling latent factors from observed variables and past factors.
method Mechanism sparsity regularization and sparse causal graphical model.
result Identifiability of latent factors up to a sparse causal graph.
New algorithm extracts shared latent space for cortico-muscular interactions.
problem Challenges of high dimensionality and limited sample sizes in multivariate cortico-muscular analysis.
method Structured and sparse partial least squares coherence (ssPLSC) algorithm.
result ssPLSC achieves competitive or better performance in scenarios with limited sample sizes and high noise levels.
The restricted isometry property (RIP) is a universal tool for data recovery. We explore the implication of the RIP in the framework of generalized sparsity and group measurements introduced in the Part I paper. It turns out that for a given measurement instrument the number of measurements for RIP can be improved by o…
Dual-sPLS improves feature selection and prediction in high-dimensional data.
problem Relating variables to a response in high-dimensional chemometric problems.
method Generalizes PLS1 algorithm with dual norm penalizations and a shrinking ratio parameter.
result Favorably compares to similar regression methods on simulated and real chemical data.
Although various distributed machine learning schemes have been proposed recently for pure linear models and fully nonparametric models, little attention has been paid on distributed optimization for semi-paramemetric models with multiple-level structures (e.g. sparsity, linearity and nonlinearity). To address these is…
We address the problem of partial index tracking, replicating a benchmark index using a small number of assets. Accurate tracking with a sparse portfolio is extensively studied as a classic finance problem. However in practice, a tracking portfolio must also be diverse in order to minimise risk -- a requirement which h…
Distributions over rankings are used to model data in various settings such as preference analysis and political elections. The factorial size of the space of rankings, however, typically forces one to make structural assumptions, such as smoothness, sparsity, or probabilistic independence about these underlying distri…
New method finds compact strong lottery tickets in partially frozen networks.
problem Finding compact strong lottery tickets in randomly pruned networks.
method Freezing a random subset of initial weights in a partially frozen network.
result Better accuracy and compression achieved with frozen networks.
New algorithm detects tensor dependence structure alterations efficiently.
problem Detecting alterations in tensor dependence structures.
method Tensor-normal distributions, decorrelation, centralization, SERA (Sparsity-Exploited Reranking Algorithm).
result The proposed SERA algorithm controls false discovery rates effectively.
Boosting as gradient descent algorithms is one popular method in machine learning. In this paper a novel Boosting-type algorithm is proposed based on restricted gradient descent with structural sparsity control whose underlying dynamics are governed by differential inclusions. In particular, we present an iterative reg…
Topic models, and more specifically the class of Latent Dirichlet Allocation (LDA), are widely used for probabilistic modeling of text. MCMC sampling from the posterior distribution is typically performed using a collapsed Gibbs sampler. We propose a parallel sparse partially collapsed Gibbs sampler and compare its spe…
New adaptive learning rate improves FTRL's adaptivity to sparsity, game-dependency, and best-of-both-worlds.
problem Improving adaptivity in sequential decision-making problems.
method Developed a stability-penalty-adaptive (SPA) learning rate for FTRL.
result First BOBW algorithm with sparsity-dependent bound.
We consider a class of sparse learning problems in high dimensional feature space regularized by a structured sparsity-inducing norm which incorporates prior knowledge of the group structure of the features. Such problems often pose a considerable challenge to optimization algorithms due to the non-smoothness and non-s…
DNNs with L2 regularization reveal feature learning dynamics and sparsity.
problem Understanding feature learning in DNNs with L2 regularization. method Reformulating loss in terms of layerwise activations and covariances.
result Proving sparsity of local minima in L2-regularized DNNs. New method for MTL with varying sparsity patterns across tasks.
problem Jointly training multiple linear models with differing sparsity patterns.
method Mixed-integer programming formulation and scalable algorithms.
result Our methods leverage shared support information to improve variable selection.
Proposes a method for coarse graph alignment using sparse partial least squares.
problem Aligning graphs with community structures when there's no natural one-to-one mapping.
method Sparse partial least squares method incorporating observed graph structures and imposing sparsity.
result Demonstrates effectiveness in simulations.
CONFETTI improves interpretability of deep learning models for MTS by providing counterfactual explanations.
problem Lack of transparency in deep learning models for multivariate time series classification.
method CONFETTI is a novel multi-objective counterfactual explanation method that balances prediction confidence, proximity, and sparsity.
result CONFETTI outperforms state-of-the-art methods in various metrics, improving interpretability and decision support.
We investigate structured sparsity methods for variable selection in regression problems where the target depends nonlinearly on the inputs. We focus on general nonlinear functions not limiting a priori the function space to additive models. We propose two new regularizers based on partial derivatives as nonlinear equi…
Proposes SPCA to incorporate structural constraints in model identification.
problem Model identification with partial structural knowledge.
method Structural Principal Component Analysis (SPCA) that leverages structural information.
result Demonstrates improved model estimates using synthetic and industrial data.
MUSIC learns coupled systems with sparse data and incomplete physics.
problem Learning coupled systems with incomplete physical constraints and missing data.
method Sparsity induced multitask neural network framework integrating partial physical constraints with data-driven learning.
result MUSIC accurately learns solutions to complex coupled systems under data-scarce and noisy conditions.
Bayesian method learns PDEs from noisy data.
problem Discovering PDEs from noisy data.
method Combining variational Bayes and sparse linear regression.
result Proposes a new method to discover PDEs accurately.
In this work we are interested in the problems of supervised learning and variable selection when the input-output dependence is described by a nonlinear function depending on a few variables. Our goal is to consider a sparse nonparametric model, hence avoiding linear or additive models. The key idea is to measure the …
Paper analyzes adaptive Lasso for high-dimensional diffusion processes, improving support recovery and bias.
problem Support recovery for high-dimensional diffusion processes under sparsity constraints.
method Adaptive Lasso estimator for d-dimensional ergodic diffusion process, focusing on linear models.
result Adaptive Lasso achieves support recovery and asymptotic normality for drift parameter under certain conditions.
HCL learns shared and modality-specific latent representations for multimodal data.
problem Binary shared-private decomposition inadequately represents shared information across subsets of modalities.
method Hierarchical Contrastive Learning framework combining latent-variable formulation, structural sparsity, and contrastive objective.
result HCL accurately recovers hierarchical structure and improves predictive performance on multimodal data.
This paper develops a fast algorithm for solving nonlinear PDEs using sparse Cholesky factorization.
problem Efficiently solving nonlinear PDEs with Gaussian processes and kernel methods.
method Sparse Cholesky factorization for near-linear complexity.
result Near-linear complexity algorithm for working with kernel matrices of nonlinear PDEs.
We consider the problem of providing nonparametric confidence guarantees for undirected graphs under weak assumptions. In particular, we do not assume sparsity, incoherence or Normality. We allow the dimension D to increase with the sample size n. First, we prove lower bounds that show that if we want accurate infe…
The paper addresses nonconvex penalized LAD estimation in partial linear models using DNNs.
problem Challenges in nonconvex penalized LAD estimation with DNNs in partial linear models.
method Parameterizes nonparametric term with DNNs, formulates penalized LAD problem, introduces proximal subgradient method.
result Establishes consistency, convergence rate, and asymptotic normality of the estimator.
Neighborhood regression has been a successful approach in graphical and structural equation modeling, with applications to learning undirected and directed graphical models. We extend these ideas by defining and studying an algebraic structure called the neighborhood lattice based on a generalized notion of neighborhoo…
We propose tensor-network compressed sensing (TNCS) by combining the ideas of compressed sensing, tensor network (TN), and machine learning, which permits novel and efficient quantum communications of realistic data. The strategy is to use the unsupervised TN machine learning algorithm to obtain the entangled state $|Ψ…
Graphical estimation of count time series dependencies.
problem Estimating dependencies between multivariate count time series.
method Parameter-driven generalized linear model with l1-type regularization and MCEM algorithm.
result Characterization of disease spread interdependence and sources/sinks in Greater Mumbai.
New layer sparsity concept improves neural networks.
problem Improving neural network efficiency and interpretability.
method Formulated layer sparsity, introduced regularization and refitting schemes.
result Generated more compact and accurate neural networks.
Estimates complex dependency structures in multi-omics data.
problem Graphical model estimation from multi-omics data with scalability and consistency.
method Pseudolikelihood-based graphical model framework with ℓ1-penalized empirical risk. result Estimates partial correlation network from dual-omic liver cancer data.
In this paper we revisit the risk bounds of the lasso estimator in the context of transductive and semi-supervised learning. In other terms, the setting under consideration is that of regression with random design under partial labeling. The main goal is to obtain user-friendly bounds on the off-sample prediction risk.…
Our work is focused on the joint sparsity recovery problem where the common sparsity pattern is corrupted by Poisson noise. We formulate the confidence-constrained optimization problem in both least squares (LS) and maximum likelihood (ML) frameworks and study the conditions for perfect reconstruction of the original r…
STR reparameterizes DNN weights with soft thresholds for better sparsity and accuracy.
problem Improving sparsity in DNNs for better accuracy and lower inference cost.
method Soft Threshold Reparameterization (STR) using the soft-threshold operator on DNN weights.
result STR achieves state-of-the-art accuracy and reduces FLOPs by up to 50%.