New concept of sparse regular variation for better understanding of extreme events.
problem Characterizing the dependence structure of extreme events in multivariate settings.
method Introducing sparse regular variation based on Euclidean projection onto the simplex.
result Sparse regular variation and regular variation are equivalent under mild assumptions.
Study compares L1 and VG sparsity priors in inverse problems.
problem Sparse regularization in inverse problems with incomplete or corrupted measurements.
method Compared L1 regularization with Variational Garrote (VG), a probabilistic method approximating L0 sparsity.
result VG often achieves lower minimum generalization error and improved stability in strongly underdetermined regimes.
Regularizes sparse Gaussian processes for better model performance.
problem Efficient learning of inducing inputs in latent variable models.
method Proposes a regularization approach to balance reconstruction and approximation performance.
result Improves both inference and prediction performance in latent variable models.
New method improves tensor completion by selectively preserving important elements.
problem Recovering corrupted high-dimensional tensor data with missing entries and noise.
method Tensor weighted correlated total variation (TWCTV) regularizer with ADMM algorithm.
result Superior performance in image completion, denoising, and background subtraction tasks.
Adaptive sparseness enhances robust regression using MCC and ARD.
problem Developing a robust regression method with adaptive sparseness.
method Integrating MCC with ARD in a Bayesian framework using variational Bayesian inference.
result MCC-ARD regression outperforms existing methods in prediction and feature selection.
In this paper, we consider the sparse regularization of manifold-valued data with respect to an interpolatory wavelet/multiscale transform. We propose and study variational models for this task and provide results on their well-posedness. We present algorithms for a numerical realization of these models in the manifold…
New method uses generative priors for compressive sensing with sparse solutions.
problem Fundamental linear inverse problem in compressive sensing.
method Sparse Bayesian learning with conditional Gaussianity.
result Ability to learn from few compressed and noisy samples without optimization.
So-called sparse estimators arise in the context of model fitting, when one a priori assumes that only a few (unknown) model parameters deviate from zero. Sparsity constraints can be useful when the estimation problem is under-determined, i.e. when number of model parameters is much higher than the number of data point…
New DRGP models improve prediction accuracy for sequential data.
problem Modeling sequential data for applications like autonomous driving.
method Introduces Deep recurrent Gaussian process (DRGP) models based on Sparse Spectrum Gaussian process (SSGP) and variational Sparse Spectrum Gaussian process (VSSGP).
result Improves prediction accuracy compared to current state of the art methods.
This paper studies least-square regression penalized with partly smooth convex regularizers. This class of functions is very large and versatile allowing to promote solutions conforming to some notion of low-complexity. Indeed, they force solutions of variational problems to belong to a low-dimensional manifold (the so…
Recently, a number of mostly ℓ1-norm regularized least squares type deterministic algorithms have been proposed to address the problem of \emph{sparse} adaptive signal estimation and system identification. From a Bayesian perspective, this task is equivalent to maximum a posteriori probability estimation under a …
We propose to align distributional data from the perspective of Wasserstein means. We raise the problem of regularizing Wasserstein means and propose several terms tailored to tackle different problems. Our formulation is based on the variational transportation to distribute a sparse discrete measure into the target do…
Regularized regression problems are ubiquitous in statistical modeling, signal processing, and machine learning. Sparse regression in particular has been instrumental in scientific model discovery, including compressed sensing applications, variable selection, and high-dimensional analysis. We propose a broad framework…
We develop a 2D travel time tomography method which regularizes the inversion by modeling groups of slowness pixels from discrete slowness maps, called patches, as sparse linear combinations of atoms from a dictionary. We propose to use dictionary learning during the inversion to adapt dictionaries to specific slowness…
Adaptive variational Bayes framework improves inference adaptively.
problem Lack of general and computationally tractable variational Bayes method for adaptive inference.
method Proposes a novel adaptive variational Bayes framework combining variational posteriors over individual models.
result Adaptive variational Bayes achieves optimal contraction rates adaptively under general conditions.
This work improves distribution recovery from sparse data using Random Forest implicit regularization.
problem Distribution recovery from limited statistics.
method Closed-form estimator for scaled beta distributions, using composite quantile and moment matching.
result Improved classification accuracy through closed-form distribution recovery and implicit regularization.
Sparse learning techniques have been routinely used for feature selection as the resulting model usually has a small number of non-zero entries. Safe screening, which eliminates the features that are guaranteed to have zero coefficients for a certain value of the regularization parameter, is a technique for improving t…
This work combines deep learning and sparse coding for CT image reconstruction.
problem Improving image quality in low-dose CT scans.
method Sparse signal representation using learned dictionaries, inspired by variational autoencoders and deep learning techniques.
result Regularization with learned dictionaries achieves competitive performance in CT reconstruction.
Paper introduces variational inference for Bayesian inverse problems with gamma hyperpriors.
problem Bayesian inverse problems with sparse solutions.
method Variational iterative alternating scheme for hierarchical models with gamma hyperpriors.
result Accurate reconstruction and meaningful uncertainty quantification.
A common strategy for sparse linear regression is to introduce regularization, which eliminates irrelevant features by letting the corresponding weights be zeros. However, regularization often shrinks the estimator for relevant features, which leads to incorrect feature selection. Motivated by the above-mentioned issue…
Novel regularization for Vision Transformers improves model generalization and sparsity.
problem Improving generalization and sparsity in Vision Transformers.
method Likelihood-guided variational Ising-based regularization.
result Improved generalization and sparsity in Vision Transformers.
Improves variational inference for sparse models using mixtures of exponential families.
problem Intractability of posterior distributions in Bayesian sparse models.
method Flexible mean field variational inference using mixtures of non-overlapping exponential families.
result Mixtures of exponential families with non-overlapping support form an exponential family, enabling analytical updates.
New method for disentangling latent factors with sparse dependencies.
problem Disentangling latent factors from observed variables and past factors.
method Mechanism sparsity regularization and sparse causal graphical model.
result Identifiability of latent factors up to a sparse causal graph.
Adaptive dropout and regularization are shown to be dual in linear networks.
problem Sparsifying deep neural networks.
method Examining dropout in the linear case, revealing a duality with regularization.
result Adaptive dropout methods lead to sparse solutions with effective penalties similar to classical sparse optimization penalties.
A new framework uses matrix flows to unify frequentist and Bayesian approaches for sparse GGMs.
problem Challenges in studying conditional independence among many variables with few observations.
method General framework for variational inference with matrix-variate Normalizing Flow in Gaussian Graphical Models.
result Unified benefits of frequentist and Bayesian frameworks for sparse GGMs.
New framework IDOL identifies latent causal processes with instantaneous relations from time series data.
problem Identifying latent causal processes with instantaneous relations from time series data.
method Sparse influence constraint and variational inference architecture with sparsity regularization.
result Our method can identify latent causal processes with instantaneous relations.
Study sparse function recovery from indirect noisy observations using ℓ1-regularization.
problem Recovering sparse functions from indirect, noisy observations.
method Proposes an ℓ1-regularized empirical risk minimizer and analyzes its statistical properties. result Established almost-sure consistency and derived high-probability convergence rates in prediction and ℓ1 norms. SeReNe prunes neurons with low sensitivity to reduce network size.
problem Large neural networks consume too many resources on resource-constrained devices.
method Exploits neural sensitivity as a regularizer to prune neurons with low sensitivity.
result Pruning neurons with low sensitivity achieves competitive compression ratios.
Nash integrates covariate-specific side info into sparse regression via neural networks.
problem Sparse linear regression struggles with covariates exhibiting structure or coming from heterogeneous sources.
method Neural Adaptive Shrinkage (Nash) framework that integrates side information into sparse regression via neural networks. Uses split variational empirical Bayes algorithm.
result Nash improves accuracy and adaptability over existing methods in real data experiments.
Deep Q-EP improves image modeling by stacking Q-EP layers.
problem Inhomogeneous subjects like images with edges are poorly modeled by standard Gaussian processes.
method Introducing shallow Q-EP as a latent variable model, stacking multiple layers, and applying sparse approximation and scalable variational strategy.
result Deep Q-EP outperforms state-of-the-art deep probabilistic models in image modeling.
Paper develops efficient variational inference for sparse deep learning with theoretical guarantees.
problem Sparse deep learning's challenge of huge storage consumption and sparse structure recovery.
method Bayesian treatment with spike-and-slab priors and continuous relaxation of Bernoulli distribution for computationally efficient variational inferences.
result Provides variational posterior contraction rate, justifying consistency of the proposed method.
Proposes ℓ0-CCA for sparse CCA with improved representation learning.
problem CCA models break with too many variables, and sparsity is beneficial.
method Sparse CCA with stochastic gates and ℓ0-regularization. result Improves representation learning by gating nuisance variables.
Paper optimizes Laplacian regularization for sparse network clustering.
problem Improving spectral clustering in sparse networks.
method Formally determines optimal Laplacian regularization.
result Proper regularization is closely tied to state-of-the-art techniques.
Deep learning models reconstruct volatility surfaces from noisy data under no-arbitrage constraints.
problem Reconstructing implied volatility surfaces from sparse and noisy option quotes.
method Compared multiple neural architectures including Transformers, U-Nets, and variational autoencoders.
result Transformer and U-Net architectures achieve strong reconstruction accuracy, especially under sparse observation regimes.
We study sparse principal components analysis in high dimensions, where p (the number of variables) can be much larger than n (the number of observations), and analyze the problem of estimating the subspace spanned by the principal eigenvectors of the population covariance matrix. We introduce two complementary not…
In this paper, a sparse Markov decision process (MDP) with novel causal sparse Tsallis entropy regularization is proposed.The proposed policy regularization induces a sparse and multi-modal optimal policy distribution of a sparse MDP. The full mathematical analysis of the proposed sparse MDP is provided.We first analyz…
New method learns sparse distributions by thresholding samples, improving performance and efficiency.
problem Sparse coding optimization in high-dimensional problems is computationally expensive and inefficient.
method Proposes a new variational sparse coding approach that learns sparse distributions by thresholding samples.
result Shows superior performance, statistical efficiency, and gradient estimation compared to other sparse distributions.
Non-convex regularizers usually improve the performance of sparse estimation in practice. To prove this fact, we study the conditions of sparse estimations for the sharp concave regularizers which are a general family of non-convex regularizers including many existing regularizers. For the global solutions of the regul…
Two new methods improve block-sparse signal recovery from noisy data.
problem Recovering block-sparse signals with unknown partitions.
method LogLOP-l2/l1 and AdaLOP-l2/l1 methods using log-sum penalty and MCP.
result Our methods outperform existing techniques in estimation accuracy.
We consider spectral clustering algorithms for community detection under a general bipartite stochastic block model (SBM). A modern spectral clustering algorithm consists of three steps: (1) regularization of an appropriate adjacency or Laplacian matrix (2) a form of spectral truncation and (3) a k-means type algorithm…
Sparse representations have proven their efficiency in solving a wide class of inverse problems encountered in signal and image processing. Conversely, enforcing the information to be spread uniformly over representation coefficients exhibits relevant properties in various applications such as digital communications. A…
Method converts sparse systems to dense ones for statistical mechanics problems.
problem Statistical mechanics on sparse graphs
method Extracts a Feedback Vertex Set, learns variational distribution, estimates free energy.
result More accurate and faster than existing methods for sparse systems.
Unified framework for sparse logistic regression with nonconvex regularization.
problem Sparse logistic regression with nonconvex regularization.
method Unified framework, line search criteria for nonconvex terms.
result Effective classification and feature selection at lower computational cost.
The paper analyzes uncertainty quantification in sparse Gaussian process regression with a Brownian motion prior.
problem Analyzing uncertainty in sparse Gaussian process regression with a Brownian motion prior.
method Theoretical guarantees and limitations for pointwise credible sets are derived for a rescaled Brownian motion prior with a sparse variational Gaussian process method.
result Theoretical characterization of asymptotic frequentist coverage for credible sets, distinguishing conservative and overconfident cases.
VASE uses Bayesian neural networks to improve exploration in sparse reward environments.
problem Exploration in environments with continuous control and sparse rewards.
method VASE uses a Bayesian neural network model of the environment dynamics and variational inference to alternately update the model's accuracy and policy.
result VASE outperforms other surprise-based exploration techniques in continuous control sparse reward environments.
IRKSN algorithm achieves sparse recovery with wider applicability conditions.
problem Sparse recovery challenges due to NP-hard nature and restrictive conditions.
method IRKSN algorithm based on k-support norm regularizer. result Achieves sparse recovery with explicit constants and standard linear rate.
Proposes a method to compress and sparsify neural networks using KD and VI.
problem Porting deep neural networks to embedded platforms with minimal accuracy loss.
method Combines knowledge distillation and variational inference to create a sparse student network.
result Significant memory footprint reduction and improved sparsity without accuracy loss.
Develops SGP-VAE for efficient sparse GP inference in multi-dimensional datasets.
problem Sparse GP approximations and missing data in multi-dimensional spatio-temporal datasets.
method Leverages partial inference networks for sparse GP approximations and amortized variational inference.
result Outperforms multi-output GPs and structured VAEs in various experiments.