CDSSD detects sparse changes in partially observable data streams.
problem Online change detection of sparse changes in partially observable high-dimensional data streams.
method Smooth-sparse decomposition, spike-slab variational Bayesian inference, adaptive sampling via Thompson sampling.
result CDSSD effectively detects sparse changes in partially observable data streams.
In compressed sensing, we wish to reconstruct a sparse signal x from observed data y. In sparse coding, on the other hand, we wish to find a representation of an observed signal y as a sparse linear combination, with coefficients x, of elements from an overcomplete dictionary. While many algorithms are competit…
Latent-EnSF improves data assimilation for high-dimensional systems with sparse observations.
problem Challenges in high-dimensional, nonlinear Bayesian filtering with sparse observations.
method A novel data assimilation method using latent representations and a coupled VAE for efficient state encoding and reconstruction.
result Latent-EnSF outperforms traditional methods in accuracy, convergence, and efficiency for complex systems.
The paper tackles tensor factorization and completion from noisy data.
problem Sparse nonnegative tensor factorization and completion from partial and noisy observations.
method Minimizes the sum of maximum likelihood estimation and tensor ℓ0 norm with nonnegativity constraints. result Error bounds and minimax lower bounds are established for the proposed model.
LD-EnSF speeds up data assimilation with sparse observations.
problem Efficiently assimilate sparse and noisy data into complex dynamical systems.
method LD-EnSF uses latent dynamics networks and history-aware LSTM encoders to process sparse observations without full-space simulations.
result Achieves significant speedups over existing methods while maintaining high accuracy.
SDIFT generates full-field dynamics from sparse, irregular data.
problem Modeling and reconstructing physical dynamics from sparse, off-grid observations.
method SDIFT uses a functional Tucker model and sequential diffusion for generating full-field evolution from irregular sparse observations.
result Significant improvements in reconstruction accuracy and computational efficiency compared to state-of-the-art approaches.
Dynamic Sparse Training finds efficient sparse networks from scratch.
problem Finding efficient sparse neural networks.
method Jointly optimizes network parameters and sparsity with trainable thresholds.
result Achieves state-of-the-art performance with minimal performance loss.
FLASH-MAX predicts electromagnetic fields from sparse data in seconds.
problem Predicting homogeneous electromagnetic fields from sparse pointwise observations.
method Exact-by-construction neural network architecture that satisfies Maxwell's equations symbolically.
result FLASH-MAX achieves sub-1% relative validation error from 1K sparse observations in seconds.
Partially observable Markov decision processes (POMDPs) with continuous state and observation spaces have powerful flexibility for representing real-world decision and control problems but are notoriously difficult to solve. Recent online sampling-based algorithms that use observation likelihood weighting have shown un…
This paper examines a general class of noisy matrix completion tasks where the goal is to estimate a matrix from observations obtained at a subset of its entries, each of which is subject to random noise or corruption. Our specific focus is on settings where the matrix to be estimated is well-approximated by a product …
Sparse VAE learns latent factors from high-dimensional data.
problem Unsupervised representation learning on high-dimensional data.
method Sparse VAE model that learns latent factors summarizing data associations.
result Sparse VAE can recover true model parameters with infinite data.
Choice models, which capture popular preferences over objects of interest, play a key role in making decisions whose eventual outcome is impacted by human choice behavior. In most scenarios, the choice model, which can effectively be viewed as a distribution over permutations, must be learned from observed data. The ob…
The paper analyzes sparse PCA for incomplete data and proves support recovery conditions.
problem Support recovery in sparse PCA with non-random missing data.
method Semidefinite relaxation of the ℓ1-regularized PCA problem. result Support of the sparse leading eigenvector can be recovered with high probability.
Networks are a unifying framework for modeling complex systems and network inference problems are frequently encountered in many fields. Here, I develop and apply a generative approach to network inference (RCweb) for the case when the network is sparse and the latent (not observed) variables affect the observed ones. …
Study on estimating sparse transition matrix of partially-observed VAR with noisy and sparse data.
problem Estimating sparse transition matrix of partially-observed VAR with noisy and sparse data.
method Yule-Walker equation, Dantzig selector, minimax lower bound.
result Near-optimality of the proposed estimator with convergence rate analysis.
Algorithm recovers sparse PCA support from incomplete data.
problem Sparse PCA with incomplete and noisy data.
method Semidefinite program (SDP) relaxation of non-convex l1-regularized PCA. result SDP enables exact recovery of true support of sparse leading eigenvector.
Multi-channel sparse blind deconvolution, or convolutional sparse coding, refers to the problem of learning an unknown filter by observing its circulant convolutions with multiple input signals that are sparse. This problem finds numerous applications in signal processing, computer vision, and inverse problems. However…
Paper develops a new algorithm for sparse signal recovery.
problem Sparse signal recovery from noisy observations.
method Iterative Stochastic Optimization using Stochastic Mirror Descent.
result Linear convergence during preliminary phase of the routine.
Study finds exact limits for sparse regression with fewer observations than usual.
problem Understanding sparse linear regression with sublinear sparsity.
method Adaptive interpolation method and modified AMP algorithm.
result Exact asymptotic expressions for mutual information and MMSE in sublinear sparsity.
Proposes a method for tensor completion with sparse factors and missing data.
problem Recovering nonnegative data from noisy observations with missing values.
method Sparse nonnegative Tucker decomposition with ℓ0 norm for sparsity, maximum likelihood estimation, and error bounds. result The method outperforms existing tensor-based or matrix-based methods in nonnegative tensor data completion.
Regularization of the classical Laplacian matrices was empirically shown to improve spectral clustering in sparse networks. It was observed that small regularizations are preferable, but this point was left as a heuristic argument. In this paper we formally determine a proper regularization which is intimately related …
Large SGD step sizes lead to sparse feature learning in neural networks.
problem Sparse feature learning in neural networks with large step sizes.
method Empirical observations and theoretical analysis of SGD dynamics.
result Large step sizes induce implicit regularization leading to sparse predictors.
MAESTRO improves multimodal learning for dynamic time series with adaptive attention and robustness.
problem Challenges in multimodal learning, especially in healthcare and daily living.
method Dynamic intra- and cross-modal interactions, symbolic tokenization, adaptive attention budgeting, sparse cross-modal attention, MoE mechanism.
result Average relative improvements of 4% and 8% over existing multimodal and multivariate approaches, respectively, under complete observations.
This paper introduces an elasticity reconstruction method based on local displacement observations of elastic bodies. Sparse reconstruction theory is applied to formulate the underdetermined inverse problems of elasticity reconstruction including unobserved areas. An online local clustering scheme called a superelement…
FM4PDE learns PDE solutions from sparse data.
problem Reconstructing PDE solutions from limited observations.
method Flow-matching generative framework that learns PDE coefficients and solutions.
result Error guarantees for guided procedures, including deterministic and stochastic samplers.
Sparse point observations can provide useful local constraints, but their benefit for radar-like fields depends on the training loss, uncertainty representation, and how observation support is encoded in the model.
problem Improving dense radar-field forecasts with sparse point observations
method A multimodal graph neural network nowcasting system over the Nordic radar domain
result Each source improves a different part of the forecast problem
Since learning is typically very slow in Boltzmann machines, there is a need to restrict connections within hidden layers. However, the resulting states of hidden units exhibit statistical dependencies. Based on this observation, we propose using l1/l2 regularization upon the activation possibilities of hidden unit…
We consider the classical sparse regression problem of recovering a sparse signal x0 given a measurement vector y=Φx0+w. We propose a tree search algorithm driven by the deep neural network for sparse regression (TSN). TSN improves the signal reconstruction performance of the deep neural network designed for sp…
Fitting high-dimensional data involves a delicate tradeoff between faithful representation and the use of sparse models. Too often, sparsity assumptions on the fitted model are too restrictive to provide a faithful representation of the observed data. In this paper, we present a novel framework incorporating sparsity i…
This paper examines fundamental error characteristics for a general class of matrix completion problems, where the matrix of interest is a product of two a priori unknown matrices, one of which is sparse, and the observations are noisy. Our main contributions come in the form of minimax lower bounds for the expected pe…
Robust testing of sparse signals in corrupted data.
problem Testing the norm of high-dimensional sparse signals in the presence of arbitrary corruption.
method Two observation models: i.i.d. samples from N(θ,Id) and sparse linear regression model. result The robust testing requires significantly more samples than non-robust testing.
New estimator learns symmetric dynamics from few observations.
problem Learning parameters of stochastic linear dynamics from limited data.
method Method of moments estimator using T=O(logN) observations. result Achieves small maximum element-wise error on symmetric matrices.
The CUR decomposition provides an approximation of a matrix X that has low reconstruction error and that is sparse in the sense that the resulting approximation lies in the span of only a few columns of X. In this regard, it appears to be similar to many sparse PCA methods. However, CUR takes a randomized algorithm…
Proposes a nonparametric tensor factorization for sparse data.
problem Handling sparse tensor data with structural and interpretability benefits.
method Hierarchical Gamma processes and Poisson random measures for tensor-valued process, Dirichlet processes for sampling entry indices, Gaussian processes for values.
result Demonstrates superior performance on benchmark datasets.
CNNs predict spatial fields from sparse data.
problem Predicting complete spatial fields from limited observations.
method Convolutional Neural Networks (CNNs) trained on a single partially observed field.
result CNNs can flexibly capture local spatial patterns without explicit covariance modeling.
We derive fundamental sample complexity bounds for recovering sparse and structured signals for linear and nonlinear observation models including sparse regression, group testing, multivariate regression and problems with missing features. In general, sparse signal processing problems can be characterized in terms of t…
We consider the scenario where one observes an outcome variable and sets of features from multiple assays, all measured on the same set of samples. One approach that has been proposed for dealing with this type of data is ``sparse multiple canonical correlation analysis'' (sparse mCCA). All of the current sparse mCCA t…
Study on estimating causal effects with limited data and multiple environments.
problem Estimating causal effects under hidden confounding with unpaired data and sparse effects.
method Instrumental variable (IV) regression with cross-fold sample splitting and ℓ1-regularized estimation. result Proposed GMM-type estimator is consistent as the number of environments grows.
Deep learning predicts dynamics from sparse data.
problem Predicting spatiotemporal dynamics from sparse data.
method Spatially dimension-independent deep learning framework.
result Predicts dynamics from sparse data sites.
New algorithm for learning RBMs with sparse latent variables.
problem Learning RBMs with sparse latent variables efficiently.
method Algorithm with time complexity O(n^(2^s+1)) for sparse RBMs.
result Improves learning time for RBMs with sparse latent variables.
Combining LETKF and RC improves chaotic system prediction from noisy, sparse data.
problem Improving chaotic system prediction from imperfect observations and models.
method Combining LETKF and RC to predict spatio-temporal chaotic systems from noisy and sparsely distributed observations.
result The proposed method using LETKF and RC outperforms LETKF in predicting chaotic systems from noisy and sparse observations.
We are motivated by problems that arise in a number of applications such as Online Marketing and Explosives detection, where the observations are usually modeled using Poisson statistics. We model each observation as a Poisson random variable whose mean is a sparse linear superposition of known patterns. Unlike many co…
Paper proposes distributed sparse multicategory discriminant analysis for classification.
problem Sparse multicategory classification with distributed data.
method Convex formulation, distributed setting, invariant discriminant subspace recovery.
result Distributed sparse multicategory linear discriminant analysis performs as good as centralized version after a few rounds of communications.
Novel method for SDE calibration from sparse data using neural flows.
problem Calibrating SDEs from sparse, noisy observations.
method Characterization of posterior SDE using neural networks trained to solve a PDE with multiplicative updates.
result Significant improvement in scalability and accuracy compared to classical methods.
In this paper, we address the challenging problem of selecting tuning parameters for high-dimensional sparse regression. We propose a simple and computationally efficient method, called path thresholding (PaTh), that transforms any tuning parameter-dependent sparse regression algorithm into an asymptotically tuning-fre…
Sparse deep learning improves prediction uncertainty for time series data.
problem Uncertainty quantification for dependent data like time series.
method Sparse recurrent neural networks (RNNs) for time series data.
result Sparse deep learning can consistently estimate and predict time series data with correct uncertainty quantification.
DCRL learns causal relationships from mixed-type discrete data.
problem Challenges in learning causal relationships from discrete, mixed-type data.
method Generative framework modeling directed acyclic graph and sparse bipartite graph, flexible measurement models for different types of data.
result Consistent recovery of latent causal structure from observed data distribution.
New method infers graph from dependent matrix data.
problem Inferring graph from dependent matrix data.
method Sparse-group lasso-based frequency-domain formulation with ADMM approach.
result Local convergence of inverse PSD estimators to true value.