In this paper, we consider the block-sparse signals recovery problem in the context of multiple measurement vectors (MMV) with common row sparsity patterns. We develop a new method for recovery of common row sparsity MMV signals, where a pattern-coupled hierarchical Gaussian prior model is introduced to characterize bo…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “sparse patterns”
New PCA method handles multiple datasets and detects sparse patterns robustly.
problem Handling multi-source data with sparse and outlier-robust PCA.
method Developed a regularization problem with a penalty for structured sparsity and outlier resistance.
result The method detects global and local patterns across multiple data sources robustly.
We consider the problem of recovering block-sparse signals whose structures are unknown \emph{a priori}. Block-sparse signals with nonzero coefficients occurring in clusters arise naturally in many practical scenarios. However, the knowledge of the block structure is usually unavailable in practice. In this paper, we d…
Transformers learn to integrate information from past positions incrementally, specializing heads in distinct patterns.
problem How transformers learn to integrate information from multiple past positions with varying statistical significance.
method High-order Markov chain task, incremental learning, sparse attention patterns, simplified differential equations, stage-wise convergence, early stopping as regularizer.
result Transformers learn to specialize heads in distinct patterns, shifting from competitive to cooperative learning dynamics.
Sparse Transformers can approximate dense Transformers with only O(n) connections.
problem Can sparse Transformers approximate arbitrary sequence-to-sequence functions?
method Proposed sufficient conditions for universal approximation and proved that sparse Transformers with O(n) connections can approximate dense models.
result Sparse Transformers with O(n) connections can approximate the same function class as dense models with n^2 connections.
A new method warps inputs to learn nonstationary kernels efficiently.
problem Learning nonstationary patterns in data with varying smoothness.
method Sparse spectrum Gaussian processes with input warping as conditional Gaussian measures.
result Efficient learning of nonstationary patterns with fewer parameters.
Proposes RBGP framework for efficient block sparse neural networks.
problem Efficiently exploit structured sparsity patterns for sparse neural networks on GPU.
method Uses Ramanujan Bipartite Graph Product to generate structured multi-level block sparse neural networks.
result Achieves 5-9x and 2-5x runtime gains over unstructured and block sparsity patterns respectively, while maintaining accuracy.
This work improves dictionary learning speed without sacrificing accuracy.
problem Prohibitive computational cost of standard dictionary learning methods.
method Approximate dictionary learning using unrolling and gradient descent.
result Unrolling outperforms standard methods in support estimation and early iterations.
Sparse mapping has been a key methodology in many high-dimensional scientific problems. When multiple tasks share the set of relevant features, learning them jointly in a group drastically improves the quality of relevant feature selection. However, in practice this technique is used limitedly since such grouping infor…
Power quality (PQ) analysis describes the non-pure electric signals that are usually present in electric power systems. The automatic recognition of PQ disturbances can be seen as a pattern recognition problem, in which different types of waveform distortion are differentiated based on their features. Similar to other …
A new distributed algorithm for fitting sparse additive models with feature division and decorrelation.
problem Fitting high-dimensional sparse additive models efficiently and accurately.
method Divide, decorrelate, and conquer approach.
result Effective and efficient recovery of sparsity patterns and statistical inference for each component.
Sparse versions of principal component analysis (PCA) have imposed themselves as simple, yet powerful ways of selecting relevant features of high-dimensional data in an unsupervised manner. However, when several sparse principal components are computed, the interpretation of the selected variables is difficult since ea…
In sparse signal representation, the choice of a dictionary often involves a tradeoff between two desirable properties -- the ability to adapt to specific signal data and a fast implementation of the dictionary. To sparsely represent signals residing on weighted graphs, an additional design challenge is to incorporate …
A novel nonstationary permanental process relaxes kernel constraints and captures complex data patterns.
problem Limitations of existing permanental processes in terms of kernel types and stationarity.
method Sparse spectral representation of nonstationary kernels and hierarchical stacking of spectral feature mappings.
result Enhanced model expressiveness and reduced computational complexity.
BN^2MF identifies unknown exposure patterns in environmental mixtures.
problem Identifying unknown exposure patterns in environmental mixtures.
method Bayesian non-parametric non-negative matrix factorization (BN^2MF) with non-negative continuous priors and a non-parametric sparse prior.
result Estimates patterns of chemical exposures without specifying the number of patterns.
We present an extension of sparse PCA, or sparse dictionary learning, where the sparsity patterns of all dictionary elements are structured and constrained to belong to a prespecified set of shapes. This \emph{structured sparse PCA} is based on a structured regularization recently introduced by [1]. While classical spa…
New neural operators learn structured patterns efficiently.
problem Learning and representing complex, structured patterns in data.
method Sparse autoencoder neural operators (SAE-NOs) parameterize concepts as functions, enabling efficient and structured representation.
result SAE-FNOs learn localized patterns and generalize across different scales and discretizations.
Random Intersection Chainsstat.ML
Random Intersection Chains selects important interactions from categorical features.
problem Heavy computational burden in considering all interactions for categorical features.
method Randomly generates chains of intersections, estimates and selects frequent patterns.
result Selected patterns are the most frequent in the data set.
Proposes a new method for estimating sparse precision matrices in GMRF-MM models.
problem Difficulty in learning GMMs with large parameters and limited data.
method Restricts GMM to GMRF-MM, proposes efficient optimization for sparse precision matrices, and debiases the estimates.
result Debiasing approach outperforms GLASSO in single-GMRF and GMRF-MM cases.
Sparse coding approximates the data sample as a sparse linear combination of some basic codewords and uses the sparse codes as new presentations. In this paper, we investigate learning discriminative sparse codes by sparse coding in a semi-supervised manner, where only a few training samples are labeled. By using the m…
The study of healthy brain development helps to better understand the brain transformation and brain connectivity patterns which happen during childhood to adulthood. This study presents a sparse machine learning solution across whole-brain functional connectivity (FC) measures of three sets of data, derived from resti…
Dockless bike sharing systems need effective bike flow prediction models.
problem Imbalanced and dynamic use of bikes leads to mandatory rebalancing operations.
method Divide urban area into regions, model spatio-temporal bike flows, extract traffic patterns, and predict bike flows.
result Interpretable bike flow prediction model provides valuable insights into bike flow analysis.
Parallel Learning by Multitasking Neural Networkscond-mat.dis-nn
New neural network can learn multiple patterns at once.
problem Learning multiple patterns simultaneously in neural networks.
method Multitasking Hebbian Network (MHN) for parallel learning of sparse data-sets.
result MHN can handle multiple patterns hierarchically or in parallel, depending on dilution.
An important problem in geostatistics is to build models of the subsurface of the Earth given physical measurements at sparse spatial locations. Typically, this is done using spatial interpolation methods or by reproducing patterns from a reference image. However, these algorithms fail to produce realistic patterns and…
As the availability and importance of temporal interaction data--such as email communication--increases, it becomes increasingly important to understand the underlying structure that underpins these interactions. Often these interactions form a multigraph, where we might have multiple interactions between two entities.…
Standard compressive sensing results state that to exactly recover an s sparse signal in R^p, one requires O(s. log(p)) measurements. While this bound is extremely useful in practice, often real world signals are not only sparse, but also exhibit structure in the sparsity pattern. We focus on group-structured patterns …
A new iterative low complexity algorithm has been presented for computing the Walsh-Hadamard transform (WHT) of an dimensional signal with a -sparse WHT, where is a power of two and , scales sub-linearly in for some . Assuming a random support model for the non-zero transform domain…
We propose a novel denoising framework for task functional Magnetic Resonance Imaging (tfMRI) data to delineate the high-resolution spatial pattern of the brain functional connectivity via dictionary learning and sparse coding (DLSC). In order to address the limitations of the unsupervised DLSC-based fMRI studies, we u…
A new matrix factorization method for high-dimensional data.
problem Exploiting sparse structures in complex data for better interpretability.
method Bayesian shrinkage priors and flexible sparse patterns modeled through row and column dependencies.
result Demonstrated practical advantages through simulation and soccer heatmap analysis.
The study of hierarchy in networks of the human brain has been of significant interest among the researchers as numerous studies have pointed out towards a functional hierarchical organization of the human brain. This paper provides a novel method for the extraction of hierarchical connectivity components in the human …
EPNE models evolving network patterns for better predictions.
problem Capturing evolving patterns in dynamic networks.
method EPNE models temporal network evolution using causal convolutions and a temporal objective function.
result EPNE outperforms other methods in various prediction tasks.
Sparse Functional Singular Value Decomposition for Biclustering and Triclustering Longitudinal Datastat.ML
Unified method for discovering biclusters and triclusters in longitudinal data.
problem High-dimensional, sparsely sampled, irregularly observed longitudinal data.
method Tri-SfSVD, a unified sparse functional Singular Value Decomposition framework.
result Identified localized structures at the subject, subject-feature, and subject-feature-time levels.
Comprehending complex systems by simplifying and highlighting important dynamical patterns requires modeling and mapping higher-order network flows. However, complex systems come in many forms and demand a range of representations, including memory and multilayer networks, which in turn call for versatile community-det…
We develop a class of rules spanning the range between quadratic discriminant analysis and naive Bayes, through a path of sparse graphical models. A group lasso penalty is used to introduce shrinkage and encourage a similar pattern of sparsity across precision matrices. It gives sparse estimates of interactions and pro…
Moirai-MoE improves time series forecasting by automatically specializing tokens without human-defined frequency.
problem Unified training on time series data remains challenging due to heterogeneity and non-stationarity.
method Uses sparse mixture of experts (MoE) within Transformers to automatically specialize tokens for diverse time series patterns.
result Moirai-MoE outperforms existing foundation models in both in-distribution and zero-shot scenarios.
Method predicts multistable system states from sparse measurements.
problem Predicting multistable system states from limited data.
method Semi-supervised classification with SPML optimization.
result 95% accuracy in predicting reaction-diffusion equation states.
HARFE approximates sparse additive functions using random features and ridge regression.
problem Approximating high-dimensional sparse additive functions.
method Hard-ridge random feature expansion with sparse ridge regression and hard-thresholding pursuit.
result HARFE method converges with a given error bound and achieves lower error than other algorithms.
Face recognition is the important field in machine learning and pattern recognition research area. It has a lot of applications in military, finance, public security, to name a few. In this paper, the combination of the tensor sparse PCA with the nearest-neighbor method (and with the kernel ridge regression method) wil…
We propose the Relational Tucker3 (RT) decomposition for multi-relational link prediction in knowledge graphs. We show that many existing knowledge graph embedding models are special cases of the RT decomposition with certain predefined sparsity patterns in its components. In contrast to these prior models, RT decouple…
Sparse Stochastic Zeroth-Order Optimization with an Application to Bandit Structured Predictionstat.ML
Stochastic zeroth-order (SZO), or gradient-free, optimization allows to optimize arbitrary functions by relying only on function evaluations under parameter perturbations, however, the iteration complexity of SZO methods suffers a factor proportional to the dimensionality of the perturbed function. We show that in scen…
Frequency-specific patterns of neural activity are traditionally interpreted as sustained rhythmic oscillations, and related to cognitive mechanisms such as attention, high level visual processing or motor control. While alpha waves (8-12 Hz) are known to closely resemble short sinusoids, and thus are revealed by Fouri…
Sparse modeling is a powerful framework for data analysis and processing. Traditionally, encoding in this framework is performed by solving an L1-regularized linear regression problem, commonly referred to as Lasso or Basis Pursuit. In this work we combine the sparsity-inducing property of the Lasso model at the indivi…
Principal component analysis (PCA) is an exploratory tool widely used in data analysis to uncover dominant patterns of variability within a population. Despite its ability to represent a data set in a low-dimensional space, the interpretability of PCA remains limited. However, in neuroimaging, it is essential to uncove…
Paper studies binary random projections with controllable sparsity patterns for computational and accuracy advantages.
problem Improving computational efficiency and accuracy in random projections.
method Proposes two sparse binary projection models with controllable sparsity patterns.
result Significant computational advantages and improved accuracies in empirical evaluations.
Identifying context-specific entity networks from aggregated data is an important task, arising often in bioinformatics and neuroimaging. Computationally, this task can be formulated as jointly estimating multiple different, but related, sparse Undirected Graphical Models (UGM) from aggregated samples across several co…
Robustness in sparse linear models: relative efficiency based on robust approximate message passingmath.ST
Understanding efficiency in high dimensional linear models is a longstanding problem of interest. Classical work with smaller dimensional problems dating back to Huber and Bickel has illustrated the benefits of efficient loss functions. When the number of parameters is of the same order as the sample size , $p \…
A novel framework infers causal direction from symbolic sequences using pattern entropy.
problem Challenges in discovering causal direction from temporal symbolic data.
method Dictionary Based Pattern Entropy () framework integrating AIT and Shannon Information Theory.
result Minimizing pattern level uncertainty yields a robust framework for causal discovery.
Sparsity helps reduce the computational complexity of deep neural networks by skipping zeros. Taking advantage of sparsity is listed as a high priority in next generation DNN accelerators such as TPU. The structure of sparsity, i.e., the granularity of pruning, affects the efficiency of hardware accelerator design as w…