Bayesian model infers factor dimensionality and sparse loading matrix adaptively.
problem Inference of high-dimensional sparse factor model with varying sparsity and factor dimensions.
method Adaptive Bayesian sparse factor model with posterior concentration.
result Posterior distribution asymptotically concentrates on true factor dimensionality and sparsity.
New method relaxes PCA orthogonality constraints using explained variance of correlated components.
problem Difficulty in using PCA for sparse design due to orthogonality constraints and non-differentiable penalty.
method Introduce expvar(Y) to measure variance explained by correlated components, relax orthogonality constraints.
result Two expvar(Y) definitions suitable for block PCA formulations without orthogonality constraints.
Sparse GCA finds linear relationships in multiple datasets, using gradient descent.
problem Finding linear relationships across multiple datasets with sparse loading vectors.
method Formulated as generalized eigenvalue problems, used a thresholded gradient descent algorithm.
result Proposed algorithm yields tight estimation error bounds and demonstrates effectiveness on synthetic datasets.
In (exploratory) factor analysis, the loading matrix is identified only up to orthogonal rotation. For identifiability, one thus often takes the loading matrix to be lower triangular with positive diagonal entries. In Bayesian inference, a standard practice is then to specify a prior under which the loadings are indepe…
We address the curse of dimensionality in dynamic covariance estimation by modeling the underlying co-volatility dynamics of a time series vector through latent time-varying stochastic factors. The use of a global-local shrinkage prior for the elements of the factor loadings matrix pulls loadings on superfluous factors…
Deep learning boosts building energy load forecasting.
problem Short-term load forecasting in buildings.
method Stacked Boosters Network architecture with sparse interactions, parameter sharing, and equivariant representations.
result Outperforms state-of-the-art models in short-term load forecasting tasks.
sPCA models may not have orthogonal scores and loadings, complicating interpretation.
problem sPCA scores and loadings may not be orthogonal.
method Illustrated and numerically demonstrated the implications of sPCA on scores, residuals, and variance explained.
result sPCA approaches perform poorly on noise-free, sparse data.
Grid security and open markets are two major smart grid goals. Transparency of market data facilitates a competitive and efficient energy environment, yet it may also reveal critical physical system information. Recovering the grid topology based solely on publicly available market data is explored here. Real-time ener…
Nonnegative Matrix Factorization (NMF) aims to factorize a matrix into two optimized nonnegative matrices appropriate for the intended applications. The method has been widely used for unsupervised learning tasks, including recommender systems (rating matrix of users by items) and document clustering (weighting matrix …
A new covariance estimator reduces dimensionality and improves portfolio forecasting.
problem Estimating high-dimensional covariance matrices with weak factors.
method Sparse Approximate Factor (SAF) model with l1-regularization. result SAF estimator outperforms other methods in portfolio forecasting.
Sparse principal component analysis (sparse PCA) aims at finding a sparse basis to improve the interpretability over the dense basis of PCA, meanwhile the sparse basis should cover the data subspace as much as possible. In contrast to most of existing work which deal with the problem by adding some sparsity penalties o…
Improves NILM with multi-label SRC, outperforming state-of-the-art.
problem Non-intrusive load monitoring (NILM) for energy disaggregation.
method Modified multi-label sparse representation based classification (SRC).
result Significant improvement over state-of-the-art techniques with minimal training data.
Sparse PCA selects variables with FDR control for improved performance.
problem Sparse PCA selects irrelevant variables when maximizing explained variance.
method Proposes FDR-controlled selection using T-Rex selector.
result Significant performance improvement over traditional sparse PCA.
New method enforces encoder sparsity in HPF for more interpretable feature selection.
problem Lack of encoder sparsity in HPF leads to lack of column-clustering property.
method Enforces encoder sparsity using a generalized additive model (GAM).
result Gains ability to perform feature selection and relates each representation to original features.
msPCA solves sparse PCA for multiple components efficiently.
problem Sparse principal component analysis with multiple components.
method Alternating maximization algorithm for sparse loading vectors, with orthogonality or zero correlation constraints.
result Achieves high variance explained with sparse components and controlled feasibility violations.
This work introduces a novel estimation method, called LOVE, of the entries and structure of a loading matrix A in a sparse latent factor model X = AZ + E, for an observable random vector X in Rp, with correlated unobservable factors Z \in RK, with K unknown, and independent noise E. Each row of A is scaled and sparse.…
New method infers co-expression networks robustly from multiple studies.
problem Challenges in inferring co-expression networks from transcriptome data.
method Robust method based on multivariate t-distribution with shared precision matrix.
result Identifies co-expression matrix up to scaling factor.
This paper proposes a submodular load clustering method for transmission-level load areas.
problem Traditional load analysis challenges with new electricity usage patterns.
method Robust Principal Component Analysis (R-PCA) and submodular cluster center selection.
result The proposed method efficiently clusters load areas and demonstrates effectiveness in PJM load data.
Paper proposes a new method for hourly load forecasting using smart meter data.
problem Challenges in short-term load forecasting at fine granularity.
method Forecasting using Matrix Factorization (fmf) for hourly load forecasting.
result Significantly outperforms state-of-the-art methods in load forecasting.
New method infers viral load from pooled tests.
problem Inefficient viral load inference in pooled testing.
method Message passing algorithm with PCR noise function.
result Accurate viral load inference possible.
New FGSPCA method captures grouping and sparse structures in PCA without prior info.
problem Capture grouping and sparse structures in PCA without prior info.
method Truncated regularization with alternating algorithm.
result FGSPCA method reduces model complexity and increases interpretability.
Although there is a rich literature on methods for allowing the variance in a univariate regression model to vary with predictors, time and other factors, relatively little has been done in the multivariate case. Our focus is on developing a class of nonparametric covariance regression models, which allow an unknown p …
Principal component regression (PCR) is a two-stage procedure that selects some principal components and then constructs a regression model regarding them as new explanatory variables. Note that the principal components are obtained from only explanatory variables and not considered with the response variable. To addre…
A time-varying cointegration model for foreign exchange rates is presented. Unlike previous studies, we allow the loading matrix in the vector error correction (VEC) model to be varying over time. Because the loading matrix in the VEC model is associated with the speed at which deviations from the long-run relationship…
A distributed framework for reducing high-dimensional matrix-variate time series data.
problem Reducing dimensionality of high-dimensional, heterogeneous matrix-variate time series data.
method Data partitioning, distributed two-dimensional tensor PCA, aggregation, final PCA, factor matrix computation.
result Preserves latent matrix structure, improves computational efficiency and information utilization.
Sparse versions of principal component analysis (PCA) have imposed themselves as simple, yet powerful ways of selecting relevant features of high-dimensional data in an unsupervised manner. However, when several sparse principal components are computed, the interpretation of the selected variables is difficult since ea…
A new model uses sparse Gaussian processes to hedge electricity market risks.
problem Risk minimization in electricity markets due to non-storability and volatility.
method Coregionalized sparse Gaussian processes to model price and load correlations.
result The model outperforms traditional average-load strategies in hedging.
R package spca computes sparse principal components efficiently.
problem Sparse principal components analysis (SPCA) for interpretable data.
method Least squares sparse principal component analysis (LS-SPCA) with efficient C++ backend.
result Computes sparse principal components that maximize variance and maintain strong correlations with PCs.
Principal component analysis (PCA) is a widely used technique for data analysis and dimension reduction with numerous applications in science and engineering. However, the standard PCA suffers from the fact that the principal components (PCs) are usually linear combinations of all the original variables, and it is thus…
Sparse APCA identifies sparse factors in financial returns over time.
problem Analyzing co-movements of high-dimensional panel data over time.
method Sparse asymptotic PCA with truncated power method for sparse factors and sequential deflation for multi-factor cases.
result Identification of nine risk factors influencing the S&P 500 stock market.
Variable selection and dimension reduction are two commonly adopted approaches for high-dimensional data analysis, but have traditionally been treated separately. Here we propose an integrated approach, called sparse gradient learning (SGL), for variable selection and dimension reduction via learning the gradients of t…
BN^2MF identifies unknown exposure patterns in environmental mixtures.
problem Identifying unknown exposure patterns in environmental mixtures.
method Bayesian non-parametric non-negative matrix factorization (BN^2MF) with non-negative continuous priors and a non-parametric sparse prior.
result Estimates patterns of chemical exposures without specifying the number of patterns.
Proposes ESCA model to analyze mixed data types in multiple sets of measurements.
problem Separating common and distinct information in mixed data types from multiple sources.
method Exponential Family Simultaneous Component Analysis (ESCA) model with structured sparse loading matrix.
result The proposed method effectively disentangles global, local common and distinct information.
SDSPCA improves PCA for disease diagnosis using sparse components and discriminative information.
problem Class ambiguity and low interpretability in traditional PCA.
method Incorporates discriminative information and sparsity into PCA, focusing on sparse components.
result SDSPCA outperforms other methods in gene selection and tumor classification on multi-view biological data.
Bayesian method improves sparse CCA for multi-view data.
problem Integrative statistical analysis of multi-view high-dimensional data.
method Bayesian infinite factor model with graphical horseshoe prior or diagonal structure to encourage sparsity.
result The proposed Bayesian ScSCCA approach achieves robust estimation of sparse CCA.
We consider the problem of sparse estimation in a factor analysis model. A traditional estimation procedure in use is the following two-step approach: the model is estimated by maximum likelihood method and then a rotation technique is utilized to find sparse factor loadings. However, the maximum likelihood estimates c…
The paper extends and applies a new shrinkage prior in Bayesian factor analysis.
problem Estimating the number of factors in sparse Bayesian factor analysis.
method Introduces and extends a generalized cumulative shrinkage process (CUSP) prior.
result Exchangeable spike-and-slab shrinkage priors imply increasing shrinkage as the column index increases.
New method for hyperparameter tuning in sparse matrix factorization.
problem Hyperparameter tuning in sparse matrix factorization.
method Numerical method based on evaluating the zero point of normalization factor in sparse matrix prior.
result Our method outperforms existing algorithms in ground-truth sparse matrix reconstruction.
Energy consumption is an important issue in continuous wireless telemonitoring of physiological signals. Compressed sensing (CS) is a promising framework to address it, due to its energy-efficient data compression procedure. However, most CS algorithms have difficulty in data recovery due to non-sparsity characteristic…
New method for inference on covariates in NMF with random effects.
problem Formal inference for covariate effects in NMF with non-negativity constraints.
method NMF-RE model with random effects, ridge updates, df-based cap, asymptotic linearization, wild bootstrap.
result Valid inference on covariates with non-negativity constraint, avoiding degeneracy.
Matrix Factorization (MF) on large scale matrices is computationally as well as memory intensive task. Alternative convergence techniques are needed when the size of the input matrix is higher than the available memory on a Central Processing Unit (CPU) and Graphical Processing Unit (GPU). While alternating least squar…
Paper develops a decoder for sparse codes without encoder matrix, achieving optimal recovery.
problem Designing a decoder for sparse codes from linear measurements alone.
method Matrix factorization to recover encoder and sparse coding matrices from measurements.
result Decoder-Expander Based Factorisation recovers encoder and sparse coding matrix at optimal measurement rate with high probability.
New optimizer designs respect symmetry, improving deep learning models.
problem Optimizers lack respect for symmetry in neural networks.
method Introduce symmetry-compatible principle for optimizer design.
result Symmetry-compatible optimizers improve model performance.
Ranky solves SVD for large sparse matrices in distributed systems.
problem Rank problem in large sparse matrices for SVD.
method Distributed approach to solve rank problem.
result Recovers SVD with negligible error for large sparse matrices.
Latent factor models are the canonical statistical tool for exploratory analyses of low-dimensional linear structure for an observation matrix with p features across n samples. We develop a structured Bayesian group factor analysis model that extends the factor model to multiple coupled observation matrices; in the cas…
Bayesian SPCA method tackles orthogonality constraint with spike and slab prior.
problem Bayesian SPCA method for high-dimensional data with orthogonality constraint.
method Parameter-expanded coordinate ascent variational inference (PX-CAVI) with spike and slab prior.
result PX-CAVI algorithm outperforms existing SPCA approaches in performance.
CP-factorization for high-dimensional tensor time series and double projection iterations
problem Identifying and estimating factor loadings in CP decomposition for high-dimensional tensor time series
method One-pass estimation procedure using standard eigen-analysis for matrix constructed based on serial dependence
result Asymptotic properties established under general settings, adapt to sparsity, accommodates weak factors
New matrix reveals cluster info in sparse directed graphs.
problem Analyzing cluster information in directed graphs.
method Proposed complex non-backtracking matrix integrating Hermitian adjacency matrix and non-backtracking matrix properties.
result The complex non-backtracking matrix holds cluster information, especially for sparse directed graphs.