Big data sets must be carefully partitioned into statistically similar data subsets that can be used as representative samples for big data analysis tasks. In this paper, we propose the random sample partition (RSP) data model to represent a big data set as a set of non-overlapping data subsets, called RSP data blocks,…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “data block model”
We consider practical data characteristics underlying federated learning, where unbalanced and non-i.i.d. data from clients have a block-cyclic structure: each cycle contains several blocks, and each client's training data follow block-specific and non-i.i.d. distributions. Such a data structure would introduce client …
Machine learning predicts Atlantic blocking using limited data.
problem Underestimation of blocking event duration in climate models.
method Transfer Learning and Explainable AI (SHAP analysis)
result High-pressure anomalies in specific regions contribute to blocking events.
This work is devoted to elaboration on the idea to use block term decomposition for group data analysis and to raise the possibility of modelling group activity with (Lr, 1) and Tucker blocks. A new generalization of block tensor decomposition was considered in application to group data analysis. Suggested approach was…
The tree reconstruction problem is to collect and analyze massive data at the th level of the tree, to identify whether there is non-vanishing information of the root, as goes to infinity. Its connection to the clustering problem in the setting of the stochastic block model, which has wide applications in machin…
BSTabDiff: Block-Subunit Diffusion Priors for HDLSS Tabular Data Generation
problem High-dimensional tabular data generation in HDLSS
method Block-subunit generative framework
result More realistic and stable synthetic data
This paper presents a novel Block Iterative Bayesian Algorithm (Block-IBA) for reconstructing block-sparse signals with unknown block structures. Unlike the existing algorithms for block sparse signal recovery which assume the cluster structure of the nonzero elements of the unknown signal to be independent and identic…
Proposes a Latent Block Model for analyzing missing data.
problem Missing data can lead to misleading conclusions if not properly accounted for.
method Co-clustering model based on Latent Block Model, with variational EM algorithm for inference and model selection criterion.
result The proposed model reveals meaningful groups and insights from non-voters in French Parliament voting records.
New model for community detection with side information improves recovery accuracy.
problem Community detection in networks with additional node data.
method Data Block Model (DBM) with Chernoff--TV divergence for threshold characterization and efficient algorithm.
result Sharp exact recovery threshold and efficient algorithm for DBM.
We generalize the stochastic block model to the important case in which edges are annotated with weights drawn from an exponential family distribution. This generalization introduces several technical difficulties for model estimation, which we solve using a Bayesian approach. We introduce a variational algorithm that …
Study community detection in multi-view data with various types of information.
problem Community detection in multi-view data with different types of information.
method Unified theoretical framework, mutual information analysis, sharp thresholds, iterative algorithms.
result Sharp thresholds for community recovery in various multi-view settings.
Extends co-clustering to mixed numerical and binary data.
problem Co-clustering of mixed data types (numerical and binary).
method Latent block models for mixed data types.
result Effectiveness of the proposed approach on simulated data.
This paper speeds up Gaussian process regression for autocorrelated data.
problem Temporal overfitting in Gaussian process models for autocorrelated data.
method Modifying existing Gaussian process approximations to handle blocked, de-correlated data.
result Proposed methods accelerate Gaussian process regression on autocorrelated data without sacrificing performance.
Method estimates number of clusters in Block Markov Chain trajectories.
problem Challenges in choosing number of clusters for sequential data.
method Spectral embedding and density-based clustering.
result Asymptotically consistent method for estimating clusters.
The proliferation of models for networks raises challenging problems of model selection: the data are sparse and globally dependent, and models are typically high-dimensional and have large numbers of latent variables. Together, these issues mean that the usual model-selection criteria do not work properly for networks…
We propose a mixture of latent trait models with common slope parameters (MCLT) for model-based clustering of high-dimensional binary data, a data type for which few established methods exist. Recent work on clustering of binary data, based on a -dimensional Gaussian latent variable, is extended by incorporating com…
This note corrects a mistake in the paper "consistent cross-validatory model-selection for dependent data: -block cross-validation" by Racine (2000). In his paper, he implied that the therein proposed -block cross-validation is consistent in the sense of Shao (1993). To get this intuition, he relied on the spec…
Latent block models are used for probabilistic biclustering, which is shown to be an effective method for analyzing various relational data sets. However, there has been no statistical test method for determining the row and column cluster numbers of latent block models. Recent studies have constructed statistical-test…
Graphical lasso models ASR utterance dependencies for consistent WER estimation.
problem Modeling dependent structure among ASR utterances for accurate significance analysis.
method Graphical lasso for dependency modeling, followed by blockwise bootstrap resampling.
result Statistically consistent variance estimator of WER under mild conditions.
The paper uses deep learning to detect financial market regimes from correlation matrices.
problem Detecting financial market regimes from correlation dynamics.
method Representation learning on block hierarchical SPD correlation matrices using SPDNet, SPD-NetBN, and U-SPDNet models.
result Deep learning models overfit in financial market data, misleading performance metrics.
A novel MCMC method clusters data faster and more accurately.
problem Efficiently clustering large datasets with unknown number of clusters.
method Master/Worker architecture for distributed MCMC inference.
result Significant improvement in clustering accuracy and speed.
A main task in data analysis is to organize data points into coherent groups or clusters. The stochastic block model is a probabilistic model for the cluster structure. This model prescribes different probabilities for the presence of edges within a cluster and between different clusters. We assume that the cluster ass…
Knowledge Graph Embedding with Electronic Health Records Data via Latent Graphical Block Modelstat.ML
Proposes a method to derive knowledge graphs from EHR data.
problem Challenges in deriving generalizable knowledge from EHR data.
method Infer conditional dependency structure via a latent graphical block model (LGBM).
result Perfect recovery of block structure demonstrated.
Recurrent Neural Networks (RNNs) are used in state-of-the-art models in domains such as speech recognition, machine translation, and language modelling. Sparsity is a technique to reduce compute and memory requirements of deep learning models. Sparse RNNs are easier to deploy on devices and high-end server processors. …
A new method selects features for clustering using a block model.
problem Finding high-quality features for clustering in linked data.
method Building a block model on the graph and using it for feature selection.
result BMGUFS outperforms state-of-the-art methods in clustering performance.
Deep learning has shown promising results on many machine learning tasks but DL models are often complex networks with large number of neurons and layers, and recently, complex layer structures known as building blocks. Finding the best deep model requires a combination of finding both the right architecture and the co…
The excellent performance of representation learning of autoencoders have attracted considerable interest in various applications. However, the structure and multi-local collaborative relationships of unlabeled data are ignored in their encoding procedure that limits the capability of feature extraction. This paper pre…
This paper introduces GLT for better input data representation in BNN and proposes a compact topology with block pruning.
problem Improving input data representation for Binary Neural Networks (BNN).
method Generic Learned Thermometer (GLT) for encoding, block pruning and Knowledge Distillation for compact topology.
result Significant accuracy gains and lightweight fully-binarized models with limited accuracy degradation.
Paper proposes ABDR for convex subspace clustering with adaptive block diagonal representation.
problem Subspace clustering with block diagonal structure for noisy data.
method ABDR explicitly pursues block diagonality without sacrificing convexity, using a specially designed convex regularizer.
result Experimental results show ABDR outperforms state-of-the-arts.
We consider the problem of identifying multiway block structure from a large noisy tensor. Such problems arise frequently in applications such as genomics, recommendation system, topic modeling, and sensor network localization. We propose a tensor block model, develop a unified least-square estimation, and obtain the t…
Spectral clustering achieves strong consistency in the stochastic block model under certain conditions.
problem Achieving strong consistency in spectral clustering for the stochastic block model.
method Entrywise analysis of the Fielder eigenvector of graph Laplacians.
result Spectral clustering achieves exact recovery of hidden communities under matching information-theoretic limits.
Spectral embedding is a popular technique for the representation of graph data. Several regularization techniques have been proposed to improve the quality of the embedding with respect to downstream tasks like clustering. In this paper, we explain on a simple block model the impact of the complete graph regularization…
There exist various types of network block models such as the Stochastic Block Model (SBM), the Degree Corrected Block Model (DCBM), and the Popularity Adjusted Block Model (PABM). While this leads to a variety of choices, the block models do not have a nested structure. In addition, there is a substantial jump in the …
Many statistical methods for network data parameterize the edge-probability by attributing latent traits to the vertices such as block structure and assume exchangeability in the sense of the Aldous-Hoover representation theorem. Empirical studies of networks indicate that many real-world networks have a power-law dist…
Two spectral algorithms for community detection in graphs with covariates are compared.
problem Detecting community structure in graphs with covariates.
method Two model-based spectral algorithms are presented and compared.
result The second algorithm often better estimates block assignments by accounting for vertex covariates.
BlockEcho method improves imputation of block-wise missing data.
problem Block-wise missing data reduces interpolation capability and predictive power.
method Integrates Matrix Factorization (MF) within Generative Adversarial Networks (GAN) to retain long-distance inter-element relationships.
result Superior performance on public datasets across three domains, especially at higher missing rates.
Two new methods improve block-sparse signal recovery from noisy data.
problem Recovering block-sparse signals with unknown partitions.
method LogLOP-l2/l1 and AdaLOP-l2/l1 methods using log-sum penalty and MCP.
result Our methods outperform existing techniques in estimation accuracy.
The Mallows model, introduced in the seminal paper of Mallows 1957, is one of the most fundamental ranking distribution over the symmetric group . To analyze more complex ranking data, several studies considered the Generalized Mallows model defined by Fligner and Verducci 1986. Despite the significant research in…
New model for clustering graphs with multiple data sources.
problem Graph clustering with multiple data sources.
method Formalized multi-view stochastic block models and developed efficient algorithms.
result Provable improvement over previous approaches in multi-view graph clustering.
We present a method based on the orthogonal symmetric non-negative matrix tri-factorization of the normalized Laplacian matrix for community detection in complex networks. While the exact factorization of a given order may not exist and is NP hard to compute, we obtain an approximate factorization by solving an optimiz…
Improved model for grouping nodes in bipartite networks.
problem Challenges in grouping nodes in bipartite graphs.
method Introduced DC-LBM and developed variational EM algorithm.
result Significantly enhanced performance on real-world data.
We introduce block-tree graphs as a framework for deriving efficient algorithms on graphical models. We define block-tree graphs as a tree-structured graph where each node is a cluster of nodes such that the clusters in the graph are disjoint. This differs from junction-trees, where two clusters connected by an edge al…
Unified multilinear model for causal factor disentanglement.
problem Disentangling causal factors from complex data without direct manipulation.
method Hierarchical block multilinear factorization (M-mode Block SVD) and incremental approach.
result Interpretable object representation robust to occlusion and reduced training data.
Tests if vertices in graphs have the same latent positions.
problem Testing equality of latent positions in random graphs.
method Empirical Mahalanobis distances from spectral embeddings.
result Test statistics follow chi-square distributions under null and local alternatives.
Paper proposes efficient methods for clustering and signal recovery in high-dimensional data with block structures.
problem High-dimensional clustering and signal recovery under block signal structures.
method CFA-PCA and MA-PCA methods for sparse and dense block signals.
result Proposed methods achieve computational minimax optimality for clustering and signal recovery.
Corporate Bond Yield Curve Modeling: A Rating-Based Regime-Switching Generalized CIR Approachq-fin.PR
Develops a new model to better predict corporate bond yields.
problem Persistent shifts in interest rates undermine single-regime models.
method Regime-switching generalized CIR model with two-state short-rate process and credit factors.
result The model improves joint curve fit and delivers interpretable probabilities.
A central problem in analyzing networks is partitioning them into modules or communities. One of the best tools for this is the stochastic block model, which clusters vertices into blocks with statistically homogeneous pattern of links. Despite its flexibility and popularity, there has been a lack of principled statist…
New algorithm optimally clusters networks with side information.
problem Improving network clustering with side information.
method Iterative clustering algorithm for Contextual Stochastic Block Model.
result Optimal performance under Contextual Symmetric Stochastic Block Model.