A novel Laplace-approximated Bayesian Tensor Network Kernel Machine (LA-TNKM) provides principled uncertainty estimates.
problem How to provide principled uncertainty estimates for tensor network kernel machines.
method Employing a linearized Laplace approximation for Bayesian inference.
result Consistently matches or surpasses Gaussian Processes and BNNs across diverse UCI regression benchmarks.
Tensor decomposition is an effective approach to compress over-parameterized neural networks and to enable their deployment on resource-constrained hardware platforms. However, directly applying tensor compression in the training process is a challenging task due to the difficulty of choosing a proper tensor rank. In o…
Bayesian Tensor Network combines prior and data likelihood for efficient prediction and parameter estimation.
problem Overfitting and poor performance in Tensor Network models.
method Introduce prior distribution, use Laplace approximation for posterior predictive distribution, and propose stable initialization for parameter estimation.
result Reduces overfitting and improves performance of Tensor Network models.
A new method reduces Volterra kernel complexity and uncertainty quantification.
problem Challenges in modeling nonlinear systems with Volterra series due to high model order.
method Bayesian Tensor Network Volterra kernel machines (BTN-V) using canonical polyadic decomposition.
result Competitive accuracy, enhanced uncertainty quantification, and reduced computational cost.
Bayesian TNKMs automatically infer model complexity and feature relevance.
problem Manual tuning of TN rank and feature dimensions is error-prone and computationally expensive.
method Bayesian approach with hierarchical priors on TN factors for automatic rank and feature selection.
result Superior performance in prediction accuracy, uncertainty quantification, interpretability, and scalability.
Tensor decomposition is an important technique for capturing the high-order interactions among multiway data. Multi-linear tensor composition methods, such as the Tucker decomposition and the CANDECOMP/PARAFAC (CP), assume that the complex interactions among objects are multi-linear, and are thus insufficient to repres…
This paper develops a method to train compact neural networks with reduced memory and computational costs.
problem Training large neural networks consumes excessive resources and energy.
method End-to-end training framework using Bayesian tensor decomposition with automatic rank determination.
result The method achieves significant parameter reduction and maintains or improves accuracy.
Streaming tensor factorization is a powerful tool for processing high-volume and multi-way temporal data in Internet networks, recommender systems and image/video data analysis. Existing streaming tensor factorization algorithms rely on least-squares data fitting and they do not possess a mechanism for tensor rank dete…
Sobol method applied to probabilistic networks for sensitivity analysis.
problem Measuring influence of probabilistic network nodes on a quantity of interest.
method Transforms global sensitivity analysis into marginalization inference exploiting network structure.
result Efficient computation of sensitivity indices for complex networks.
Bayesian tensor train kernel machine uses Laplace approximation for scalable GP regression.
problem Scalability limitations of Gaussian process regression.
method Bayesian tensor train kernel machine with Laplace approximation and variational inference.
result VI replaces cross-validation and offers up to 65x faster training.
It is known that describing or calculating the conditional probabilities of multiple events is exponentially expensive. In this work, Bayesian tensor network (BTN) is proposed to efficiently capture the conditional probabilities of multiple sets of events with polynomial complexity. BTN is a directed acyclic graphical …
We present a scalable Bayesian model for low-rank factorization of massive tensors with binary observations. The proposed model has the following key properties: (1) in contrast to the models based on the logistic or probit likelihood, using a zero-truncated Poisson likelihood for binary data allows our model to scale …
This paper shows how infinitely wide Tensor Networks converge to Gaussian Processes.
problem Understanding the relationship between Tensor Networks and Gaussian Processes.
method Analyzing the infinite-width limit of Tensor Networks and comparing them to Gaussian Processes.
result Infinitely wide Tensor Networks converge to Gaussian Processes, proving their equivalence.
SPIDER uses deep neural networks for streaming tensor factorization.
problem Lack of effective approach for deep tensor factorization of streaming data.
method Bayesian neural networks with spike-and-slab prior, Taylor expansions, moment matching, and EPI framework.
result Effective incremental updates for latent factors and NN weights.
Bayesian-TPNN improves ANOVA-TPNN for detecting higher-order components.
problem Difficulty in incorporating higher-order components in ANOVA-TPNN due to computational and memory constraints.
method Bayesian inference procedure for functional ANOVA model with TPNN basis functions.
result Bayesian-TPNN detects higher-order components with reduced computational cost.
Bayesian hierarchical tensor factorization model for international trade flows
problem Sparse semi-continuous tensor data modeling
method Bayesian hierarchical tensor factorization with Poisson and Gamma models
result Identifies multiway dependence in trade flows
Bayesian tensor train method recovers streaming data with high accuracy.
problem Recovering high-order, incomplete, and noisy streaming data.
method Bayesian tensor train decomposition using streaming variational Bayes method.
result The proposed SPTT algorithm excels in recovering streaming data compared to state-of-the-art methods.
MARS automatically selects tensor decomposition ranks, improving performance in neural network tasks.
problem Determining optimal decomposition ranks in tensor decompositions.
method MARS uses binary masks to learn optimal tensor structure during training via relaxed MAP estimation.
result MARS achieves better results than previous methods in various tasks.
Study reveals efficient recovery of multi-modal signals via Bayesian methods and sequential learning.
problem Recovering multiple high-dimensional signals from correlated modalities.
method Bayesian Approximate Message Passing and Sequential Curriculum Learning.
result Sequential learning strategy optimally recovers weak signals in multi-modal settings.
We introduce Bayesian multi-tensor factorization, a model that is the first Bayesian formulation for joint factorization of multiple matrices and tensors. The research problem generalizes the joint matrix-tensor factorization problem to arbitrary sets of tensors of any depth, including matrices, can be interpreted as u…
We introduce a dynamic generative model, Bayesian allocation model (BAM), which establishes explicit connections between nonnegative tensor factorization (NTF), graphical models of discrete probability distributions and their Bayesian extensions, and the topic models such as the latent Dirichlet allocation. BAM is base…
Paper bounds tensor decomposition's RLCT, aiding Bayesian inference.
problem Unclear mathematical property of tensor decomposition.
method Algebraic geometrical method for upper bound derivation.
result Upper bound of real log canonical threshold (RLCT) derived.
Nonparametric extension of tensor regression is proposed. Nonlinearity in a high-dimensional tensor space is broken into simple local functions by incorporating low-rank tensor decomposition. Compared to naive nonparametric approaches, our formulation considerably improves the convergence rate of estimation while maint…
Paper projects GP basis functions using tensor networks to reduce complexity.
problem Efficiently approximating Gaussian process regression with a large number of basis functions.
method Develops a method using tensor networks to approximate GP regression with an exponential number of basis functions without exponential computational complexity.
result Shows efficient GP regression on an 18-dimensional benchmark data set.
BRTR improves robust tensor completion with automatic rank detection.
problem Robust tensor completion from incomplete data with outliers.
method Bayesian robust tensor ring decomposition (BRTR) with variational Bayesian (VB) algorithm.
result Automatic detection of TR rank and improved performance over state-of-the-art methods.
This paper reviews Bayesian methods for sparsity-aware modeling.
problem Uncertainty evaluation and robustness in sparsity-aware models.
method Incorporates sparsity-promoting priors into deep neural networks, Gaussian processes, and tensor decomposition.
result Bayesian methods improve model robustness and uncertainty evaluation.
Current high-throughput data acquisition technologies probe dynamical systems with different imaging modalities, generating massive data sets at different spatial and temporal resolutions posing challenging problems in multimodal data fusion. A case in point is the attempt to parse out the brain structures and networks…
Anomaly Detection has several important applications. In this paper, our focus is on detecting anomalies in seller-reviewer data using tensor decomposition. While tensor-decomposition is mostly unsupervised, we formulate Bayesian semi-supervised tensor decomposition to take advantage of sparse labeled data. In addition…
Bayesian method for imputing missing values in tensor data.
problem Missing data in multi-way arrays (tensors) in biomedical studies.
method Flexible Bayesian framework with conjugate priors for CP factorization.
result Accurately captures uncertainty in microbiome profiles at missing timepoints.
Bayesian model improves image completion accuracy by automatically learning low rank structure.
problem Improving image completion accuracy with limited data and avoiding overfitting.
method Developed a Bayesian low rank tensor ring model with multiplicative interaction and Student-T distribution for sparse core factors.
result The proposed method outperforms state-of-the-art image completion techniques, especially in recovery accuracy.
Bayesian Tensor Ring factorization improved for scalability and handling of discrete data.
problem Scalability issues and handling of discrete data in Bayesian Tensor Ring factorization.
method Proposes a novel Bayesian Tensor Ring model with a nonparametric Multiplicative Gamma Process prior and Pólya-Gamma augmentation for discrete data. Developed efficient Gibbs sampler and online EM algorithm for scalability.
result Significantly improved scalability and handling of discrete data compared to previous methods.
Probabilistic approaches for tensor factorization aim to extract meaningful structure from incomplete data by postulating low rank constraints. Recently, variational Bayesian (VB) inference techniques have successfully been applied to large scale models. This paper presents full Bayesian inference via VB on both single…
Paper presents a method for imputing and forecasting structural response from incomplete sensor data.
problem Missing sensor data in structural health monitoring (SHM).
method Incremental Bayesian tensor learning for spatiotemporal missing data reconstruction and forecasting.
result The proposed method achieves accurate and robust imputation and prediction even with high rates of missing data.
BKTF uses tensor factorization for Bayesian optimization of complex functions.
problem Complex functions with nonstationary, nonseparable, and multimodal features.
method Bayesian Kernelized Tensor Factorization (BKTF) approximates complex functions using a low-rank tensor CP decomposition with GP priors.
result BKTF provides flexible and effective surrogate modeling with uncertainty quantification.
We present a Bayesian tensor factorization model for inferring latent group structures from dynamic pairwise interaction patterns. For decades, political scientists have collected and analyzed records of the form "country i took action a toward country j at time t"---known as dyadic events---in order to form an…
A new metric tensor improves Riemann manifold Monte Carlo for Bayesian models.
problem Improving sampling efficiency in Bayesian hierarchical models.
method Metric tensor derived from log-density gradient covariance matrices.
result Metric tensors enhance sampling for complex Bayesian models.
Bayesian model predicts phenotype effects from multi-environmental factors.
problem Predict phenotype effects from multi-environmental trials.
method Bayesian tensor regression with spike-and-slab structure.
result Model outperforms previous methods in simulation and real-world data.
A multi-way factor analysis model is introduced for tensor-variate data of any order. Each data item is represented as a (sparse) sum of Kruskal decompositions, a Kruskal-factor analysis (KFA). KFA is nonparametric and can infer both the tensor-rank of each dictionary atom and the number of dictionary atoms. The model …
The vast majority of current machine learning algorithms are designed to predict single responses or a vector of responses, yet many types of response are more naturally organized as matrices or higher-order tensor objects where characteristics are shared across modes. We present a new machine learning algorithm BaTFLE…
KReTTaH uses tensor trains and Hadamard overparameterization for fast, interpretable multi-way data imputation.
problem Multi-way data imputation for high-dimensional functional MRI and dynamic graph recovery.
method Reformulates imputation as RKHS regression with TT-constrained coefficients and Hadamard overparameterization. Optimizes TT coefficients and kernel matrices on Riemannian manifolds.
result Consistently outperforms state-of-the-art methods in modeling accuracy.
KReTTaH uses tensor trains and Hadamard overparameterization for fast, interpretable multi-way data imputation.
problem Multi-way data imputation in high-dimensional spaces.
method Reformulates imputation as RKHS regression with TT-constrained coefficients, optimized on manifold frameworks.
result Consistently outperforms state-of-the-art methods in accuracy.
This paper presents a Bayesian method for estimating the rank of a low-rank tensor model of joint PMF.
problem Estimating the rank of a low-rank tensor model of joint PMF from observed data.
method Bayesian framework for estimating low-rank components and rank simultaneously, using variational inference.
result Automatic rank detection and improved estimation accuracy compared to cross-validation methods.
FunBaT extends Tucker decomposition to handle continuous-indexed tensor data.
problem Handling continuous-indexed tensor data that doesn't fit traditional Tucker decomposition.
method FunBaT treats continuous-indexed data as interactions between a core tensor and a group of latent functions modeled by Gaussian processes (GP). It converts each GP into a state-space prior and uses advanced message-passing techniques for scalable inference.
result FunBaT effectively handles real-world data with continuous indexes, demonstrating its advantage in synthetic and real-world applications.
A new probabilistic BTD method for tensor data.
problem Modeling higher-order tensors with robust inference.
method Probabilistic Block-Term Decomposition using variational Bayesian inference and von-Mises Fisher distribution.
result The proposed pBTD can quantify multi-linear structures robustly.
CANDECOMP/PARAFAC (CP) tensor factorization of incomplete data is a powerful technique for tensor completion through explicitly capturing the multilinear latent factors. The existing CP algorithms require the tensor rank to be manually specified, however, the determination of tensor rank remains a challenging problem e…
BKTR models spatiotemporal data with scalable tensor regression.
problem High computational cost in applying STVC to large-scale spatiotemporal data.
method Summarize STVC coefficients in a tensor, reformulate as low-rank tensor regression, incorporate GP priors for local dependencies.
result BKTR efficiently models large spatiotemporal datasets with reduced parameters and local dependencies.
This paper presents a multi-dimensional computational method to predict the spatial variation data inside and across multiple dies of a wafer. This technique is based on tensor computation. A tensor is a high-dimensional generalization of a matrix or a vector. By exploiting the hidden low-rank property of a high-dimens…
A scalable method for efficient inference in Gaussian process regression networks.
problem Intractable inference in Gaussian process regression networks (GPRN).
method Tensorization of output space, tensor/matrix-normal variational posteriors, joint optimization, and exploiting Kronecker product structure.
result Captures posterior dependencies and improves inference quality for large number of outputs.