A novel Laplace-approximated Bayesian Tensor Network Kernel Machine (LA-TNKM) provides principled uncertainty estimates.
problem How to provide principled uncertainty estimates for tensor network kernel machines.
method Employing a linearized Laplace approximation for Bayesian inference.
result Consistently matches or surpasses Gaussian Processes and BNNs across diverse UCI regression benchmarks.
A new method reduces Volterra kernel complexity and uncertainty quantification.
problem Challenges in modeling nonlinear systems with Volterra series due to high model order.
method Bayesian Tensor Network Volterra kernel machines (BTN-V) using canonical polyadic decomposition.
result Competitive accuracy, enhanced uncertainty quantification, and reduced computational cost.
Bayesian TNKMs automatically infer model complexity and feature relevance.
problem Manual tuning of TN rank and feature dimensions is error-prone and computationally expensive.
method Bayesian approach with hierarchical priors on TN factors for automatic rank and feature selection.
result Superior performance in prediction accuracy, uncertainty quantification, interpretability, and scalability.
Bayesian tensor train kernel machine uses Laplace approximation for scalable GP regression.
problem Scalability limitations of Gaussian process regression.
method Bayesian tensor train kernel machine with Laplace approximation and variational inference.
result VI replaces cross-validation and offers up to 65x faster training.
Tensor networks constrain kernel machines to Gaussian processes.
problem Speeding up kernel machines with reduced model complexity.
method Proving CPD and TT-constrained models recover Gaussian processes with i.i.d. priors.
result TT-constrained models exhibit more Gaussian process behavior than CPD for the same parameters.
BKTF uses tensor factorization for Bayesian optimization of complex functions.
problem Complex functions with nonstationary, nonseparable, and multimodal features.
method Bayesian Kernelized Tensor Factorization (BKTF) approximates complex functions using a low-rank tensor CP decomposition with GP priors.
result BKTF provides flexible and effective surrogate modeling with uncertainty quantification.
Proposes a new kernel technique for tensor data in SVM.
problem Handling tensorial data in machine learning.
method Kernelized support tensor train machine for image classification.
result Tensorizes the standard SVM on its input structure and kernel mapping scheme.
Bayesian tensor network reduces conditional probability calculation to polynomial time.
problem Exponential cost of calculating conditional probabilities for multiple events.
method Bayesian tensor network (BTN) with polynomial complexity.
result Competitive performance in image recognition with simple tree structures.
KReTTaH uses tensor trains and Hadamard overparameterization for fast, interpretable multi-way data imputation.
problem Multi-way data imputation for high-dimensional functional MRI and dynamic graph recovery.
method Reformulates imputation as RKHS regression with TT-constrained coefficients and Hadamard overparameterization. Optimizes TT coefficients and kernel matrices on Riemannian manifolds.
result Consistently outperforms state-of-the-art methods in modeling accuracy.
KReTTaH uses tensor trains and Hadamard overparameterization for fast, interpretable multi-way data imputation.
problem Multi-way data imputation in high-dimensional spaces.
method Reformulates imputation as RKHS regression with TT-constrained coefficients, optimized on manifold frameworks.
result Consistently outperforms state-of-the-art methods in accuracy.
This paper shows how infinitely wide Tensor Networks converge to Gaussian Processes.
problem Understanding the relationship between Tensor Networks and Gaussian Processes.
method Analyzing the infinite-width limit of Tensor Networks and comparing them to Gaussian Processes.
result Infinitely wide Tensor Networks converge to Gaussian Processes, proving their equivalence.
A new kernel improves tensor classification accuracy and reduces computation time.
problem Challenges in classifying high-dimensional tensor data.
method Proposes a weighted subspace exponential kernel based on Tucker decomposition.
result The new kernel outperforms existing methods in accuracy and computational efficiency.
BKTR models spatiotemporal data with scalable tensor regression.
problem High computational cost in applying STVC to large-scale spatiotemporal data.
method Summarize STVC coefficients in a tensor, reformulate as low-rank tensor regression, incorporate GP priors for local dependencies.
result BKTR efficiently models large spatiotemporal datasets with reduced parameters and local dependencies.
Low-rank tensor regression, a new model class that learns high-order correlation from data, has recently received considerable attention. At the same time, Gaussian processes (GP) are well-studied machine learning models for structure learning. In this paper, we demonstrate interesting connections between the two, espe…
Bayesian Optimization improves machine learning for detecting network attacks.
problem Detecting network attacks to secure critical information.
method Bayesian Optimization for tuning machine learning algorithms (SVM-RBF, RF, k-NN).
result The proposed framework achieves high accuracy and low false alarms.
Efficiently fine-tunes patient-independent seizure detection models with tensor kernel machine.
problem Improving seizure detection accuracy for wearable devices.
method Transfer learning with tensor kernel machine using canonical polyadic decomposition.
result Patient fine-tuned model achieves high performance with smaller model size.
Novel quasi-Bayesian method for IV regression using machine learning models.
problem Uncertainty quantification in IV regression with machine learning models.
method Quasi-Bayesian procedure based on kernelized IV models and dual formulation.
result Established minimax optimal contraction rates and scalable inference algorithm.
Adaptive kernels from neural networks improve model performance.
problem Improving neural network performance through adaptive kernels.
method Deriving adaptive kernels from infinite-width neural networks using feature learning and gradient flow training.
result Adaptive kernels achieve lower test loss compared to traditional kernels.
The interest in machine learning with tensor networks has been growing rapidly in recent years. We show that tensor-based methods developed for learning the governing equations of dynamical systems from data can, in the same way, be used for supervised learning problems and propose two novel approaches for image classi…
Bayesian-TPNN improves ANOVA-TPNN for detecting higher-order components.
problem Difficulty in incorporating higher-order components in ANOVA-TPNN due to computational and memory constraints.
method Bayesian inference procedure for functional ANOVA model with TPNN basis functions.
result Bayesian-TPNN detects higher-order components with reduced computational cost.
Paper bounds tensor decomposition's RLCT, aiding Bayesian inference.
problem Unclear mathematical property of tensor decomposition.
method Algebraic geometrical method for upper bound derivation.
result Upper bound of real log canonical threshold (RLCT) derived.
Tensor decomposition is an effective approach to compress over-parameterized neural networks and to enable their deployment on resource-constrained hardware platforms. However, directly applying tensor compression in the training process is a challenging task due to the difficulty of choosing a proper tensor rank. In o…
Develops a new tensor classification method for high-dimensional data.
problem Efficient learning algorithms exploiting tensorial structure in high-dimensional multi-way arrays.
method Tensor Train Multi-way Multi-level Kernel (TT-MMK) combining Canonical Polyadic decomposition, Dual Structure-preserving Support Vector Machine, and Tensor Train approximation.
result The TT-MMK method provides higher prediction accuracy and is more reliable computationally compared to other techniques.
Bayesian Tensor Network combines prior and data likelihood for efficient prediction and parameter estimation.
problem Overfitting and poor performance in Tensor Network models.
method Introduce prior distribution, use Laplace approximation for posterior predictive distribution, and propose stable initialization for parameter estimation.
result Reduces overfitting and improves performance of Tensor Network models.
The study investigates how data variability impacts the generalization of neural networks.
problem Understanding the impact of data variability on neural network generalization.
method Developed a field-theoretic formalism to compute generalization properties of neural networks, focusing on data variability.
result Data variability leads to non-Gaussian action, affecting the learning curve and generalization properties of neural networks.
A Python package for GPU-accelerated signature kernel computation.
problem Efficient computation of signature kernels for sequential data.
method GPU-accelerated algorithms and tensor sketches.
result New algorithm outperforms existing methods.
Convolutional DKMs improve kernel methods on MNIST, CIFAR-10, and CIFAR-100.
problem Improving kernel methods for image classification.
method Developed a novel inter-domain inducing point approximation and introduced various techniques to extend DKMs to convolutional networks.
result Achieved state-of-the-art performance on image classification benchmarks.
Bayesian approach models neurodegenerative diseases without clinical labels.
problem Personalized, predictive modeling of neurodegenerative diseases.
method Probabilistic programmed deep kernel learning combining Gaussian processes and neural networks.
result Surpasses deep learning in accuracy and timeliness of predicting neurodegeneration.
Inference in popular nonparametric Bayesian models typically relies on sampling or other approximations. This paper presents a general methodology for constructing novel tractable nonparametric Bayesian methods by applying the kernel trick to inference in a parametric Bayesian model. For example, Gaussian process regre…
This paper reviews Bayesian methods for sparsity-aware modeling.
problem Uncertainty evaluation and robustness in sparsity-aware models.
method Incorporates sparsity-promoting priors into deep neural networks, Gaussian processes, and tensor decomposition.
result Bayesian methods improve model robustness and uncertainty evaluation.
A new method discovers equations from data using Bayesian and kernel techniques.
problem Discovering equations from data is hard due to sparsity and noise.
method Kernel regression for function estimation and Bayesian spike-and-slab prior for uncertainty quantification.
result KBASS method outperforms state-of-the-art methods on benchmark tasks.
RBM and DBM are represented as 2D tensor networks, revealing their expressive power and efficiency.
problem Understanding and optimizing RBM and DBM models.
method Representing RBM and DBM as 2D tensor networks and developing an efficient tensor network contraction algorithm.
result The proposed algorithm for computing partition functions is more accurate than state-of-the-art methods.
Tensor decomposition is an important technique for capturing the high-order interactions among multiway data. Multi-linear tensor composition methods, such as the Tucker decomposition and the CANDECOMP/PARAFAC (CP), assume that the complex interactions among objects are multi-linear, and are thus insufficient to repres…
Bayesian model connects KMs and ELMs for multitask regression.
problem Multitask regression with shared features and sparsity.
method Bayesian framework using RFFs and RBF kernels.
result Significant performance improvements over state-of-the-art methods.
Hybrid tensor networks improve machine learning by combining quantum and classical methods.
problem Limitations of regular tensor networks in machine learning.
method Quantum-classical hybrid tensor networks (HTN) combining tensor networks and classical neural networks.
result HTN overcomes limitations of regular tensor networks and enables deep learning training.
Quantum kernel improves probabilistic time series forecasting.
problem Quantifying uncertainty in probabilistic time series predictions.
method Integrates quantum kernel with Gaussian process regression.
result Quantum kernel enhances forecasting performance.
The paper revisits and improves on a Bayesian relevance vector machine method for small sample sizes.
problem Statistical modeling with small sample sizes relative to the number of covariates.
method Introduces a new class of global-local priors and provides theoretical properties.
result Results on posterior consistency and contraction rates are provided.
HOTCAKE compresses CNNs by decomposing kernels into smaller parts.
problem Compressing deep CNNs without significant accuracy loss.
method Input channel decomposition, guided Tucker rank selection, higher order Tucker decomposition, fine-tuning.
result HOTCAKE produces highly compressed CNN models with good accuracy.
Graph-based kernels improve GP performance on graph data.
problem Improving Gaussian process performance on graph-structured data.
method Introduced graph neural network-inspired kernels into Gaussian processes.
result Graph convolutional networks are equivalent to certain GP kernels when infinitely wide.
In many problems of supervised tensor learning (STL), real world data such as face images or MRI scans are naturally represented as matrices, which are also called as second order tensors. Most existing classifiers based on tensor representation, such as support tensor machine (STM) need to solve iteratively which occu…
KFT improves tensor forecasting by incorporating side information.
problem Tensor factorization weaknesses in latent factors.
method Kernel Fried Tensor (KFT) with variational inference.
result Superior performance over LightGBM and FFM.
This paper develops a method to train compact neural networks with reduced memory and computational costs.
problem Training large neural networks consumes excessive resources and energy.
method End-to-end training framework using Bayesian tensor decomposition with automatic rank determination.
result The method achieves significant parameter reduction and maintains or improves accuracy.
The vast majority of current machine learning algorithms are designed to predict single responses or a vector of responses, yet many types of response are more naturally organized as matrices or higher-order tensor objects where characteristics are shared across modes. We present a new machine learning algorithm BaTFLE…
The study uses response theory to understand RNNs processing input signals.
problem Understanding how RNNs process sequential data.
method Deriving a Volterra series representation for SRNNs output using response theory from nonequilibrium statistical mechanics.
result SRNNs can be viewed as kernel machines operating on a reproducing kernel Hilbert space associated with the response feature.
Bayesian neural networks explore rare fluctuations for better feature learning.
problem Understanding rare but dominant fluctuations in Bayesian neural networks.
method Large-deviation theory and joint optimization over predictors and internal kernels.
result Posterior rate function optimization reveals data-dependent kernel selection.
Efficiently searches through Gaussian process kernels using symbolic representation and Bayesian optimization.
problem Manual selection of kernels in Gaussian processes is complex and computationally expensive.
method Proposes a novel method using symbolic representation and Bayesian optimization to search through a structured kernel space.
result Empirically shows a computationally more efficient way of searching through a discrete kernel space.
Kernel-based Bayesian filter for nonlinear systems using infinite-dimensional operators.
problem Modeling and predicting nonlinear dynamical systems.
method Functional Bayesian perspective, reproducing kernel Hilbert space, Gaussian kernel.
result Effective approximation and accurate results for nonlinear systems.
Theoretical studies have proven that the Hilbert space has remarkable performance in many fields of applications. Frames in tensor product of Hilbert spaces were introduced to generalize the inner product to high-order tensors. However, these techniques require tensor decomposition which could lead to the loss of infor…