Adaptive algorithm learns tensor network structures from data.
problem Identifying optimal tensor network structure from data.
method Greedy approach starting from rank one tensor, small rank increments.
result Adaptive algorithm identifies efficient tensor network structures.
Hybrid tensor networks improve machine learning by combining quantum and classical methods.
problem Limitations of regular tensor networks in machine learning.
method Quantum-classical hybrid tensor networks (HTN) combining tensor networks and classical neural networks.
result HTN overcomes limitations of regular tensor networks and enables deep learning training.
Study spectral learning for odeco tensors, addressing initialization bottlenecks.
problem Recovering orthogonally decomposable tensors under noise.
method Investigates perturbation bounds, non-convex optimization, and initialization strategies.
result Initialization is the main bottleneck for efficient algorithms.
Graphical notation simplifies tensor operations and decompositions.
problem Complex tensor operations are difficult to understand and represent.
method Introduces graphical notation to represent tensor operations.
result Simplified representation of tensor operations and decompositions.
The paper develops tensor learning methods exploiting symmetries of tensor functions.
problem Efficiently handling tensors in various scientific contexts.
method Equivariant machine learning architectures exploiting orthogonal, Lorentz, and symplectic symmetries.
result Equivariant models outperform non-equivariant baselines in time series analysis.
The paper explores tensor decompositions in deep learning models.
problem Compressing parameter space and creating richer representations.
method Tensor decompositions applied to deep learning models.
result Tensor methods can yield richer adaptive representations of complex data.
A tensor model for meta-learning adapts to task-specific features.
problem Learning shared representations for diverse tasks without task-specific observable information.
method Modeling meta-parameters as an order-3 tensor, estimating through tensor regression and method of moments.
result Tensor-based approach improves meta-learning performance with fewer samples.
Tensors are multidimensional arrays of numerical values and therefore generalize matrices to multiple dimensions. While tensors first emerged in the psychometrics community in the 20th century, they have since then spread to numerous other disciplines, including machine learning. Tensors and their decomposi…
The paper shows how gradient flow on over-parametrized tensor decomposition behaves like deflation.
problem Understanding the training dynamics of gradient flow on tensor decomposition.
method Empirical observation and mathematical proof of gradient flow dynamics for orthogonally decomposable tensors.
result Gradient flow dynamics for orthogonally decomposable tensors follows a tensor deflation process, recovering all tensor components.
Improved machine learning with reduced tensor rank constraints and dropout.
problem Efficiently approximating large tensors in machine learning.
method Tree tensor networks with CP rank constraints and tensor dropout.
result Low-rank TTN classifier achieves 90.3% accuracy in Fashion-MNIST.
Tensor networks have found a wide use in a variety of applications in physics and computer science, recently leading to both theoretical insights as well as practical algorithms in machine learning. In this work we explore the connection between tensor networks and probabilistic graphical models, and show that it motiv…
Proposes FATTNN for tensor-on-tensor regression with improved prediction and reduced computation.
problem Tensor-on-tensor regression with complex tensor structures and nonlinear relationships.
method Integrates tensor factor models into deep neural networks to handle nonlinearity and reduce data dimensionality.
result Significant improvements in prediction accuracy and computational efficiency over traditional methods.
Tensor decomposition recovers Gaussian mixtures from moments.
problem Recovering Gaussian mixture models from datasets.
method Symmetric tensor decomposition of moment tensors built from empirical moments.
result Identifiable tensors with interpolation degree less than half their order.
Deterministic bounds for tensor singular values and vectors, differing from matrix cases.
problem Spectral learning of higher-order orthogonally decomposable tensors.
method Deterministic perturbation bounds for singular values and vectors of orthogonally decomposable tensors.
result Perturbation affects each essential singular value/vector in isolation, independent of multiplicity and distance from other singular values.
Paper develops a method for causal representation learning from irregular tensors.
problem Complex patterns in high-dimensional, irregular tensor data.
method Novel causal formulation and CaRTeD framework integrating temporal causal representation learning with irregular tensor decomposition.
result Framework provides theoretical guarantees and outperforms state-of-the-art techniques.
Tensor networks are efficient representations of high-dimensional tensors which have been very successful for physics and mathematics applications. We demonstrate how algorithms for optimizing such networks can be adapted to supervised learning tasks by using matrix product states (tensor trains) to parameterize models…
The paper analyzes implicit regularization in tensor factorization using neural networks.
problem Understanding implicit regularization in tensor factorization.
method Dynamical systems perspective and gradient descent analysis.
result Gradient descent induces a form of greedy low tensor rank search.
We introduce Bayesian multi-tensor factorization, a model that is the first Bayesian formulation for joint factorization of multiple matrices and tensors. The research problem generalizes the joint matrix-tensor factorization problem to arbitrary sets of tensors of any depth, including matrices, can be interpreted as u…
A new diffusion model generates structured tensors for high-dimensional data.
problem Generating a structured tensor with a target distribution.
method Tucker diffusion model with Tucker-Unet architecture.
result Generated tensors converge to the true data distribution at a rate dependent on tensor mode dimensions.
Paper proposes a transfer learning framework for tensor Gaussian graphical models.
problem Pooling heterogeneous tensor data for improved estimation and variable selection.
method Transfer learning framework that uses data-adaptive weights from auxiliary domains.
result Significant improvement in estimation errors and variable selection consistency.
Paper develops RGN method for estimating low-rank tensors from noisy measurements.
problem Estimating low-rank tensors from noisy linear measurements.
method Riemannian Gauss-Newton (RGN) method for efficient low-rank tensor estimation.
result First local quadratic convergence guarantee of RGN for low-rank tensor estimation in noisy settings.
Machine learning models accurately predict molecular magnetic anisotropy tensors.
problem Accurately modeling molecular magnetic anisotropy tensors.
method Gaussian-moment neural-network approach for machine learning.
result Achieved accuracy of 0.3--0.4 cm−1 for magnetic anisotropy tensor predictions. Deep learning models predict option prices from 3D tensor data.
problem Predicting option prices for risk management and trading.
method 3D tensor representation of financial data, deep learning models (2D tensors in 3 channels).
result Proposed models outperform traditional methods like B-S model and vector-based LSTM.
A simple self-supervised model for tensor RPCA using deep unfolding.
problem Tensor robust principal component analysis (RPCA) challenges in practical applications.
method Deep unfolding with only four hyperparameters.
result Competitive or superior performance compared to supervised methods, even in data-starved scenarios.
Improved tensor rank learning for CPD models using a generalized hyperbolic prior.
problem Inaccurate tensor rank determination leads to overfitting or underfitting in CPD models.
method Introduced a generalized hyperbolic prior for automatic tensor rank learning in probabilistic CPD models.
result Significantly improved performance in learning both low and high tensor ranks, even for low SNR cases.
Proposes GTTN for discovering all low-rank structures in deep multi-task learning.
problem Discovering all low-rank structures among tasks in deep multi-task models.
method Introduces GTTN, a convex combination of matrix trace norms of all tensor flattenings, to automatically determine the importance of components.
result Demonstrates the effectiveness of GTTN on real-world datasets.
RBM and DBM are represented as 2D tensor networks, revealing their expressive power and efficiency.
problem Understanding and optimizing RBM and DBM models.
method Representing RBM and DBM as 2D tensor networks and developing an efficient tensor network contraction algorithm.
result The proposed algorithm for computing partition functions is more accurate than state-of-the-art methods.
We present a new method for online prediction and learning of tensors (N-way arrays, N>2) from sequential measurements. We focus on the specific case of 3-D tensors and exploit a recently developed framework of structured tensor decompositions proposed in [1]. In this framework it is possible to treat 3-D tensors …
Study reveals efficient recovery of multi-modal signals via Bayesian methods and sequential learning.
problem Recovering multiple high-dimensional signals from correlated modalities.
method Bayesian Approximate Message Passing and Sequential Curriculum Learning.
result Sequential learning strategy optimally recovers weak signals in multi-modal settings.
Tensor networks improve medical image classification performance.
problem Improving medical image classification accuracy.
method Extending tensor networks to medical image analysis, focusing on 2D images.
result Tensor networks achieve comparable performance to deep learning methods with fewer hyperparameters and resources.
No free lunch theorem formalized for tensor network models.
problem Understanding limitations of tensor network machine learning models.
method Formalized rigorous no-free-lunch theorem for specific tensor network models.
result Revealed intrinsic limitations of tensor network-based learning models.
Low-rank structure emerges in neural networks during learning.
problem Understanding the evolution of synaptic connectivity over learning.
method Investigated the rank of 3-tensor formed by weight matrices throughout learning.
result Inferred weights are low-tensor-rank and evolve in a fixed low-dimensional subspace.
Tensor networks improve unsupervised learning performance.
problem Improving unsupervised machine learning models.
method Autoregressive Matrix Product States (AMPS) combining quantum and machine learning.
result AMPS significantly outperforms existing tensor network models and neural networks.
Tensor trains speed up option pricing for multi-asset options.
problem Speeding up option pricing for multi-asset options.
method Tensor train learning algorithms to compress functions with parameter dependence.
result The proposed method outperforms Monte Carlo-based pricing in computational complexity.
Proposes a new tensor factorization model for better link prediction in knowledge graphs.
problem Lack of information in treating missing and non-existing relations equally in tensor factorization models.
method Introduces a binary tensor factorization model with probit link to address the issue.
result Shows improved prediction accuracy and interpretability compared to existing models.
Paper learns meaningful state and action representations from MDP trajectories.
problem Learning good state and action representations from MDP trajectories.
method Tensor decomposition, kernelization, importance sampling, low-Tucker-rank approximation.
result The learned state/action abstractions provide accurate approximations to latent block structures.
In recent years, a class of dictionaries have been proposed for multidimensional (tensor) data representation that exploit the structure of tensor data by imposing a Kronecker structure on the dictionary underlying the data. In this work, a novel algorithm called "STARK" is provided to learn Kronecker structured dictio…
We present a novel analysis of the dynamics of tensor power iterations in the overcomplete regime where the tensor CP rank is larger than the input dimension. Finding the CP decomposition of an overcomplete tensor is NP-hard in general. We consider the case where the tensor components are randomly drawn, and show that …
Biological neurons learn tensor decompositions of higher-order correlations using nonlinear Hebbian plasticity.
problem Learning higher-order correlations in biological neurons.
method Introduce and study generalized nonlinear Hebbian learning rules.
result Neurons can learn tensor eigenvectors of higher-order input correlation tensors.
A new tensor ring mixture model improves density estimation efficiency.
problem Efficient probability density estimation in statistical machine learning.
method Tensor ring decomposition with mixture model for adaptive weights.
result Enhanced expressive capability and flexibility in density estimation.
Improved image learning using elliptically contoured tensor-variate distributions.
problem Inadequate statistical analysis for tensor-valued data, especially with heavier or lighter tails.
method Developed a family of elliptically contoured tensor-variate distributions and derived their properties and procedures for estimation.
result Tensor-variate classification rules and tensor-on-tensor regression better predict and characterize data than TVN-based methods.
Deep tensor factorization benefits from implicit regularization with polynomial growth.
problem Tensor factorization's implicit regularization effect in deep networks is not well understood.
method Investigated the implicit regularization in deep tensor factorization, showing polynomial growth.
result Implicit regularization in deep tensor factorization grows polynomially with depth, improving estimation accuracy and convergence.
Tensor factorizations have become increasingly popular approaches for various learning tasks on structured data. In this work, we extend the RESCAL tensor factorization, which has shown state-of-the-art results for multi-relational learning, to account for the binary nature of adjacency tensors. We study the improvemen…
Paper optimizes tensor deflation for non-orthogonal signals.
problem Recovering low-rank signals from noisy tensors with correlated components.
method Developed an asymptotic analysis and optimized deflation procedure using random tensor theory.
result Proposed an efficient tensor deflation algorithm that optimizes a parameter introduced in the deflation mechanism.
There has been growing interest in extending traditional vector-based machine learning techniques to their tensor forms. An example is the support tensor machine (STM) that utilizes a rank-one tensor to capture the data structure, thereby alleviating the overfitting and curse of dimensionality problems in the conventio…
Low rank tensor learning, such as tensor completion and multilinear multitask learning, has received much attention in recent years. In this paper, we propose higher order matching pursuit for low rank tensor learning problems with a convex or a nonconvex cost function, which is a generalization of the matching pursuit…
New method uses tensor decompositions to overcome the curse of dimensionality for large-scale learning.
problem Large-scale machine learning problems with kernel methods.
method Deterministic Fourier features combined with low-rank tensor decomposition for tensor product structure.
result Demonstrated consistent performance and superior results compared to random Fourier features.
New model uses PEPS for image classification, outperforming tree-like networks.
problem Efficiently modeling and classifying 2D data like images.
method Feature map followed by PEPS contraction with trainable parameters.
result Significantly superior to tree-like networks on MNIST and Fashion-MNIST.