T-Basis represents neural network tensors with fewer parameters.
problem Efficiently representing neural network tensors with fewer parameters.
method T-Basis uses Tensor Rings to represent tensors in a neural network, parameterizing them with a small number of coefficients.
result T-Basis achieves high compression rates with minimal performance loss.
Characterizes a specific type of neural network for alternating group equivariance.
problem Understanding and characterizing neural networks with alternating group equivariance.
method Characterization of all possible An-equivariant neural networks using tensor powers of Rn. result Found a basis of matrices for learnable, linear An-equivariant layer functions. Characterizes group-equivariant neural networks for three groups.
problem Understanding equivariant neural networks for orthogonal, special orthogonal, and symplectic groups.
method Characterized all possible group-equivariant neural networks for three groups.
result Found spanning sets of matrices for learnable, linear equivariant layer functions.
New basis for permutation equivariant layers reduces computation costs.
problem Efficiently computing permutation equivariant layers in neural networks.
method Generalized partition algebra basis with low-rank tensors.
result Low-rank tensors enable faster computation compared to orbit basis.
Unified tensor network formalism for combining neural and symbolic AI.
problem Combining neural and symbolic AI approaches remains a challenge.
method Introduces a tensor network formalism capturing sparsity principles.
result Unified treatment identifies tensor network contractions as a fundamental inference class.
A neural network models pressure-Hessian from local velocity gradients in turbulent flows.
problem Modeling the pressure-Hessian from local velocity gradients in turbulent flows.
method Tensor basis neural network (TBNN) trained on DNS data.
result Neural network accurately captures key alignment statistics of the pressure-Hessian tensor.
Neural Chaos uses neural networks instead of polynomials for stochastic modeling.
problem Challenges in constructing surrogate models with uncertainty quantification for complex or high-dimensional stochastic processes.
method Adopting spectral expansion formalism with neural network basis functions, identifying them data-drivenly without prior assumptions.
result Demonstrates effectiveness of the proposed scheme through numerical examples of varying complexity.
Tensor Neural Networks improve pricing accuracy for interest rate derivatives.
problem Inaccurate pricing of Bermudan Swaptions using traditional methods.
method Leveraging Tensor Neural Networks to solve backward Stochastic Differential Equations.
result Tensor Neural Networks provide more accurate and robust prices than Dense Neural Networks.
Tensor Neural Networks improve regression accuracy and efficiency.
problem Nonparametric regression problems with complex, high-dimensional functions.
method Integrates statistical regression and numerical integration within a tensor neural network framework.
result Superior performance in approximation accuracy and generalization capacity compared to FFNs and RBNs.
Proposes ANOVA-TPNN for stable interpretation of complex functions.
problem Stability issues in estimating components of functional ANOVA models.
method Introduces ANOVA-TPNN based on tensor product basis expansion.
result ANOVA-TPNN provides stable estimation of components.
Bayesian-TPNN improves ANOVA-TPNN for detecting higher-order components.
problem Difficulty in incorporating higher-order components in ANOVA-TPNN due to computational and memory constraints.
method Bayesian inference procedure for functional ANOVA model with TPNN basis functions.
result Bayesian-TPNN detects higher-order components with reduced computational cost.
Paper projects GP basis functions using tensor networks to reduce complexity.
problem Efficiently approximating Gaussian process regression with a large number of basis functions.
method Develops a method using tensor networks to approximate GP regression with an exponential number of basis functions without exponential computational complexity.
result Shows efficient GP regression on an 18-dimensional benchmark data set.
Tensor logic aims to unify AI types with scalable and transparent features.
problem Lack of a unified AI programming language with scalability and transparency.
method Introduces tensor logic, a new AI language based on tensor equations.
result Tensor logic enables key AI forms like transformers, formal reasoning, and graphical models.
New neural networks learn graph symmetries.
problem Learning from graph data without considering vertex relations.
method Constructs equivariant neural networks to Aut(G) group.
result Characterizes learnable, linear, Aut(G)-equivariant functions.
Deep model integrates MRI and DTI for autism severity prediction.
problem Predicting spectrum-level deficits in autism using multimodal brain imaging.
method Generative deep-learning framework combining rs-fMRI and DTI data.
result Hybrid model outperforms existing methods in predicting autism severity.
Defines tensor eigenvalues and singular values without basis, simplifying analysis.
problem Defines tensor eigenvalues and singular values without basis.
method Intrinsic definition of tensor eigenvalues and singular values using concepts from pure mathematics.
result Shows the relationship between tensor analysis and pure mathematics.
Improves deep neural network training and accuracy with adaptive basis approach.
problem Gap between theoretical and practical performance of deep neural networks.
method Adaptive basis viewpoint, novel initializations, hybrid optimizer.
result Dramatic increases in accuracy and convergence rate for various DNN applications.
Framework integrates brain connectivity data for clinical predictions.
problem Predicting clinical outcomes from brain connectivity data.
method Structurally-regularized Dynamic Dictionary Learning (sr-DDL) and LSTM-ANN block.
result Framework outperforms state-of-the-art approaches in clinical outcome prediction.
New algorithms for interpreting complex multivariate functions.
problem Hard interpretation of multivariate functions due to many parameters.
method Filtered tensor decompositions of derivative information.
result Nonparametric estimates of smooth decoupled functions.
TRNN combines tensor geometry with neural network nonlinearity for HD data.
problem Modeling high-dimensional data with preserved tensor geometry and nonlinear interactions.
method Introduces TRNN that integrates tensor geometry and neural network nonlinearity.
result TRNN preserves tensor geometry while offering nonlinearity.
Tensor decomposition is an effective approach to compress over-parameterized neural networks and to enable their deployment on resource-constrained hardware platforms. However, directly applying tensor compression in the training process is a challenging task due to the difficulty of choosing a proper tensor rank. In o…
Adaptive neural networks learn functional data bases for improved performance.
problem Applying deep learning to functional data is challenging due to high dimensionality.
method Proposes adaptive neural networks with Basis Layers that learn relevant basis functions.
result Empirically outperforms other neural network approaches across various tasks.
We propose a new method for learning deep neural network models that is based on a greedy learning approach: we add one basis function at a time, and a new basis function is generated as a non-linear activation function applied to a linear combination of the previous basis functions. Such a method (growing deep neural …
New neural network models for functional data.
problem Handling non-linear functional data.
method Functional Direct Neural Network (FDNN) and Functional Basis Neural Network (FBNN) with gradient-based optimization.
result Demonstrated effectiveness in complex functional models.
Quantum neural network and tensor network models outperform classical models in Japanese stock market predictions.
problem Improving stock return predictions using quantum and quantum-inspired machine learning.
method Evaluation of quantum neural network and tensor network models against classical models like linear and neural networks.
result Tensor network model outperforms classical models in Japanese stock market, including linear and neural network models.
We show that a basis of a semisimple Lie algebra of compact type, for which any diagonal left-invariant metric has a diagonal Ricci tensor, is characterized by the Lie algebraic condition of being "nice". Namely, the bracket of any two basis elements is a multiple of another basis element. This extends the work of Laur…
Novel Bayesian prior for neural networks encodes amplitude and lengthscale.
problem Lack of user-friendly priors for specifying basic properties in Bayesian neural networks.
method Introduced Poisson Process Radial Basis Function Networks (PP-RBFN) as a novel prior.
result PP-RBFN allows decoupled specification of amplitude and lengthscale, and estimated function is consistent.
The paper analyzes implicit regularization in tensor factorization using neural networks.
problem Understanding implicit regularization in tensor factorization.
method Dynamical systems perspective and gradient descent analysis.
result Gradient descent induces a form of greedy low tensor rank search.
Tensor regression networks achieve high compression rate of neural networks while having slight impact on performances. They do so by imposing low tensor rank structure on the weight matrices of fully connected layers. In recent years, tensor regression networks have been investigated from the perspective of their comp…
New model for network analysis using functional data.
problem Existing network models treat nodes as functions, but this paper introduces functional edges.
method Transform adjacency matrix into functional adjacency tensor, apply Tucker decomposition, regularize basis matrices, and solve tensor completion problem.
result The model effectively captures community structure and handles irregular functional edge data.
This paper develops a method to train compact neural networks with reduced memory and computational costs.
problem Training large neural networks consumes excessive resources and energy.
method End-to-end training framework using Bayesian tensor decomposition with automatic rank determination.
result The method achieves significant parameter reduction and maintains or improves accuracy.
Causal deep learning tackles causal inference using tensor factor analysis.
problem Addressing causal questions in data using neural networks.
method Tensor factor analysis and neural network architectures (causal capsules, tensor transformer, multilinear projection algorithm).
result Derives deep neural networks for causal inference with tensor factor analysis.
Deep neural network predicts molecular wave functions in minimal basis.
problem Improving accuracy and efficiency in quantum chemistry calculations.
method Adapted SchNet for Orbitals (SchNOrb) model in quasi-atomic minimal basis.
result Model accurately predicts molecular orbital energies and wavefunctions for large molecules.
RI-DeepONet learns neural operators from arbitrary sensor data.
problem Discretization of input functions limits practical applications of DeepONet.
method Introduces RI-DeepONet and two dictionary learning algorithms for INRs.
result RINO handles arbitrary sensor data robustly and applies to various problems.
Advances understanding of neural network generalization via tensor analysis.
problem Understanding the generalizability of deep neural networks.
method Tensor analysis to measure compressibility and generalizability.
result Proposed generalization bound outperforms previous methods, especially for tensor-based networks.
Proposes FATTNN for tensor-on-tensor regression with improved prediction and reduced computation.
problem Tensor-on-tensor regression with complex tensor structures and nonlinear relationships.
method Integrates tensor factor models into deep neural networks to handle nonlinearity and reduce data dimensionality.
result Significant improvements in prediction accuracy and computational efficiency over traditional methods.
The paper explains implicit regularization in hierarchical tensor factorization and deep CNNs.
problem Understanding implicit regularization in complex neural network architectures.
method Theoretical analysis using dynamical systems to overcome challenges in hierarchy.
result Established implicit regularization towards low hierarchical tensor rank, equivalent to locality in CNNs.
The paper uses tensor decompositions to improve neural network models for tree data.
problem Encoding structural knowledge from tree-structured data efficiently.
method Introduces new aggregation functions using Canonical and Tensor-Train decompositions.
result Proposed models outperform traditional methods on tree classification tasks.
Tensor neural network improves human pose classification from 3D skeleton data.
problem Efficiently processing spatiotemporal data for human pose classification.
method Proposes a tensor-based neural network with three components: spatiotemporal feature construction, tensor fusion, and tensor-based neural network processing.
result Achieves state-of-the-art performance in human pose classification.
New fusion blocks improve equivariant neural networks for molecular dynamics.
problem Designing equivariant neural networks for tasks with global symmetries.
method Using fusion diagrams from tensor networks to design novel equivariant components.
result Improved performance with fewer parameters on chemical problems.
New neural networks respect symmetries in symmetric tensors, improving efficiency and generalization.
problem Learning from symmetric tensors efficiently and respecting their inherent symmetries.
method Developed two characterizations of linear permutation equivariant functions between symmetric power spaces of R^n.
result These functions are highly data efficient compared to standard MLPs and generalize well to different sizes of symmetric tensors.
Introduces P-tensors for generalized higher-order message passing in graph neural networks.
problem Expanding the expressive power of graph neural networks through higher-order structures.
method Introduces P-tensors to define the most general form of permutation equivariant message passing.
result Achieves state-of-the-art performance on molecular datasets.
Neural networks can approximate functionals on RKHS with error bounds.
problem Approximating functionals on RKHS using neural networks.
method Interpolating orthogonal projections in RKHS using point evaluations.
result Explicit error bounds for various kernels (inverse multiquadric, Gaussian, Sobolev).
New tensor formulation reveals gradient flow's bias in linear neural networks.
problem Understanding implicit bias in linear neural network training.
method Tensor formulation of neural networks, including fully-connected, diagonal, and convolutional networks.
result Gradient flow on linear tensor networks converges to solutions of specific optimization problems.
This paper deals with the problem of describing the vector spaces of divergence-free, natural tensors on a pseudo-Riemannian manifold that are second-order; i.e., that are defined using only second derivatives of the metric. The main result establishes isomorphisms between these spaces and certain spaces of tensors (at…
This paper shows how infinitely wide Tensor Networks converge to Gaussian Processes.
problem Understanding the relationship between Tensor Networks and Gaussian Processes.
method Analyzing the infinite-width limit of Tensor Networks and comparing them to Gaussian Processes.
result Infinitely wide Tensor Networks converge to Gaussian Processes, proving their equivalence.
Recently, path norm was proposed as a new capacity measure for neural networks with Rectified Linear Unit (ReLU) activation function, which takes the rescaling-invariant property of ReLU into account. It has been shown that the generalization error bound in terms of the path norm explains the empirical generalization b…
MARS automatically selects tensor decomposition ranks, improving performance in neural network tasks.
problem Determining optimal decomposition ranks in tensor decompositions.
method MARS uses binary masks to learn optimal tensor structure during training via relaxed MAP estimation.
result MARS achieves better results than previous methods in various tasks.