Stochastic trace estimation with tensor train random vectors
problem Stochastic trace estimation for large-scale matrices
method Gaussian random tensor train vectors
result Median-of-means variant achieves dimension-independent guarantees
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Tensor train”
This paper studies how key tensor properties are inherited in subtensors of tensor train decompositions.
problem Theoretical development of property inheritance for subtensors in tensor train decompositions.
method Theoretical analysis of incoherence and condition number preservation, and tensor train rank preservation through fiber-wise sampling.
result Key tensor properties (incoherence and condition number) can be well preserved to subtensors formed via fiber-wise sampling.
Proposes a new method for high-dimensional density estimation.
problem Estimating high-dimensional probability density functions efficiently.
method Tensorizing flow method combining tensor-train and flow-based generative modeling.
result Efficiently constructs an approximate density in tensor-train form and trains a flow model to match empirical distribution.
Tensor trains speed up option pricing for multi-asset options.
problem Speeding up option pricing for multi-asset options.
method Tensor train learning algorithms to compress functions with parameter dependence.
result The proposed method outperforms Monte Carlo-based pricing in computational complexity.
This paper develops a method to train compact neural networks with reduced memory and computational costs.
problem Training large neural networks consumes excessive resources and energy.
method End-to-end training framework using Bayesian tensor decomposition with automatic rank determination.
result The method achieves significant parameter reduction and maintains or improves accuracy.
There has been growing interest in extending traditional vector-based machine learning techniques to their tensor forms. An example is the support tensor machine (STM) that utilizes a rank-one tensor to capture the data structure, thereby alleviating the overfitting and curse of dimensionality problems in the conventio…
We introduce a new parameterization method for deep learning layers using spectral tensor train decomposition.
problem Efficiency and stability in deep learning models with weight matrix compression.
method Spectral Tensor Train Parameterization (STTP) of weight matrices.
result Improved compression and training stability in neural networks.
KReTTaH uses tensor trains and Hadamard overparameterization for fast, interpretable multi-way data imputation.
problem Multi-way data imputation for high-dimensional functional MRI and dynamic graph recovery.
method Reformulates imputation as RKHS regression with TT-constrained coefficients and Hadamard overparameterization. Optimizes TT coefficients and kernel matrices on Riemannian manifolds.
result Consistently outperforms state-of-the-art methods in modeling accuracy.
KReTTaH uses tensor trains and Hadamard overparameterization for fast, interpretable multi-way data imputation.
problem Multi-way data imputation in high-dimensional spaces.
method Reformulates imputation as RKHS regression with TT-constrained coefficients, optimized on manifold frameworks.
result Consistently outperforms state-of-the-art methods in accuracy.
New method compresses LSTM networks using MPS tensor trains.
problem Challenges in maintaining performance of compressed RNNs.
method Use of MPS tensor trains for LSTM network compression.
result MPS tensor trains outperform MPOs in storage and inference time.
Gradient descent can find better tensor decompositions than lazy training in over-parameterized settings.
problem Finding better tensor decompositions in over-parameterized settings.
method Gradient descent on over-parameterized tensor decomposition problems.
result Gradient descent can find an approximate tensor decomposition with rank , while lazy training requires .
Bayesian tensor train method recovers streaming data with high accuracy.
problem Recovering high-order, incomplete, and noisy streaming data.
method Bayesian tensor train decomposition using streaming variational Bayes method.
result The proposed SPTT algorithm excels in recovering streaming data compared to state-of-the-art methods.
Bayesian tensor train kernel machine uses Laplace approximation for scalable GP regression.
problem Scalability limitations of Gaussian process regression.
method Bayesian tensor train kernel machine with Laplace approximation and variational inference.
result VI replaces cross-validation and offers up to 65x faster training.
The paper shows how gradient flow on over-parametrized tensor decomposition behaves like deflation.
problem Understanding the training dynamics of gradient flow on tensor decomposition.
method Empirical observation and mathematical proof of gradient flow dynamics for orthogonally decomposable tensors.
result Gradient flow dynamics for orthogonally decomposable tensors follows a tensor deflation process, recovering all tensor components.
Tensor decomposition is an effective approach to compress over-parameterized neural networks and to enable their deployment on resource-constrained hardware platforms. However, directly applying tensor compression in the training process is a challenging task due to the difficulty of choosing a proper tensor rank. In o…
We propose gradient adversarial training, an auxiliary deep learning framework applicable to different machine learning problems. In gradient adversarial training, we leverage a prior belief that in many contexts, simultaneous gradient updates should be statistically indistinguishable from each other. We enforce this c…
New model for density estimation using tensor trains.
problem Estimation of high-dimensional probability density functions.
method Tensor train-based density estimation (TTDE) with Riemannian optimization.
result TTDE outperforms competitors in training speed and performance.
In this paper, we propose a Tensor Train Neighborhood Preserving Embedding (TTNPE) to embed multi-dimensional tensor data into low dimensional tensor subspace. Novel approaches to solve the optimization problem in TTNPE are proposed. For this embedding, we evaluate novel trade-off gain among classification, computation…
TTRP method preserves distances in high-dimensional data with reduced storage and speed.
problem Preserving distances in high-dimensional datasets efficiently and accurately.
method Tensor train random projection (TTRP) using TT-ranks of one.
result TTRP is an expected isometric projection with bounded variance.
Paper generalizes tensor-train approximation for complex random variables.
problem Characterizing intractable high-dimensional random variables.
method Extends inverse Rosenblatt transform to general reference measures and integrates into deep variable transformation framework.
result Deep inverse Rosenblatt transport significantly expands tensor approximations for complex random variables.
Adaptive algorithm learns tensor network structures from data.
problem Identifying optimal tensor network structure from data.
method Greedy approach starting from rank one tensor, small rank increments.
result Adaptive algorithm identifies efficient tensor network structures.
The paper uses tensor decompositions to improve neural network models for tree data.
problem Encoding structural knowledge from tree-structured data efficiently.
method Introduces new aggregation functions using Canonical and Tensor-Train decompositions.
result Proposed models outperform traditional methods on tree classification tasks.
Tensor network surrogate for efficient option pricing in large portfolios.
problem Large-scale portfolio revaluation problems in market risk management.
method Tensor-train (TT) approximation for high-dimensional price surfaces, direct inference using Laplacian kernel and TT representations.
result Tensor surrogate achieves lower test error and faster evaluation times compared to standard GPR.
Tensor trains simplify solving complex PDEs efficiently.
problem Solving high-dimensional parabolic PDEs using traditional methods is computationally infeasible.
method Reformulate PDEs as backward stochastic differential equations and use tensor train format for compression and efficient computation.
result Tensor train methods achieve a good balance between accuracy and computational efficiency.
We propose a novel technique for faster deep neural network training which systematically applies sample-based approximation to the constituent tensor operations, i.e., matrix multiplications and convolutions. We introduce new sampling techniques, study their theoretical properties, and prove that they provide the same…
An increasing amount of collected data are high-dimensional multi-way arrays (tensors), and it is crucial for efficient learning algorithms to exploit this tensorial structure as much as possible. The ever-present curse of dimensionality for high dimensional data and the loss of structure when vectorizing the data moti…
Locality preserving projections (LPP) are a classical dimensionality reduction method based on data graph information. However, LPP is still responsive to extreme outliers. LPP aiming for vectorial data may undermine data structural information when it is applied to multidimensional data. Besides, it assumes the dimens…
A new method estimates rare events using tensor trains.
problem Estimating rare event probabilities in high-dimensional problems.
method Approximating optimal importance distribution via tensor-train decompositions and compositions.
result Better variance reduction and efficient computation of rare event probabilities.
We propose a restricted class of tensor network state, built from number-state preserving tensors, for supervised learning tasks. This class of tensor network is argued to be a natural choice for classifiers as (i) they map classical data to classical data, and thus preserve the interpretability of data under tensor tr…
Introduces TT-NF for more compact neural field representations.
problem Finding more compact and easy-to-fit neural field representations.
method Tensor Train parameterization trained with backpropagation.
result Low-rank compression improves downstream task quality metrics.
New method for tensor completion from specific mode observations.
problem Recovering multiway data tensors from partial observations.
method Tensor train decomposition for fiber-wise observations.
result Deterministic recovery guarantees for specific observation patterns.
Tensor train (TT) decomposition provides a space-efficient representation for higher-order tensors. Despite its advantage, we face two crucial limitations when we apply the TT decomposition to machine learning problems: the lack of statistical theory and of scalable algorithms. In this paper, we address the limitations…
Low-bit training framework reduces energy consumption in CNNs.
problem Reducing energy consumption in convolutional neural networks.
method Low-bit training framework using MLS tensor format with dynamic quantization.
result Achieves superior trade-off between accuracy and bit-width.
A new method computes Greeks for multi-asset options using tensor trains and Fourier transforms.
problem Efficient computation of Greeks for multi-asset options with high accuracy and low sample complexity.
method Tensor train (TT) representations of Fourier-based pricing functions, combined with numerical differentiation or analytical approaches.
result Significant speed-ups of up to over Monte Carlo simulations while maintaining comparable accuracy.
Exponential Machinesstat.ML
Modeling interactions between features improves the performance of machine learning solutions in many domains (e.g. recommender systems or sentiment analysis). In this paper, we introduce Exponential Machines (ExM), a predictor that models all interactions of every order. The key idea is to represent an exponentially l…
Tensor, a multi-dimensional data structure, has been exploited recently in the machine learning community. Traditional machine learning approaches are vector- or matrix-based, and cannot handle tensorial data directly. In this paper, we propose a tensor train (TT)-based kernel technique for the first time, and apply it…
A new RNN architecture reduces model size and improves performance.
problem Overparameterization and resource limitations in RNNs.
method Jointly encodes weight matrices using tensor-train factorization.
result Reduces model size by several orders of magnitude without sacrificing performance.
TensorGuide improves LoRA efficiency and expressivity through joint tensor-train optimization.
problem Limited expressivity and generalization of standard LoRA.
method TensorGuide uses a unified tensor-train structure with controlled Gaussian noise to generate correlated low-rank matrices.
result TensorGuide achieves superior accuracy and scalability with fewer parameters compared to standard LoRA and TT-LoRA.
MARS automatically selects tensor decomposition ranks, improving performance in neural network tasks.
problem Determining optimal decomposition ranks in tensor decompositions.
method MARS uses binary masks to learn optimal tensor structure during training via relaxed MAP estimation.
result MARS achieves better results than previous methods in various tasks.
Optimizes neural network training by dynamically updating Tucker decomposition ranks.
problem Redundant parameters in neural network architectures.
method Geometry-aware training of factorized layers in tensor Tucker format.
result Optimal locally approximating the original dynamics without initial rank knowledge.
Tensor networks constrain kernel machines to Gaussian processes.
problem Speeding up kernel machines with reduced model complexity.
method Proving CPD and TT-constrained models recover Gaussian processes with i.i.d. priors.
result TT-constrained models exhibit more Gaussian process behavior than CPD for the same parameters.
A new algorithm reduces memory usage for deep learning models.
problem Training deep learning models requires significant memory.
method Dynamic Tensor Rematerialization (DTR) is a greedy online algorithm that dynamically plans recomputations.
result DTR achieves comparable performance to optimal static checkpointing with only a small memory budget.
Paper identifies tensor ranks via prior predictive matching, solving system of equations.
problem Determining the latent dimensions (ranks) in tensor factorization models.
method Prior predictive moment matching to transform moment matching conditions into a log-linear system of equations.
result Identifies which tensor models have identifiable ranks and derives rank estimators.
THS-GAN uses tensorizing and high-order pooling for AD diagnosis.
problem Early diagnosis of Alzheimer's Disease (AD) using MRI images.
method Tensorizing a three-player cooperative game framework with high-order pooling for MRI images.
result THS-GAN achieves superior performance in AD diagnosis compared to existing methods.
Tensor networks are efficient representations of high-dimensional tensors which have been very successful for physics and mathematics applications. We demonstrate how algorithms for optimizing such networks can be adapted to supervised learning tasks by using matrix product states (tensor trains) to parameterize models…
Low-rank structure emerges in neural networks during learning.
problem Understanding the evolution of synaptic connectivity over learning.
method Investigated the rank of 3-tensor formed by weight matrices throughout learning.
result Inferred weights are low-tensor-rank and evolve in a fixed low-dimensional subspace.
In this paper, we consider the tensor completion problem representing the solution in the tensor train (TT) format. It is assumed that tensor is high-dimensional, and tensor values are generated by an unknown smooth function. The assumption allows us to develop an efficient initialization scheme based on Gaussian Proce…
Tensorized Rademacher projections outperform Gaussian projections in reducing tensor dimensions.
problem Reducing the dimension of high-dimensional tensors for machine learning.
method Tensorized Rademacher random projections using Tensor Train decomposition.
result Tensorized Rademacher projections can replace Gaussian projections in tensor compression.