Low-bit training framework reduces energy consumption in CNNs.
problem Reducing energy consumption in convolutional neural networks.
method Low-bit training framework using MLS tensor format with dynamic quantization.
result Achieves superior trade-off between accuracy and bit-width.
Extends geometrical description of tensor manifolds in tree-based formats.
problem Geometrical description of tensor manifolds in tree-based formats.
method Provided a new geometrical description of manifolds of tensors in tree-based format.
result Geometrical description compatible with Tucker format.
The main goal of this paper is to study the geometric structures associated with the representation of tensors in subspace based formats. To do this we use a property of the so-called minimal subspaces which allows us to describe the tensor representation by means of a rooted tree. By using the tree structure and the d…
A guide to AI+ML for portfolio weight formation.
problem Optimizing portfolio weights using AI and ML techniques.
method Analysis of machine learning tools and their performance in portfolio weight formation.
result Nodewise regression with Global Minimum Variance portfolio weights deliver high Sharpe Ratios and returns.
The ML-Schema, proposed by the W3C Machine Learning Schema Community Group, is a top-level ontology that provides a set of classes, properties, and restrictions for representing and interchanging information on machine learning algorithms, datasets, and experiments. It can be easily extended and specialized and it is a…
Estimates high-dimensional distributions using tree tensor networks.
problem Estimating high-dimensional probability distributions from i.i.d. samples.
method Tree-based tensor formats, empirical risk minimization, L2 contrast, orthogonal bases.
result Effective approximation of classical probabilistic models like Gaussian and graphical models.
Tensorized random projections reduce high-dimensional tensor size efficiently.
problem Efficiently reducing the dimension of very high-dimensional tensors.
method Proposes two tensorized random projection maps using TT and CP decompositions.
result TT format offers superior performance in terms of required random projection size.
Bayesian tensor train method recovers streaming data with high accuracy.
problem Recovering high-order, incomplete, and noisy streaming data.
method Bayesian tensor train decomposition using streaming variational Bayes method.
result The proposed SPTT algorithm excels in recovering streaming data compared to state-of-the-art methods.
E2M optimizes tensor density estimation by relaxing α-divergence to KL-divergence.
problem Analytical challenges in traditional α-divergence optimization for tensor-based density estimation. method E2M algorithm: relaxes optimization to KL-divergence, then applies tensor many-body approximation. result Flexible modeling of various low-rank structures and their mixtures.
Tensorized Rademacher projections outperform Gaussian projections in reducing tensor dimensions.
problem Reducing the dimension of high-dimensional tensors for machine learning.
method Tensorized Rademacher random projections using Tensor Train decomposition.
result Tensorized Rademacher projections can replace Gaussian projections in tensor compression.
Efficiently price high-dimensional Bermudan options using tensor compression.
problem High-dimensional option pricing with computational complexity.
method Hierarchical tensor compression for Monte Carlo and dual martingale methods.
result Tensor compression alleviates the curse of dimensionality for Bermudan option pricing.
Residual neural networks improve collision prediction in planetary simulations.
problem Accurate prediction of planetary collisions in N-body simulations.
method Residual neural networks trained on collision data.
result Residual neural networks outperform existing methods in prediction accuracy and generalization.
Modeling interactions between features improves the performance of machine learning solutions in many domains (e.g. recommender systems or sentiment analysis). In this paper, we introduce Exponential Machines (ExM), a predictor that models all interactions of every order. The key idea is to represent an exponentially l…
MEI model improves knowledge graph completion by efficiently modeling interactions between embeddings.
problem Efficiently modeling interactions between knowledge graph embeddings to predict missing links.
method MEI divides embeddings into partitions and uses Tucker and block term formats to model interactions efficiently.
result Achieves state-of-the-art performance on link prediction tasks.
Extends multivariate regression for tensor-variate data, identifying brain regions and facial characteristics.
problem Challenges in fitting regression models with multivariate responses and covariates.
method Low-rank tensor formats on regression coefficients and tensor-variate normal distribution for errors.
result Maximum likelihood estimators for tensor-on-tensor regression via block-relaxation algorithms.
Tensor trains simplify solving complex PDEs efficiently.
problem Solving high-dimensional parabolic PDEs using traditional methods is computationally infeasible.
method Reformulate PDEs as backward stochastic differential equations and use tensor train format for compression and efficient computation.
result Tensor train methods achieve a good balance between accuracy and computational efficiency.
With few exceptions, the field of Machine Learning (ML) research has largely ignored the browser as a computational engine. Beyond an educational resource for ML, the browser has vast potential to not only improve the state-of-the-art in ML research, but also, inexpensively and on a massive scale, to bring sophisticate…
An important and natural question in the analysis of Ricci flow singularity formation in dimensions four and above is as follows: What are the weakest conditions that provide control of the norm of the Riemann curvature tensor? In this short note, we show that on a compact manifold, the trace-free Ricci tensor is contr…
Recent years have seen rapid advances in the data-driven analysis of dynamical systems based on Koopman operator theory and related approaches. On the other hand, low-rank tensor product approximations -- in particular the tensor train (TT) format -- have become a valuable tool for the solution of large-scale problems …
Graph neural networks predict solid-state NMR parameters from atomic structures.
problem Efficiently predicting NMR parameters from atomic structures for complex materials.
method Graph neural networks applied to tensor quantities for anisotropic magnetic shielding and electric field gradient.
result Improved accuracy in predicting NMR properties from diverse and complex materials.
New approach uses random matrix theory to understand tensor estimation performance.
problem Understanding the performance of estimators for low-rank signals in noisy tensors.
method Developed a new approach using random matrix theory to study random tensors.
result Discovered a fixed-point equation that matches the performance of the maximum likelihood estimator.
Guaranteed convergence for tensor factorization using Riemannian gradient descent.
problem Recovering tensor train format from linear measurements.
method Optimization over left-orthogonal TT format using Riemannian gradient descent on Stiefel manifold.
result RGD converges linearly to the ground-truth tensor with polynomial error growth in tensor order.
Deep neural networks are commonly developed and trained in 32-bit floating point format. Significant gains in performance and energy efficiency could be realized by training and inference in numerical formats optimized for deep learning. Despite advances in limited precision inference in recent years, training of neura…
Paper proposes efficient tensor completion method using Gaussian Process.
problem Tensor completion in high-dimensional data with unknown smooth functions.
method Gaussian Process Regression for initialization and TT-cross approximation for tensor rank selection.
result Improved reconstruction error compared to random initialization.
This paper presents the first comprehensive empirical study demonstrating the efficacy of the Brain Floating Point (BFLOAT16) half-precision format for Deep Learning training across image classification, speech recognition, language modeling, generative networks and industrial recommendation systems. BFLOAT16 is attrac…
Optimizes neural network training by dynamically updating Tucker decomposition ranks.
problem Redundant parameters in neural network architectures.
method Geometry-aware training of factorized layers in tensor Tucker format.
result Optimal locally approximating the original dynamics without initial rank knowledge.
The interest in machine learning with tensor networks has been growing rapidly in recent years. We show that tensor-based methods developed for learning the governing equations of dynamical systems from data can, in the same way, be used for supervised learning problems and propose two novel approaches for image classi…
Tribuo integrates ML models with provenance in Java.
problem Tracking and ensuring ML models behave appropriately as they grow in number and complexity.
method Integrates model training, inference, type-safety, runtime checking, and automatic provenance recording into a single Java library.
result Automatic recording of full processing pipeline and training details inside the model object.
Spin-opstrings from QMC simulations enable ML of quantum phases.
problem Capturing and predicting quantum phase transitions using ML.
method Spin-opstrings derived from QMC simulations used as ML input.
result Spin-opstrings accurately predict quantum phase transitions.
Efficiently samples complex distributions using tensor train format.
problem Sampling from high-dimensional complex probability densities efficiently.
method Integrates tensor train format with backward stochastic differential equations (BSDEs) for fast, robust, and accurate sampling.
result Improved efficiency in sampling from challenging target distributions.
Study uses AI and ML to predict and optimize corrosion resistance of aluminum alloys.
problem Corrosion resistance of aluminum alloys in marine environments.
method Investigated two ML approaches: direct and inverse, using Random Forest, neural network, and Gaussian Process Regression.
result Gaussian Process Regression with hybrid kernel functions provided superior predictive performance.
SurvSet offers a repository of 76 T2E datasets for ML benchmarking.
problem Lack of open-source T2E dataset repositories for ML benchmarking.
method Consistent formatting of datasets for various ML algorithms and statistical methods.
result SurvSet provides a comprehensive resource for T2E analysis.
Analysis of reactive-diffusion simulations requires a large number of independent model runs. For each high-fidelity simulation, inputs are varied and the predicted mixing behavior is represented by changes in species concentration. It is then required to discern how the model inputs impact the mixing process. This tas…
Paper compares ML models for a wall-following robot, achieving high accuracy.
problem Improving prediction accuracy of a wall-following robot's direction.
method Trained various machine learning models on a dataset of ultrasound sensor readings.
result Presented machine learning models with higher accuracy than previous work.
This paper is concerned with the approximation of high-dimensional functions in a statistical learning setting, by empirical risk minimization over model classes of functions in tree-based tensor format. These are particular classes of rank-structured functions that can be seen as deep neural networks with a sparse arc…
A substantial progress in development of new and efficient tensor factorization techniques has led to an extensive research of their applicability in recommender systems field. Tensor-based recommender models push the boundaries of traditional collaborative filtering techniques by taking into account a multifaceted nat…
Machine learning predicts obesity causes using genetic and imaging data.
problem Predicting causes of obesity in children and adults.
method Use ML techniques like decision trees, SVM, RF, GBM, LASSO, BN, and ANN on genetic and imaging data.
result ML models accurately predict obesity causes and chronic diseases.
Today's telecommunication networks have become sources of enormous amounts of widely heterogeneous data. This information can be retrieved from network traffic traces, network alarms, signal quality indicators, users' behavioral data, etc. Advanced mathematical tools are required to extract meaningful information from …
Machine learning accelerates Lie algebra computations.
problem Computing tensor products and branching rules of Lie algebras.
method Machine learning for Lie algebra computations.
result Achieves significant speed-ups in Lie algebra computations.
TTRP method preserves distances in high-dimensional data with reduced storage and speed.
problem Preserving distances in high-dimensional datasets efficiently and accurately.
method Tensor train random projection (TTRP) using TT-ranks of one.
result TTRP is an expected isometric projection with bounded variance.
The main goal of this paper is to extend the so-called Dirac-Frenkel Variational Principle in the framework of tensor Banach spaces. To this end we observe that a tensor product of normed spaces can be described as a union of disjoint connected components. Then we show that each of these connected components, composed …
GrateTile optimizes CNN feature map storage for efficient data access.
problem Efficient storage and access of sparse CNN feature maps.
method Divides feature maps into uneven-sized subtensors, compresses and stores them in a compressed yet accessible format.
result Average 55% DRAM bandwidth reduction with minimal indexing overhead.
SimTensor is a multi-platform, open-source software for generating artificial tensor data (either with CP/PARAFAC or Tucker structure) for reproducible research on tensor factorization algorithms. SimTensor is a stand-alone application based on MATALB. It provides a wide range of facilities for generating tensor data w…
CliMB-DC combines human guidance and data-centric tools to improve ML for non-technical experts.
problem Lack of data-centric handling in LLM co-pilots for non-technical users.
method Human-guided, data-centric framework combining advanced tools and LLM reasoning.
result Significantly outperforms existing co-pilot baselines for data-centric challenges.
Causal deep learning tackles causal inference using tensor factor analysis.
problem Addressing causal questions in data using neural networks.
method Tensor factor analysis and neural network architectures (causal capsules, tensor transformer, multilinear projection algorithm).
result Derives deep neural networks for causal inference with tensor factor analysis.
Develops a new feature theory for robust machine learning.
problem Creating robust machine learning features from training data.
method Stochastic tensor space feature theory with Karhunen-Loeve expansion and hierarchical subspaces.
result Dramatic increases in accuracy for predicting Alzheimer's disease stages.
Expected signatures map data streams to lower dimensions, improving ML performance.
problem Leveraging model-free embeddings for domain-agnostic machine learning.
method Expected signatures map data streams to lower dimensions, with convergence results bridging empirical and theoretical estimators.
result A modified expected signature estimator with lower mean squared error for martingale processes.
By restricting the iterate on a nonlinear manifold, the recently proposed Riemannian optimization methods prove to be both efficient and effective in low rank tensor completion problems. However, existing methods fail to exploit the easily accessible side information, due to their format mismatch. Consequently, there i…