Unified algorithm for tensor decomposition supports multiple loss functions and models.
problem Efficient tensor decomposition for various models and loss functions.
method Hierarchical combination of ADMM and MM for optimization.
result Wide-range applications can be solved by the proposed algorithm.
The paper uses tensor decompositions to improve neural network models for tree data.
problem Encoding structural knowledge from tree-structured data efficiently.
method Introduces new aggregation functions using Canonical and Tensor-Train decompositions.
result Proposed models outperform traditional methods on tree classification tasks.
In this paper, we study a polynomial decomposition model that arises in problems of system identification, signal processing and machine learning. We show that this decomposition is a special case of the X-rank decomposition --- a powerful novel concept in algebraic geometry that generalizes the tensor CP decomposition…
The paper proposes and discusses semiorthogonal decompositions for moduli spaces of vector bundles.
problem Decompositions of moduli spaces of vector bundles with fixed determinant of odd degree.
method Semiorthogonal decompositions, Grothendieck ring of varieties, mirror symmetry, graph potentials, Fukaya category.
result Evidence for a conjectural semiorthogonal decomposition of moduli spaces of rank 2 bundles with odd determinant.
Proposes a Gaussian process for Koopman mode decomposition.
problem Estimating Koopman mode decomposition quantities and latent variables.
method Unsupervised Gaussian process for simultaneous estimation.
result Efficient parameter estimation through low-rank approximations.
Paper compresses RNNs using HT decomposition for better performance.
problem Large model sizes of RNNs in sequence analysis.
method Hierarchical Tucker (HT) tensor decomposition for model compression.
result HT-LSTM achieves better compression and accuracy than state-of-the-art methods.
Decomposing tensors into orthogonal factors is a well-known task in statistics, machine learning, and signal processing. We study orthogonal outer product decompositions where the factors in the summands in the decomposition are required to be orthogonal across summands, by relating this orthogonal decomposition to the…
Model predicts short-term Amazon rainforest fires with high accuracy.
problem Accurate short-term forecasting of Amazon rainforest fires is challenging.
method Used Seasonal and Trend decomposition based on Loess combined with multi-month-ahead load forecasting algorithms.
result Proposed decomposition-ensemble models provide more accurate forecasts than other models.
New method models matrix time series using tensor CP-decomposition.
problem Modeling matrix time series with reduced complexity.
method One-pass estimation via generalized eigenanalysis and refined projection.
result Component coefficient vectors estimated consistently with certain rates.
To ensure interpretability of extracted sources in tensor decomposition, we introduce in this paper a dictionary-based tensor canonical polyadic decomposition which enforces one factor to belong exactly to a known dictionary. A new formulation of sparse coding is proposed which enables high dimensional tensors dictiona…
Neural-ANOVA breaks down neural networks into simpler models.
problem Understanding complex neural network decision-making processes.
method Formulates a learning problem to decompose neural networks into lower-order models using ANOVA.
result Demonstrates improved approximation properties compared to other regression methods.
Tensor decomposition recovers Gaussian mixtures from moments.
problem Recovering Gaussian mixture models from datasets.
method Symmetric tensor decomposition of moment tensors built from empirical moments.
result Identifiable tensors with interpolation degree less than half their order.
We analyze bias-variance of margin losses.
problem Understanding model overfitting/underfitting.
method Bias-variance decomposition for strictly convex margin losses.
result Expected risk decomposes into central model risk and data variation.
Matrix factorizations and their extensions to tensor factorizations and decompositions have become prominent techniques for linear and multilinear blind source separation (BSS), especially multiway Independent Component Analysis (ICA), NonnegativeMatrix and Tensor Factorization (NMF/NTF), Smooth Component Analysis (Smo…
Bayesian neural networks (BNNs) with latent variables are probabilistic models which can automatically identify complex stochastic patterns in the data. We describe and study in these models a decomposition of predictive uncertainty into its epistemic and aleatoric components. First, we show how such a decomposition ar…
This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models---including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation---which exploits a certain tensor structure in their low-order observable moments (typically…
Generalizes Hoeffding's decomposition for dependent inputs under mild conditions.
problem Performing global sensitivity analysis on black-box models with dependent inputs.
method Proposes a novel framework based on probability theory, functional analysis, and combinatorics to handle dependencies.
result Any square-integrable, real-valued function of random elements with mild dependence assumptions can be uniquely additively decomposed.
Extends Alòs' formula to Barndorff-Nielsen and Shephard model.
problem Modeling call option prices in a stochastic volatility model.
method Uses Alòs' decomposition formula and Ito's formula for an Ornstein-Uhlenbeck model with infinite jumps.
result First Alòs type decomposition formula for Barndorff-Nielsen and Shephard model.
Researchers propose a new SSL risk decomposition method to evaluate and improve self-supervised learning models.
problem Self-supervised learning evaluation is limited to a single metric, providing little insight into model performance and improvement.
method Proposes an SSL risk decomposition that considers four error components: approximation, representation usability, probe generalization, and encoder generalization.
result Analysis of 169 SSL vision models reveals the main sources of error and provides insights for improving SSL models in specific settings.
Tensor decomposition methods are widely used for model compression and fast inference in convolutional neural networks (CNNs). Although many decompositions are conceivable, only CP decomposition and a few others have been applied in practice, and no extensive comparisons have been made between available methods. Previo…
We provide a unified view of additive explanations for dependent inputs.
problem Challenges in obtaining a tractable representation and estimating the decomposition for dependent inputs.
method Combining Hilbert space methods with generalized functional ANOVA, we build an explicit decomposition Riesz Basis.
result Proposed a simple yet powerful algorithm to estimate the decomposition from data.
New method uses random decompositions for high-dimensional Bayesian optimization.
problem Learning accurate decompositions for high-dimensional black-box functions.
method Data-independent random tree-based decomposition sampling.
result Random decomposition upper-confidence bound algorithm (RDUCB) yields significant empirical gains.
Adaptive tensor modeling preserves continuity in multidimensional data.
problem Discretization of continuous multidimensional data loses important information.
method Functional Tucker decomposition (FTD) with RKHS modeling.
result FTD enables adaptive and expressive tensor modeling.
Study models of Gödel Universe using Lie groups and Iwasawa decomposition.
problem Modeling the Gödel Universe as a Lie group with specific metrics.
method Iwasawa decomposition for semisimple Lie groups, left-invariant Lorentz metric on SL(2,R).
result Isometry between sub-Riemannian Lie groups induced by Iwasawa decomposition.
FunBaT extends Tucker decomposition to handle continuous-indexed tensor data.
problem Handling continuous-indexed tensor data that doesn't fit traditional Tucker decomposition.
method FunBaT treats continuous-indexed data as interactions between a core tensor and a group of latent functions modeled by Gaussian processes (GP). It converts each GP into a state-space prior and uses advanced message-passing techniques for scalable inference.
result FunBaT effectively handles real-world data with continuous indexes, demonstrating its advantage in synthetic and real-world applications.
A new framework for efficient Bayesian network inference.
problem High-dimensional Bayesian networks are hard to infer due to computational scaling.
method Directed convex subgraphs and minimal d-decomposition tree for decomposition, enabling parallel computation.
result The method reduces computational cost and enables parallel computation.
Modeling inverse dynamics is crucial for accurate feedforward robot control. The model computes the necessary joint torques, to perform a desired movement. The highly non-linear inverse function of the dynamical system can be approximated using regression techniques. We propose as regression method a tensor decompositi…
Tensor decomposition is an important technique for capturing the high-order interactions among multiway data. Multi-linear tensor composition methods, such as the Tucker decomposition and the CANDECOMP/PARAFAC (CP), assume that the complex interactions among objects are multi-linear, and are thus insufficient to repres…
MARS automatically selects tensor decomposition ranks, improving performance in neural network tasks.
problem Determining optimal decomposition ranks in tensor decompositions.
method MARS uses binary masks to learn optimal tensor structure during training via relaxed MAP estimation.
result MARS achieves better results than previous methods in various tasks.
Generalized Robertson-Walker (GRW) spaces constitute a quite important family in Lorentzian geometry, and it is an interesting question to know whether a Lorentzian manifold can be decomposed in such a way. It is well known that the existence of a suitable vector field guaranties the local decomposition of the manifold…
In this paper we study the Föllmer-Schweizer decomposition of a square integrable random variable ξ with respect to a given semimartingale S under restricted information. Thanks to the relationship between this decomposition and that of the projection of ξ with respect to the given information flow, we characteri…
This work improves tensor decomposition methods, especially for large datasets.
problem Lack of efficient methods for estimating Tucker decompositions.
method Applies Johnson-Lindenstrauss type guarantees to Tucker decompositions with random embeddings.
result Effective dimension reduction with minimal error for large tensors.
New algorithm for tensor decomposition and Gaussian mixture models.
problem Efficiently decompose overcomplete order-3 tensors and estimate parameters of Gaussian mixtures.
method Proposes Jennrich's algorithm adapted for tensor decomposition and Gaussian mixture models.
result Efficient algorithm for decomposing symmetric overcomplete order-3 tensors and estimating parameters of Gaussian mixtures.
We propose the Relational Tucker3 (RT) decomposition for multi-relational link prediction in knowledge graphs. We show that many existing knowledge graph embedding models are special cases of the RT decomposition with certain predefined sparsity patterns in its components. In contrast to these prior models, RT decouple…
A new probabilistic BTD method for tensor data.
problem Modeling higher-order tensors with robust inference.
method Probabilistic Block-Term Decomposition using variational Bayesian inference and von-Mises Fisher distribution.
result The proposed pBTD can quantify multi-linear structures robustly.
Study on signal-plus-noise decomposition in nonlinear spiked random matrices.
problem Nonlinear spiked random matrix models with rank-one signal and noise.
method Signal-plus-noise decomposition and phase transition analysis.
result Identified precise phase transitions in signal components at critical thresholds.
TreeHFD algorithm explains tree ensemble models through hierarchical orthogonality.
problem Difficulty in explaining black-box tree ensemble models.
method TreeHFD algorithm using hierarchical orthogonality constraints.
result TreeHFD estimates Hoeffding decomposition from data samples.
Paper introduces Modular Jets for diagnosing model decompositions in pipelines.
problem Evaluating model decompositions in pipelines for unique identification.
method Estimates empirical jets from module-level representations to diagnose mirage vs identifiable decompositions.
result Proves jet-identifiability theorem for two-module linear regression pipelines.
Study on spin random fields using chaos decomposition for cosmic microwave background modeling.
problem Modeling polarization of Cosmic Microwave Background using spin random fields.
method Explicit Wiener-Itô chaos decomposition of area measures of level sets.
result Reveals a clear difference between high frequency regime and zero spin case.
Study handles in 3D shapes, applies to material patterns.
problem Understanding 3D shapes through handlebody decompositions.
method Introduced handlebody decompositions, showed stability, applied to materials.
result Stable equivalence of handlebody decompositions in 3-manifolds.
Scalable and robust TR decomposition for large-scale data with missing entries and outliers.
problem Handling large-scale tensor data with missing entries and outliers.
method Auto-weighted steepest descent method for missing entries and outliers identification, FGMC and RStS strategies.
result Outperforms existing TR decomposition methods in the presence of outliers and runs faster than robust tensor completion algorithms.
This work introduces a bias-variance decomposition for proper scores, improving uncertainty estimation in predictive models.
problem Reliable uncertainty estimation for predictions in safety-critical applications, especially under domain drift.
method Developed a general bias-variance decomposition for proper scores, introducing the Bregman Information as the variance term.
result The decomposition provides novel formulations for different predictive tasks, including classification and model ensembles.
Estimates MLDS using tensor decomposition, improving upon existing methods.
problem Learning mixtures of linear dynamical systems from input-output data.
method Proposes a moment-based estimator using tensor decomposition.
result Improves sample complexity bounds for estimating MLDS.
We give a description of the model theoretic relation of forking independence in terms of the notion of JSJ decompositions in non abelian free groups.
We present an approach for penalized tensor decomposition (PTD) that estimates smoothly varying latent factors in multi-way data. This generalizes existing work on sparse tensor decomposition and penalized matrix decompositions, in a manner parallel to the generalized lasso for regression and smoothing problems. Our ap…
The paper classifies decompositions of 3-sphere and lens spaces with handlebodies.
problem Classifying decompositions of 3-manifolds with handlebodies.
method Studied decompositions of 3-sphere and lens spaces with three handlebodies, using stabilizations.
result Determined whether decompositions are stabilized.
Analogues of JSJ decompositions were developed for Poincaré duality pairs in [19]. These decompositions depend only on the group. Our focus will be on describing the edge splittings of these decompositions more precisely. We use our results to compare these decompositions with two other closely related decompositions.
We set up an abstract framework that allows the investigation of Iwasawa decompositions for involutive infinite-dimensional Lie groups modeled on Banach spaces. As an application, we construct Iwasawa decompositions for classical real or complex Banach-Lie groups associated with the Schatten ideals ${\mathfrak S}_p({\m…