Paper proposes efficient methods for high-order clustering in tensor block models.
problem High-order clustering of multiway datasets in neuroimaging, genomics, etc.
method Tensor block model and computationally efficient algorithms (HLloyd, HSC)
result Achieves high-order exact clustering with statistical optimality and computational efficiency.
Algorithm for exact partitioning of high-order models using convex tensor relaxation.
problem Exact partitioning of high-order models.
method Defining a general class of m-degree Homogeneous Polynomial Models, relaxing the high-order combinatorial problem to a convex conic form problem, defining the Carathéodory symmetric tensor cone, and constructing a primal-dual certificate. result The solution of the convex relaxation is correct and provides a statistical upper bound for exact partitioning.
THS-GAN uses tensorizing and high-order pooling for AD diagnosis.
problem Early diagnosis of Alzheimer's Disease (AD) using MRI images.
method Tensorizing a three-player cooperative game framework with high-order pooling for MRI images.
result THS-GAN achieves superior performance in AD diagnosis compared to existing methods.
Exact partitioning of high-order planted models achieved through convex optimization.
problem Efficiently partitioning hypergraphs generated by high-order planted models.
method Solving a computationally efficient convex optimization problem with a tensor nuclear norm constraint.
result Exact recovery of true underlying cluster structures with high probability.
Bayesian tensor train method recovers streaming data with high accuracy.
problem Recovering high-order, incomplete, and noisy streaming data.
method Bayesian tensor train decomposition using streaming variational Bayes method.
result The proposed SPTT algorithm excels in recovering streaming data compared to state-of-the-art methods.
RotEqNet preserves rotation symmetry in fluid systems using high-order tensors.
problem Lack of rotational symmetry in machine learning models for fluid systems.
method Introduces RotEqNet, a network that guarantees rotation-equivariance for high-order tensors.
result RotEqNet reduces errors and maintains rotation-equivariance in fluid systems.
Develops methods to estimate high rank tensors from noisy data.
problem Estimating high rank tensors from noisy observations.
method Generative latent variable tensor model, polynomial-time spectral algorithm.
result Achieves computationally optimal rate for signal tensor estimation.
In this paper, we propose a general framework for sparse and low-rank tensor estimation from cubic sketchings. A two-stage non-convex implementation is developed based on sparse tensor decomposition and thresholded gradient descent, which ensures exact recovery in the noiseless case and stable recovery in the noisy cas…
Optimal low rank tensor recovery requires a minimum number of entries for accurate reconstruction.
problem Exact recovery of high order tensors of low rank from a subset of their entries.
method Riemannian optimization algorithm with initial value from a spectral method, leveraging tensor restricted isometry property and curvature of the manifold.
result Tensor of size nimesnimes⋯imesn of ranks (r,⋯,r) can be reconstructed with high probability from O((rd+dnr)log(d)) entries. GETF efficiently decomposes large-scale Boolean tensors.
problem Efficiently factorizing large-scale Boolean tensors.
method Geometric Expansion for all-order Tensor Factorization (GETF).
result GETF significantly improves reconstruction accuracy and efficiency.
DTCCA learns nonlinear transformations of multi-view data for high-order correlation.
problem Learning complex nonlinear transformations of multiple data views.
method Maximizes high-order canonical correlation by jointly learning transformations of each view using a reformulated tensor decomposition.
result DTCCA efficiently handles high-dimensional and large number of views, overcoming scalability issues.
Paper explores limits of high-order clustering with planted structures.
problem Statistical and computational limits of high-order clustering with planted structures.
method Developed methods for detection and recovery of clusters, identified signal-to-noise ratio boundaries.
result Sharp boundaries of signal-to-noise ratio for statistical and computational feasibility.
New algorithms improve tensor CP decomposition under mild conditions.
problem Improving tensor CP decomposition with theoretical guarantees under mild incoherence conditions.
method Composite PCA and Concurrent Orthogonalization algorithms.
result Theoretical guarantees and practical superiority over existing methods.
In this paper, we investigate effective sketching schemes via sparsification for high dimensional multilinear arrays or tensors. More specifically, we propose a novel tensor sparsification algorithm that retains a subset of the entries of a tensor in a judicious way, and prove that it can attain a given level of approx…
Optimizes mixture models without parametrizing distributions using tensor decomposition.
problem Estimating conditionally-independent mixture models in high dimensions.
method Alternating least squares optimization scheme for tensor decomposition.
result Competitive performance and applicability to various models and applications.
In this article, we consider the sparse tensor singular value decomposition, which aims for dimension reduction on high-dimensional high-order data with certain sparsity structure. A method named Sparse Tensor Alternating Thresholding for Singular Value Decomposition (STAT-SVD) is proposed. The proposed procedure featu…
Modeling interactions between features improves the performance of machine learning solutions in many domains (e.g. recommender systems or sentiment analysis). In this paper, we introduce Exponential Machines (ExM), a predictor that models all interactions of every order. The key idea is to represent an exponentially l…
PolyGAN uses high-order polynomials to generate data without activation functions.
problem Learning generative models for high-dimensional distributions.
method PolyGAN models the generator as a high-order polynomial represented by high-order tensors, using tensor decompositions to reduce parameters.
result PolyGAN can approximate data distributions without activation functions.
Proposes a bilinear form to efficiently represent high-order temporal action information.
problem Efficiently capturing subtle and precise actions in long videos.
method Low-rank frontal tensors and bilinear form for extracting high-order information.
result Bilinear form outperforms state-of-the-art methods on temporal action segmentation.
A new method for embedding sparse high-order interactions.
problem Learning embeddings from sparse high-order interaction events.
method Hybridizing sparse hypergraph and matrix Gaussian processes.
result Strong asymptotic bounds on sparsity ratio.
Rank-R FNN handles high-dimensional data efficiently.
problem Handling irregularities in high-dimensional data.
method Imposes Canonical/Polyadic decomposition on parameters.
result Achieves state-of-the-art performance on higher-order tensor data.
Tensor decomposition is an important technique for capturing the high-order interactions among multiway data. Multi-linear tensor composition methods, such as the Tucker decomposition and the CANDECOMP/PARAFAC (CP), assume that the complex interactions among objects are multi-linear, and are thus insufficient to repres…
Tensor decompositions have rich applications in statistics and machine learning, and developing efficient, accurate algorithms for the problem has received much attention recently. Here, we present a new method built on Kruskal's uniqueness theorem to decompose symmetric, nearly orthogonally decomposable tensors. Unlik…
Sparse sampling method for tensor factorization and completion of high rank tensors.
problem Completion of high rank tensors with missing data in recommendation systems.
method Sparse measurements and message-passing algorithms in a high-dimensional limit.
result Theoretical insights and performance analysis of tensor factorization in dense limit.
High-dimensional tensors or multi-way data are becoming prevalent in areas such as biomedical imaging, chemometrics, networking and bibliometrics. Traditional approaches to finding lower dimensional representations of tensor data include flattening the data and applying matrix factorizations such as principal component…
Study of Langevin dynamics for tensor PCA recovery in high dimensions.
problem Recovering hidden signal vectors (spikes) from noisy Gaussian tensor observations.
method Langevin dynamics approach for nonconvex optimization.
result Sample complexity matches the single-spike case but degrades for all spikes.
The reduction theorems for general linear and classical connections are generalized for operators with values in higher order gauge-natural bundles. We prove that natural operators depending on the s1-jets of classical connections, on the s2-jets of general linear connections and on the r-jets of tensor fields …
Nowadays, with the availability of massive amount of trade data collected, the dynamics of the financial markets pose both a challenge and an opportunity for high frequency traders. In order to take advantage of the rapid, subtle movement of assets in High Frequency Trading (HFT), an automatic algorithm to analyze and …
SLTR model preserves tensor structure and reduces prediction time costs.
problem Efficiently predicting tensor data relationships with structural preservation.
method SLTR model enforces sparsity and low-rankness via proximal gradient method.
result SLTR achieves better solutions with significantly reduced time costs.
Deterministic bounds for tensor singular values and vectors, differing from matrix cases.
problem Spectral learning of higher-order orthogonally decomposable tensors.
method Deterministic perturbation bounds for singular values and vectors of orthogonally decomposable tensors.
result Perturbation affects each essential singular value/vector in isolation, independent of multiplicity and distance from other singular values.
A new method estimates rare events using tensor trains.
problem Estimating rare event probabilities in high-dimensional problems.
method Approximating optimal importance distribution via tensor-train decompositions and compositions.
result Better variance reduction and efficient computation of rare event probabilities.
Unified approach tackles high-dimensional tensor bandits with convex optimization and weakly decomposable regularizers.
problem Challenges in high-dimensional generalized tensor bandits where existing algorithms fail.
method Proposes a generalized linear tensor bandits algorithm with a unified analytical framework using convex optimization and weakly decomposable regularizers.
result Unified analytical framework provides better bounds and broader applicability compared to existing methods.
Efficient video captioning model captures cross-modal interactions.
problem Capturing frame-level cross-modal interactions in video captioning.
method Proposes High-Order Cross-Modal Attention (HOCA) and Low-Rank HOCA.
result Low-Rank HOCA achieves state-of-the-art performance.
Estimates cohomology dimensions for Nakano q-semipositive line bundles.
problem Estimating cohomology dimensions for Nakano q-semipositive line bundles.
method Asymptotic estimates for high tensor powers of semipositive line bundles over q-convex manifolds and various complex manifolds.
result Optimal order estimates for cohomology dimensions.
Develops TOFU for tensor bandits with low-rank structure.
problem Linear bandit models fail to capture high-dimensional, low-rank tensor structures.
method Develops TOFU, a tensor bandit algorithm that estimates low-dimensional subspaces and uses norm constraints.
result Improves regret bound by a multiplicative factor that grows exponentially in system order.
Study of asymmetric rank-one tensor models with non-Gaussian noise.
problem Analyzing maximum-likelihood estimators for asymmetric rank-one tensor models.
method Spectrally separated branch analysis, resolvent methods, cumulant expansions, Efron-Stein-type variance bounds.
result Asymptotic singular value and mode-wise alignments are robust to non-Gaussian noise.
Constraining linear layers in neural networks to respect symmetry transformations from a group G is a common design principle for invariant networks that has found many applications in machine learning. In this paper, we consider a fundamental question that has received little attention to date: Can these networks ap…
HTE improves PINNs for high-dimensional, high-order PDEs by reducing computational cost and memory usage.
problem Challenges in solving high-dimensional, high-order PDEs with PINNs due to computational cost and memory constraints.
method Introduces Hutchinson Trace Estimation (HTE) to transform Hessian matrix calculations into Hessian vector products (HVP), reducing computational cost and memory usage.
result HTE significantly reduces memory consumption and computational cost, enabling faster and more efficient solution of high-dimensional and high-order PDEs.
Paper finds formulas for mutual information and MMSE in matrix tensor product problems.
problem High-dimensional inference problems involving matrix tensor products.
method Single-letter formulas for mutual information and MMSE, using new techniques.
result Analytical formulas describe leading order terms in mutual information and MMSE.
Nonnegative CANDECOMP/PARAFAC (NCP) decomposition is an important tool to process nonnegative tensor. Sometimes, additional sparse regularization is needed to extract meaningful nonnegative and sparse components. Thus, an optimization method for NCP that can impose sparsity efficiently is required. In this paper, we co…
In this paper we propose a tensor-based nonlinear model for high-order data classification. The advantages of the proposed scheme are that (i) it significantly reduces the number of weight parameters, and hence of required training samples, and (ii) it retains the spatial structure of the input samples. The proposed mo…
The completion of tensors, or high-order arrays, attracts significant attention in recent research. Current literature on tensor completion primarily focuses on recovery from a set of uniformly randomly measured entries, and the required number of measurements to achieve recovery is not guaranteed to be optimal. In add…
Neural networks learn faster with correlated latent variables.
problem Efficiently learning from higher-order correlations in neural networks.
method Analytical derivation and simulations of two-layer neural networks.
result Correlations between latent variables speed up learning from higher-order correlations.
The paper proposes a novel tensor-based method for non-parametric density estimation.
problem Effective non-parametric density estimation in high-dimensional multivariate data.
method Tensor factorization and low-rank model of characteristic tensor for improved density estimation.
result The method significantly improves density estimation especially for high-dimensional data and/or sample-starved regimes.
Study recovers spike order in noisy tensor estimation without SNR assumptions.
problem Estimating multiple signal vectors from noisy tensor observations.
method Gradient flow optimization of a nonconvex function.
result Determines sample complexity for efficient permutation recovery.
The performance of most the clustering methods hinges on the used pairwise affinity, which is usually denoted by a similarity matrix. However, the pairwise similarity is notoriously known for its vulnerability of noise contamination or the imbalance in samples or features, and thus hinders accurate clustering. To tackl…
A new method uses CPD to efficiently model feature interactions in non-sequential data.
problem Efficiently modeling feature interactions in non-sequential data with high computational and memory costs.
method Implicitly represent model parameters as a tensor, factorize into a compact Tensor Train (TT) format, and use Canonical Polyadic (CP) Decomposition for invariance to feature ordering.
result The proposed CP-based predictor outperforms other TN-based predictors on sparse data and matches neural network performance on dense non-sequential tasks.
Dynamic tensor data are becoming prevalent in numerous applications. Existing tensor clustering methods either fail to account for the dynamic nature of the data, or are inapplicable to a general-order tensor. Also there is often a gap between statistical guarantee and computational efficiency for existing tensor clust…