arXiv research

Ansor generates high-performance tensor programs for deep learning.

Study Gaussian-process limits of neural networks using tensor programs.

New algorithms solve tensor problems with random components using SDP.

TF-Coder simplifies tensor manipulation programming in TensorFlow.

It is a significant challenge to design probabilistic programming systems that can accommodate a wide variety of inference strategies within a unified framework. Noting that the versatility of modern automatic differentiation frameworks is based in large part on the unifying concept of tensors, we describe a software a…

Paper analyzes infinite-width attention layers using Tensor Programs.

Optimizing the execution time of tensor program, e.g., a convolution, involves finding its optimal configuration. Searching the configuration space exhaustively is typically infeasible in practice. In line with recent research using TVM, we propose to learn a surrogate model to overcome this issue. The model is trained…

SOS programming verifies MTW tensor non-negativity for optimal transport maps.

We introduce a learning-based framework to optimize tensor programs for deep learning workloads. Efficient implementations of tensor operators, such as matrix multiplication and high dimensional convolution, are key enablers of effective deep learning systems. However, existing systems rely on manually optimized librar…

Tensor programs prove neural network limits for any architecture.

New algorithm completes nonnegative tensors with fewer samples and faster convergence.

Mathematical study supports connection between 3D manifolds and modular tensor categories.

In this paper, we consider the Tensor Robust Principal Component Analysis (TRPCA) problem, which aims to exactly recover the low-rank and sparse components from their sum. Our model is based on the recently proposed tensor-tensor product (or t-product). Induced by the t-product, we first rigorously deduce the tensor sp…

We develop fast spectral algorithms for tensor decomposition that match the robustness guarantees of the best known polynomial-time algorithms for this problem based on the sum-of-squares (SOS) semidefinite programming hierarchy. Our algorithms can decompose a 4-tensor with $n$-dimensional orthonormal components in the…

Efficiently price high-dimensional Bermudan options using tensor compression.

New algorithm for nonnegative tensor completion with linear convergence rate.

Tensor completion recovers a multi-dimensional array from a limited number of measurements. Using the recently proposed tensor ring (TR) decomposition, in this paper we show that a d-order tensor of dimensional size n and TR rank r can be exactly recovered with high probability by solving a convex optimization program,…

Batch-splitting (data-parallelism) is the dominant distributed Deep Neural Network (DNN) training strategy, due to its universal applicability and its amenability to Single-Program-Multiple-Data (SPMD) programming. However, batch-splitting suffers from problems including the inability to train very large models (due to…

We investigate the triviality of compact Ricci solitons under general scalar conditions involving the Weyl tensor. More precisely, we show that a compact Ricci soliton is Einstein if a generic linear combination of divergences of the Weyl tensor contracted with suitable covariant derivatives of the potential function v…

We describe a simple, low-level approach for embedding probabilistic programming in a deep learning ecosystem. In particular, we distill probabilistic programming down to a single abstraction---the random variable. Our lightweight implementation in TensorFlow enables numerous applications: a model-parallel variational …

OGRePy simplifies tensor calculations in general relativity.

This work studies the Tensor Robust Principal Component Analysis (TRPCA) problem, which aims to exactly recover the low-rank and sparse components from their sum. Our model is motivated by the recently proposed linear transforms based tensor-tensor product and tensor SVD. We define a new transforms depended tensor rank…

This paper conducts a rigorous analysis for provable estimation of multidimensional arrays, in particular third-order tensors, from a random subset of its corrupted entries. Our study rests heavily on a recently proposed tensor algebraic framework in which we can obtain tensor singular value decomposition (t-SVD) that …

Tensor decomposition has been extensively used as a tool for exploratory analysis. Motivated by neuroscience applications, we study tensor decomposition with Boolean factors. The resulting optimization problem is challenging due to the non-convex objective and the combinatorial constraints. We propose Binary Matching P…

New tensor completion method converges linearly and is highly practical.

SimTensor is a multi-platform, open-source software for generating artificial tensor data (either with CP/PARAFAC or Tucker structure) for reproducible research on tensor factorization algorithms. SimTensor is a stand-alone application based on MATALB. It provides a wide range of facilities for generating tensor data w…

A wide class of machine learning algorithms can be reduced to variable elimination on factor graphs. While factor graphs provide a unifying notation for these algorithms, they do not provide a compact way to express repeated structure when compared to plate diagrams for directed graphical models. To exploit efficient t…

The paper analyzes tensor recovery from symmetric rank-one measurements using information theory.

Tensor logic aims to unify AI types with scalable and transparent features.

In the tensor completion problem, one seeks to estimate a low-rank tensor based on a random sample of revealed entries. In terms of the required sample size, earlier work revealed a large gap between estimation with unbounded computational resources (using, for instance, tensor nuclear norm minimization) and polynomial…

In this technical note we give a purely geometric understanding of discrete torsion, as an analogue of orbifold Wilson lines for two-form tensor field potentials. In order to introduce discrete torsion in this context, we describe gerbes and the description of certain type II supergravity tensor field potentials as con…

Paper introduces ZIPTF and C-ZIPTF for better tensor factorization of zero-inflated count data.

In this paper, the notion of strongly typed language will be borrowed from the field of computer programming to introduce a calculational framework for linear algebra and tensor calculus for the purpose of detecting errors resulting from inherent misuse of objects and for finding natural formulations of various objects…

This paper studies a tensor-structured linear regression model with a scalar response variable and tensor-structured predictors, such that the regression parameters form a tensor of order $d$ (i.e., a $d$-fold multiway array) in $\mathbb{R}^{n_1 \times n_2 \times \cdots \times n_d}$. It focuses on the task of estimatin…

Advancing research in the emerging field of deep graph learning requires new tools to support tensor computation over graphs. In this paper, we present the design principles and implementation of Deep Graph Library (DGL). DGL distills the computational patterns of GNNs into a few generalized sparse tensor operations su…

NEST optimizes deep learning training by placing devices efficiently across networks and memory.

Adapts scanning algorithm for odd Khovanov homology.

The paper bounds eigenvalues and integrals of eigenfunctions on hyperbolic manifolds.

A brief history of the investigation of the Weil-Petersson curvature and a summary of Teichmüller theory are provided. A report is presented on the program to describe an intrinsic geometry with the Weil-Petersson metric and geodesic-length functions. Formulas for the metric, covariant derivative and formulas for the c…

New modular data from torus bundles via particle-hole equivariantization.

We give a new approach to the dictionary learning (also known as "sparse coding") problem of recovering an unknown $n\times m$ matrix $A$ (for $m \geq n$) from examples of the form \[ y = Ax + e, \] where $x$ is a random vector in $\mathbb R^m$ with at most $τm$ nonzero coordinates, and $e$ is a random noise vector in …

We propose the convex factorization machine (CFM), which is a convex variant of the widely used Factorization Machines (FMs). Specifically, we employ a linear+quadratic model and regularize the linear term with the $\ell_2$-regularizer and the quadratic term with the trace norm regularizer. Then, we formulate the CFM o…

Monte Carlo methods are critical to many routines in quantitative finance such as derivatives pricing, hedging and risk metrics. Unfortunately, Monte Carlo methods are very computationally expensive when it comes to running simulations in high-dimensional state spaces where they are still a method of choice in the fina…

We address the structure identification and the uniform approximation of sums of ridge functions $f(x)=\sum_{i=1}^m g_i(a_i\cdot x)$ on ${\mathbb R}^d$, representing a general form of a shallow feed-forward neural network, from a small number of query samples. Higher order differentiation, as used in our constructive a…

A toy model shows how locality can emerge in the universe's Hamiltonian and initial state.

SED integrates synthesis, execution, and debugging for neural program synthesis.

We introduce a tensor-based clustering method to extract sparse, low-dimensional structure from high-dimensional, multi-indexed datasets. This framework is designed to enable detection of clusters of data in the presence of structural requirements which we encode as algebraic constraints in a linear program. Our cluste…

Improves probabilistic programming by analyzing program structure.