arXiv research

We study algebraic varieties of ReLU networks to understand their representable functions.

Piecewise linear activations create many spurious local minima in neural networks.

Constructs finite element spaces for $(p,q)$-forms, excluding one subspace.

Causal deep learning tackles causal inference using tensor factor analysis.

Proposes MLDP for modeling multilinear data.

New method forecasts multilinear data using tensor autoregression.

Geometrically, tensors of fixed rank form a minimal submanifold.

Unified multilinear model for causal factor disentanglement.

We learn sensor trees from training data to minimize sensor acquisition costs during test time. Our system adaptively selects sensors at each stage if necessary to make a confident classification. We pose the problem as empirical risk minimization over the choice of trees and node decision rules. We decompose the probl…

New algorithm solves $\ell_0$-norm constrained multilinear logistic regression for tensor data.

Tucker decomposition is the cornerstone of modern machine learning on tensorial data analysis, which have attracted considerable attention for multiway feature extraction, compressive sensing, and tensor completion. The most challenging problem is related to determination of model complexity (i.e., multilinear rank), e…

In this paper we present a new model and an algorithm for unsupervised clustering of 2-D data such as images. We assume that the data comes from a union of multilinear subspaces (UOMS) model, which is a specific structured case of the much studied union of subspaces (UOS) model. For segmentation under this model, we de…

Wide neural networks become linear, but adding bottlenecks makes them bilinear or multilinear.

GMT improves interpretability of XGNNs by approximating SubMT.

The main results of our paper deal with the lifting problem for multilinear differential operators between complexes of horizontal de Rham forms on the infinite jet bundle. We answer the question when does an n-multilinear differential operator from the space of (N,0)-forms (where N is the dimension of the base) to the…

Study uses random matrix theory to improve tensor approximation accuracy.

Principal component analysis (PCA) is an unsupervised method for learning low-dimensional features with orthogonal projections. Multilinear PCA methods extend PCA to deal with multidimensional data (tensors) directly via tensor-to-tensor projection or tensor-to-vector projection (TVP). However, under the TVP setting, i…

Matrix factorizations and their extensions to tensor factorizations and decompositions have become prominent techniques for linear and multilinear blind source separation (BSS), especially multiway Independent Component Analysis (ICA), NonnegativeMatrix and Tensor Factorization (NMF/NTF), Smooth Component Analysis (Smo…

Proposes FMPCA for federated tensor data dimensionality reduction.

Given a multifunction from $X$ to the $k-$fold symmetric product $Sym_k(X)$, we use the Dold-Thom Theorem to establish a homological selection Theorem. This is used to establish existence of Nash equilibria. Cost functions in problems concerning the existence of Nash Equilibria are traditionally multilinear in the mixe…

Derives a primal-dual MLSVD formulation for multilinear data.

New model generates unseen attribute combinations from limited data.

Dimensionality reduction is a main step in the learning process which plays an essential role in many applications. The most popular methods in this field like SVD, PCA, and LDA, only can be applied to data with vector format. This means that for higher order data like matrices or more generally tensors, data should be…

MCCA extracts shared structure from multiple tensor datasets.

Efficiently optimizes boolean functions using multilinear polynomials and exponential weight updates.

The aim of this work is to lay the foundations of differential geometry and Lie theory over the general class of topological base fields and -rings for which a differential calculus has been developed in recent work (collaboration with H. Gloeckner and K.-H. Neeb), without any restriction on the dimension or on the cha…

Paper introduces a new multilinear functional for spectral triples and computes its properties.

Nowadays, with the availability of massive amount of trade data collected, the dynamics of the financial markets pose both a challenge and an opportunity for high frequency traders. In order to take advantage of the rapid, subtle movement of assets in High Frequency Trading (HFT), an automatic algorithm to analyze and …

Extends De Leeuw theorems to noncommutative groups and multipliers.

A linear Lie rack structure on a finite dimensional vector space $V$ is a Lie rack operation $(x,y)\mapsto x\rhd y$ pointed at the origin and such that for any $x$, the left translation $\mathrm{L}_x:y\mapsto \mathrm{L}_x(y)= x\rhd y$ is linear. A linear Lie rack operation $\rhd$ is called analytic if for any $x,y\in V…

Four algorithms improve sparse tensor BR1Approx with theoretical guarantees.

We give an algorithm for completing an order-$m$ symmetric low-rank tensor from its multilinear entries in time roughly proportional to the number of tensor entries. We apply our tensor completion algorithm to the problem of learning mixtures of product distributions over the hypercube, obtaining new algorithmic result…

The goal of tensor completion is to fill in missing entries of a partially known tensor (possibly including some noise) under a low-rank constraint. This may be formulated as a least-squares problem. The set of tensors of a given multilinear rank is known to admit a Riemannian manifold structure, thus methods of Rieman…

Algorithm identifies sources in product distributions with improved complexity.

We use partial actions, as formalized by Exel, to construct various commensurating actions. We use this in the context of groups piecewise preserving a geometric structure, and we interpret the transfixing property of these commensurating actions as the existence of a model for which the group acts preserving the geome…

Investigates stability of piecewise flat Ricci flow using analysis and simulations.

We are interested in approximation of a multivariate function $f(x_1,\dots,x_d)$ by linear combinations of products $u^1(x_1)\cdots u^d(x_d)$ of univariate functions $u^i(x_i)$, $i=1,\dots,d$. In the case $d=2$ it is a classical problem of bilinear approximation. In the case of approximation in the $L_2$ space the bili…

Neural networks can represent complex piecewise functions efficiently.

Theorem proves integrability for piecewise-smooth distributions.

Extends RRR to capture nonlinear interactions in multi-response regression.

Piecewise flat approximations for curvature in Euclidean and non-Euclidean spaces.

We study conformal deformation problems on manifolds with boundary which include prescribing $σ_k\equiv0$ in the interior. In particular, we prove a Dirichlet principle when the induced metric on the boundary is fixed and an Obata-type theorem on the upper hemisphere. We introduce some conformally covariant multilinear…

Optimal tensor PCA for estimating factors and loadings in high-dimensional panel data.

Study geometrically characterizes piecewise circular curves with decreasing curvature.

We introduce the problem of learning mixtures of $k$ subcubes over $\{0,1\}^n$, which contains many classic learning theory problems as a special case (and is itself a special case of others). We give a surprising $n^{O(\log k)}$-time learning algorithm based on higher-order multilinear moments. It is not possible to l…

We propose a new framework for the analysis of low-rank tensors which lies at the intersection of spectral graph theory and signal processing. As a first step, we present a new graph based low-rank decomposition which approximates the classical low-rank SVD for matrices and multi-linear SVD for tensors. Then, building …

The center of a quotient group of piecewise linear homeomorphisms is trivial.

Nonnegative Tucker decomposition (NTD) is a powerful tool for the extraction of nonnegative parts-based and physically meaningful latent components from high-dimensional tensor data while preserving the natural multilinear structure of data. However, as the data tensor often has multiple modes and is large-scale, exist…