arXiv research

New basis for permutation equivariant layers reduces computation costs.

Finsler space is differentiable manifold for which Minkowski space is the fiber of the tangent bundle. To understand structure of the reference frame in Finsler space, we need to understand the structure of orthonormal basis in Minkowski space. In this paper, I considered the definition of orthonormal basis in Minkowsk…

New basis and Schur-Weyl duality for loop Hecke algebra defined.

New framework models complex spatial data with basis functions and graphical vectors.

We study a novel spline-like basis, which we name the "falling factorial basis", bearing many similarities to the classic truncated power basis. The advantage of the falling factorial basis is that it enables rapid, linear-time computations in basis matrix multiplication and basis matrix inversion. The falling factoria…

We propose a new method for learning deep neural network models that is based on a greedy learning approach: we add one basis function at a time, and a new basis function is generated as a non-linear activation function applied to a linear combination of the previous basis functions. Such a method (growing deep neural …

In this paper we give a new basis, $Λ$, for the Homflypt skein module of the solid torus, $\mathcal{S}({\rm ST})$, which was predicted by Jozef Przytycki, using topological interpretation. The basis $Λ$ is different from the basis $Λ^{\prime}$, discovered independently by Hoste--Kidwell \cite{HK} and Turaev \cite{Tu} w…

Ordinal Regression (OR) aims to model the ordering information between different data categories, which is a crucial topic in multi-label learning. An important class of approaches to OR models the problem as a linear combination of basis functions that map features to a high dimensional non-linear space. However, most…

New neural network models for functional data.

Finsler space is differentiable manifold for which Minkowski space is the fiber of the tangent bundle. To understand structure of the reference frame in Finsler space, we need to understand the structure of orthonormal basis in Minkowski space. In this paper, we considered the definition of orthonormal basis in Minkows…

Derives representations invariant under crystallographic groups for functions.

Lower bounds show OLS outperforms basis pursuit in overparameterized linear regression.

Nonnegative matrix factorization (NMF) is a widely used linear dimensionality reduction technique for nonnegative data. NMF requires that each data point is approximated by a convex combination of basis elements. Archetypal analysis (AA), also referred to as convex NMF, is a well-known NMF variant imposing that the bas…

The parallel linear transports defined by flat linear connection are axiomatically described. On this basis a number of properties, some of which are new, of these transports and connections are derived.

Sharp results link DLN gradient flow to basis pursuit optimization and GHA phase transitions.

BP fails to find sparsest solution for structured matrices.

A linear model approximates Gaussian processes for efficient control.

Introduces a neural network-based method for efficient state and parameter estimation in complex systems.

This paper provides estimation and inference methods for the best linear predictor (approximation) of a structural function, such as conditional average structural and treatment effects, and structural derivatives, based on modern machine learning (ML) tools. We represent this structural function as a conditional expec…

New proof of SL(n) skein algebra for twice punctured sphere, showing it's a polynomial algebra.

A new optimizer for deep learning improves accuracy and reduces training time.

Approximate linear programs (ALPs) are well-known models based on value function approximations (VFAs) to obtain policies and lower bounds on the optimal policy cost of discounted-cost Markov decision processes (MDPs). Formulating an ALP requires (i) basis functions, the linear combination of which defines the VFA, and…

Kernel principal component analysis (KPCA) provides a concise set of basis vectors which capture non-linear structures within large data sets, and is a central tool in data analysis and learning. To allow for non-linear relations, typically a full $n \times n$ kernel matrix is constructed over $n$ data points, but this…

In this paper, we introduce a large class of convergent numerical methods, based on (linear) basis function regression technique, to approximate the solution to a forward-backward stochastic differential equation with jumps (FBSDEJ hereafter). Numerical experiment shows good applicability of the proposed method.

Many applications that use empirically estimated functions face a curse of dimensionality, because the integrals over most function classes must be approximated by sampling. This paper introduces a novel regression-algorithm that learns linear factored functions (LFF). This class of functions has structural properties …

In view of the result of Kontsevich, now often called ``the fundamental theorem of Vassiliev theory'', identifying the graded dual of the associated graded vector space to the space of Vassiliev invariants filtered by degree with the linear span of chord diagrams modulo the ``4T-relation'' (and in the unframed case, th…

New ODE-Block handles stateful layers with continuous-in-depth functions using basis functions.

This paper considers method of creation of an advisor and indicator based on the spectral stochastic analysis model, both with linear and non-linear approximation. The problem of entrance to one or another trade position is solved on the basis of combined analysis of dynamics of quotations of all currency pairs, what a…

In many compressive sensing problems today, the relationship between the measurements and the unknowns could be nonlinear. Traditional treatment of such nonlinear relationships have been to approximate the nonlinearity via a linear model and the subsequent un-modeled dynamics as noise. The ability to more accurately ch…

Paper projects GP basis functions using tensor networks to reduce complexity.

The paper presents a method to infer unknown forcing functions in differential equations using Gaussian processes and adjoints.

We consider learning a convex combination of basis models, and present some new theoretical and empirical results that demonstrate the effectiveness of a greedy approach. Theoretically, we first consider whether we can use linear, instead of convex, combinations, and obtain generalization results similar to existing on…

DBKs enable scalable GPs with tractable inference for large datasets.

This work provides closed-form solutions and minimum achievable errors for a large class of low-rank approximation problems in Hilbert spaces. The proposed theorem generalizes to the case of bounded linear operators the previous results obtained in the finite dimensional case for the Frobenius norm. The theorem provide…

Characterizes a specific type of neural network for alternating group equivariance.

New sparse Gaussian process method tackles unconstrained regression problems.

A new method integrates Fourier basis expansion and mapping for improved time series forecasting.

Improved ResNets and DenseNets models for better feature reuse.

We orthogonalize the NSS model to condition and diagnose its ill-conditioned parameters.

New method controls linear systems with partial info and disturbances.

Consider the smooth quadric Q_6 in P^7. The middle homology group H_6(Q_6,Z) is two-dimensional with a basis given by two classes of linear subspaces. We classify all threefolds of bidegree (1,p) inside Q_6.

We focus on the high-dimensional linear regression problem, where the algorithmic goal is to efficiently infer an unknown feature vector $β^*\in\mathbb{R}^p$ from its linear measurements, using a small number $n$ of samples. Unlike most of the literature, we make no sparsity assumption on $β^*$, but instead adopt a dif…

In this paper we provide a \emph{global} investigation of the geometry of parallelizable manifolds (or absolute parallelism geometry) frequently used for application. We discuss the different linear connections and curvature tensors from a global point of view. We give an existence and uniqueness theorem for a remarkab…

Recently, path norm was proposed as a new capacity measure for neural networks with Rectified Linear Unit (ReLU) activation function, which takes the rescaling-invariant property of ReLU into account. It has been shown that the generalization error bound in terms of the path norm explains the empirical generalization b…

This article considers algorithmic and statistical aspects of linear regression when the correspondence between the covariates and the responses is unknown. First, a fully polynomial-time approximation scheme is given for the natural least squares optimization problem in any constant dimension. Next, in an average-case…

BP, a method for sparse recovery, shows generalization error decreases with more features.

Kernel methods are widespread in machine learning; however, they are limited by the quadratic complexity of the construction, application, and storage of kernel matrices. Low-rank matrix approximation algorithms are widely used to address this problem and reduce the arithmetic and storage cost. However, we observed tha…

Unified determinants via a single equation.