arXiv research

Bilinear MLPs offer a new way to interpret deep learning models without complex nonlinearities.

MLP-Mixer achieves better performance through sparsity and wider architecture.

PAC-Bayesian bounds for MLPs with cross entropy loss validated.

Interpolated-MLPs control inductive bias for better performance in low-compute tasks.

ES-MLP combines Graph-MLP with edge splitting for node classification on both homophilic and heterophilic graphs.

A new GNM model outperforms MLP for tabular data.

This study compares GNNs and GA-MLPs, finding GA-MLPs can distinguish graphs but not count walks.

This paper presents a novel unifying framework of bilinear LSTMs that can represent and utilize the nonlinear interaction of the input features present in sequence datasets for achieving superior performance over a linear LSTM and yet not incur more parameters to be learned. To realize this, our unifying framework allo…

New algorithm reduces regret in graphical bilinear bandits.

KANs replace fixed MLP weights with learnable edge functions, improving accuracy and interpretability.

Generalization of twistor spinors to Kähler manifolds which are called Kählerian twistor spinors are considered. We find the differential equation satisfied by the bilinear forms of Kählerian twistor spinors. We show that the bilinear form equation reduces to Kählerian conformal Killing-Yano equation under special cond…

Artificial Neural Networks(ANN) has been phenomenally successful on various pattern recognition tasks. However, the design of neural networks rely heavily on the experience and intuitions of individual developers. In this article, the author introduces a mathematical structure called MLP algebra on the set of all Multi…

Revisits neural collaborative filtering vs. matrix factorization, showing dot product superiority.

Algorithm identifies bilinear dynamical systems from noisy data.

Study uses MLP models to predict large-cap US stocks, finding 2-3 hidden layers more flexible.

In this paper, we extend Su-Zhang's Cheeger-Mueller type theorem for symmetric bilinear torsions to manifolds with boundary in the case that the Riemannian metric and the non-degenerate symmetric bilinear form are of product structure near the boundary. Our result also extends Bruening-Ma's Cheeger-Mueller type theorem…

SpArX creates faithful explanations of neural networks' decision-making.

Identifies bilinear systems from a single trajectory with optimal sample complexity.

Generalizes Riemann's results on flat coordinates for non-symmetric bilinear forms.

This paper proposes a new approach to Transformers by integrating hierarchical associative memory with MetaFormers.

Designs an MLP from LDA for multi-Gaussian class classification.

The theory of harmonic symmetric bilinear forms on a Riemannian manifold is an analogue of the theory of harmonic exterior differential forms on this manifold. To show this, we must consider every symmetric bilinear form on a Riemannian manifold as a one-form with values in the cotangent bundle of this manifold. In thi…

m-arcsinh improves SVM and MLP reliability and speed in scikit-learn.

This study compares MLPs and KANs in low-data regimes, finding MLPs with personalized activation functions outperform KANs.

Paper reduces sample complexity for bilinear systems identification to nearly constant.

This note provides a neat and enjoyable expansion and application of the magnificent Ordentlich-Cover theory of "universal portfolios." I generalize Cover's benchmark of the best constant-rebalanced portfolio (or 1-linear trading strategy) in hindsight by considering the best bilinear trading strategy determined in hin…

Enhances knot invariants using bilinear forms on vector spaces.

Constructs a bilinear form from a quasimorphism on symplectic manifold groups.

New approach finds minimum width for deep, narrow MLPs.

Investigates transfer learning in spatial statistics.

In this paper, we propose to employ a bank of modality-dedicated Convolutional Neural Networks (CNNs), fuse, train, and optimize them together for person classification tasks. A modality-dedicated CNN is used for each modality to extract modality-specific features. We demonstrate that, rather than spatial fusion at the…

Hybrid model improves wind speed prediction accuracy using MLP and WOA.

S-GAI initializes MLPs using spectral geometry from data, improving performance.

We define a type of biquandle which is a generalization of symplectic quandles. We use the extra structure of these bilinear biquandles to define new knot and link invariants and give some examples.

Closed-form polynomial approximations replace MLPs in transformers, enabling new interpretability methods.

ResNets promote smoother interpolations than MLPs, enhancing generalization.

A parsimonious model reduces over-parameterization in skewed matrix variate mixtures.

Study learns linear system dynamics from noisy bilinear data.

Topo-MLP learns network representations without message passing.

Group invariant and equivariant Multilayer Perceptrons (MLP), also known as Equivariant Networks, have achieved remarkable success in learning on a variety of data structures, such as sequences, images, sets, and graphs. Using tools from group theory, this paper proves the universality of a broad class of equivariant M…

MLPs can approximate any function in context, challenging the importance of in-context universality.

We use the Jones-Wenzl idempotents to construct a basis of Temperley-Lieb algebra TL_n. This allows a short calculation for a Gram determinant of Lickorish's bilinear form on the Temperley-Lieb algebra.

Non-bilinear observations make optimal control harder, showing non-convex costs and non-affine optimal controllers.

This thesis is concerned with the theory of invariant bilinear differential pairings on parabolic geometries. It introduces the concept formally with the help of the jet bundle formalism and provides a detailed analysis. More precisely, after introducing the most important notations and definitions, we first of all giv…

Unified bounds for sketched bilinear forms in machine learning and statistics.

BiN normalizes financial time-series for better forecasting.

Transformers show better in-context learning resilience under distribution shifts than simple MLPs.

Transformers with MLP heads outperform linear baselines in in-context learning.