arXiv research

Piecewise linear activations create many spurious local minima in neural networks.

Equivariant neural networks use symmetry to interpret complex data.

New proof shows efficient ReLU networks for piecewise linear functions.

A new complexity measure for neural networks improves upon classical methods.

Simplifies PLNNs to interpretable models for better explainability.

SGD trains ReLU networks to implement piecewise linear maps with at most 3 knot points.

We study algebraic varieties of ReLU networks to understand their representable functions.

Two new criteria help understand the advantage of deep neural networks.

Upper bounds on fixed points in PWL neural networks with hyperplane analysis.

We present a new, unifying approach following some recent developments on the complexity of neural networks with piecewise linear activations. We treat neural network layers with piecewise linear activations as tropical polynomials, which generalize polynomials in the so-called $(\max, +)$ or tropical algebra, with pos…

GraN-GAN normalizes gradients for better GAN performance.

Paper proposes LANN to measure model complexity of neural networks with curve activation functions.

Unified piecewise-linear Ricci flows improve community detection.

Study growth patterns in random networks using i.i.d. perturbations.

The center of a quotient group of piecewise linear homeomorphisms is trivial.

New framework interprets deep neural networks through input structure.

We study the complexity of functions computable by deep feedforward neural networks with piecewise linear activations in terms of the symmetries and the number of linear regions that they have. Deep networks are able to sequentially map portions of each layer's input-space to the same output. In this way, deep models c…

This article provides an attempt to extend concepts from the theory of Riemannian manifolds to piecewise linear spaces. In particular we propose an analogue of the Ricci tensor, which we give the name of an Einstein vector field. On a given set of piecewise linear spaces we define and discuss (normalized) Ricci flows. …

Global approximation for piecewise linear paths via signatures.

We prove that every piecewise linear manifold of dimension up to four on which a finite group acts by piecewise linear homeomorphisms admits a compatible smooth structure with respect to which the group acts smoothly. This solves a challenge posed by Thurston in dimension three and confirms a conjecture by Kwasik and L…

Paper introduces $\ell$-DER for regression tasks using morphological operators and convex-concave procedure.

New algorithms optimize neural networks with ReLU activations using sampling.

Signature uniquely identifies piecewise linear surfaces up to thin homotopy.

Hi-fi priors enhance BNNs by learning flexible activations.

To help understand the underlying mechanisms of neural networks (NNs), several groups have, in recent years, studied the number of linear regions $\ell$ of piecewise linear functions generated by deep neural networks (DNN). In particular, they showed that $\ell$ can grow exponentially with the number of network paramet…

Efficiently checks local robustness in neural networks using geometric projections.

Study shows efficient neural network approach for stochastic bandits.

PARC uses piecewise linear predictors for regression and classification.

In this paper, we introduce a bordism category $\mathcal{C}_d^{PL}$ whose objects are bundles of closed $(d-1)$-dimensional piecewise linear manifolds and whose morphisms are bundles of $d$-dimensional piecewise linear cobordisms. In the main theorem of this article, we show that the classifying space $B\mathcal{C}_d^{…

Unified approach to verify NN properties using ReLU's unique polytope structure.

This work generalizes bounds on the number of linear regions in CPWL NNs.

The study estimates the expressiveness of GCNs with bounds on the number of linear regions.

Deep neural nets on 1-D data are convex Lasso models with reflection features.

Most existing interpretable methods explain a black-box model in a post-hoc manner, which uses simpler models or data analysis techniques to interpret the predictions after the model is learned. However, they (a) may derive contradictory explanations on the same predictions given different methods and data samples, and…

Enhances deep neural networks for MRI reconstruction by increasing expressivity.

The approximation power of general feedforward neural networks with piecewise linear activation functions is investigated. First, lower bounds on the size of a network are established in terms of the approximation error and network depth and width. These bounds improve upon state-of-the-art bounds for certain classes o…

ReLU neural networks define piecewise linear functions of their inputs. However, initializing and training a neural network is very different from fitting a linear spline. In this paper, we expand empirically upon previous theoretical work to demonstrate features of trained neural networks. Standard network initializat…

Paper presents ABGD for efficient piecewise linear regression in high dimensions.

We consider smooth isotropic immersions from the 2-dimensional torus into $R^{2n}$, for $n \geq 2$. When $n = 2$ the image of such map is an immersed Lagrangian torus of $R^4$. We prove that such isotropic immersions can be approximated by arbitrarily $C^0$-close piecewise linear isotropic maps. If $n \geq 3$ the piece…

New cones in 4D space found with minimal mass.

Modified ReLU networks improve regression estimation rates.

It is well-known that the expressivity of a neural network depends on its architecture, with deeper networks expressing more complex functions. In the case of networks that compute piecewise linear functions, such as those with ReLU activation, the number of distinct linear regions is a natural measure of expressivity.…

Paper proposes variational inference for piecewise-linear systems.

The developments of deep neural networks (DNN) in recent years have ushered a brand new era of artificial intelligence. DNNs are proved to be excellent in solving very complex problems, e.g., visual recognition and text understanding, to the extent of competing with or even surpassing people. Despite inspiring and enco…

The paper tackles prescribing discrete Gaussian curvature on polyhedral surfaces.

First explicit isometric immersion of a flat Klein bottle in 3D space.

Hilbert initiated the standpoint in foundations of mathematics. From this standpoint, we allow only a finite number of repetitions of elementary operations when we construct objects and morphisms. When we start from a subset of a Euclidean space. Then we assume that any element of the line has only a finite number of c…

This paper introduces a new method to better understand financial market causality.