arXiv research

DREAM model improves computational efficiency for non-linear effects in relational event models.

The paper generalizes equivariant neural networks on homogeneous spaces to the non-linear setting.

This work optimizes reservoir computing models by linking recurrence and non-linear dynamics.

New neural network models for functional data.

Unified framework for non-linear attention using modern Hopfield networks.

Paper introduces non-linearity signature to measure deep neural network performance.

Gradient descent with growing learning rate enables learning non-linear features in neural networks.

Study on dynamics of non-linear autoencoders learning principal components.

Capacity analysis has been recently introduced as a way to analyze how linear models distribute their modelling capacity across the input space. In this paper, we extend the notion of capacity allocation to the case of neural networks with non-linear layers. We show that under some hypotheses the problem is equivalent …

GP-KAN uses Gaussian Processes in KANs for robust, parameter-efficient non-linear modeling.

Combining neural networks and multiscale decomposition for financial market analysis.

Bayesian neural networks improve uncertainty quantification in non-linear dimensionality reduction.

Enhances Cox model for survival analysis with symbolic non-linear log-risk functions.

New neural network models for complex functional data analysis.

We propose a new notion of `non-linearity' of a network layer with respect to an input batch that is based on its proximity to a linear system, which is reflected in the non-negative rank of the activation matrix. We measure this non-linearity by applying non-negative factorization to the activation matrix. Considering…

Deep networks without non-linearities are equivalent to shallow ones.

Paper converts deep networks to flat, equivalent kernel machines.

DiffKnock improves feature selection in neural networks with complex dependencies and non-linear associations.

This paper shows neural networks can learn non-linear sparse parities.

This paper studies the dynamic generator model for spatial-temporal processes such as dynamic textures and action sequences in video data. In this model, each time frame of the video sequence is generated by a generator model, which is a non-linear transformation of a latent state vector, where the non-linear transform…

Deep learning improves sensor performance optimization.

Financial markets are complex adaptive systems, and are commonly studied as complex networks. Most of such studies fall short in two respects: they do not account for non-linearity of the studied relationships, and they create one network for the whole studied time series, providing an average picture of a very long, e…

We provide several new depth-based separation results for feed-forward neural networks, proving that various types of simple and natural functions can be better approximated using deeper networks than shallower ones, even if the shallower networks are much larger. This includes indicators of balls and ellipses; non-lin…

Learning weights in a spiking neural network with hidden neurons, using local, stable and online rules, to control non-linear body dynamics is an open problem. Here, we employ a supervised scheme, Feedback-based Online Local Learning Of Weights (FOLLOW), to train a network of heterogeneous spiking neurons with hidden l…

This work proposes a mathematical framework for loss landscapes and optimization in deep neural networks.

Paper investigates Lipschitz constants of self-attention modules in neural networks.

Improved texture synthesis using wavelet-based statistics with rectifier non-linearity.

This paper extends semi-structured networks to functional data.

A new method learns state and proposal dynamics in state-space models using neural networks.

Deep networks prioritize easier examples over harder ones, leading to faster training.

New algorithm tackles non-linear utility in MNL bandits with $ilde{O}(\sqrt{T})$ regret.

Activation functions play a key role in neural networks so it becomes fundamental to understand their advantages and disadvantages in order to achieve better performances. This paper will first introduce common types of non linear activation functions that are alternative to the well known sigmoid function and then eva…

KalmanNet uses neural networks to improve state estimation in systems with unknown dynamics.

In the last years efforts in econophysics have been shifted to study how network theory can facilitate understanding of complex financial markets. Main part of these efforts is the study of correlation-based hierarchical networks. This is somewhat surprising as the underlying assumptions of research looking at financia…

New algorithms improve contextual bandits with neural networks and energy models.

RNN-HAR model improves VaR forecasting with long-memory and non-linear dynamics.

This paper explores loss landscapes of sparse neural networks, finding unique characteristics compared to dense networks.

Simplifies deep learning scaling analysis without sacrificing accuracy.

A semi-parametric, non-linear regression model in the presence of latent variables is applied towards learning network graph structure. These latent variables can correspond to unmodeled phenomena or unmeasured agents in a complex system of interacting entities. This formulation jointly estimates non-linearities in the…

New neural network models improve Granger Causality detection in non-linear systems.

We provide an overview of several non-linear activation functions in a neural network architecture that have proven successful in many machine learning applications. We conduct an empirical analysis on the effectiveness of using these function on the MNIST classification task, with the aim of clarifying which functions…

We consider a dynamical model of distress propagation on complex networks, which we apply to the study of financial contagion in networks of banks connected to each other by direct exposures. The model that we consider is an extension of the DebtRank algorithm, recently introduced in the literature. The mechanics of di…

KPCA improves OoD detection by separating InD and OoD data.

Deep neural networks solve noisy, complex problems accurately.

New activation networks improve model efficiency and performance.

Neural networks improve geospatial data analysis by relaxing linearity assumptions.

Two Fisher information matrix estimators are analyzed for neural networks, focusing on their variances and trade-offs.

New method optimizes non-linear functionals over probability measures.