arXiv research

Paper proposes LANN to measure model complexity of neural networks with curve activation functions.

Gaussian Mixture Models (GMM) have found many applications in density estimation and data clustering. However, the model does not adapt well to curved and strongly nonlinear data. Recently there appeared an improvement called AcaGMM (Active curve axis Gaussian Mixture Model), which fits Gaussians along curves using an …

Fragility curves which express the failure probability of a structure, or critical components, as function of a loading intensity measure are nowadays widely used (i) in Seismic Probabilistic Risk Assessment studies, (ii) to evaluate impact of construction details on the structural performance of installations under se…

Neural network models improve ROC curve evaluation of biomarkers, focusing on age's role in physical activity-mortality association.

Study active nematic forces on curved surfaces, revealing new coupling mechanisms.

This paper explores adaptive neural activation in RNNs for better learning.

Develops active intervals for geodesics in Teichmüller space.

Optimal AFs minimize RFR test error and sensitivity.

Activation functions influence behavior and performance of DNNs. Nonlinear activation functions, like Rectified Linear Units (ReLU), Exponential Linear Units (ELU) and Scaled Exponential Linear Units (SELU), outperform the linear counterparts. However, selecting an appropriate activation function is a challenging probl…

Adaptive pricing framework for perpetual contracts using liquidity curves and oracles.

Improved neural population modeling using shared features and ensemble detection.

We present an approach to analyze $C^1(\mathbb{R}^m)$ functions that addresses limitations present in the Active Subspaces (AS) method of Constantine et al.(2015; 2014). Under appropriate hypotheses, our Active Manifolds (AM) method identifies a 1-D curve in the domain (the active manifold) on which nearly all values o…

Logistic regression is by far the most widely used classifier in real-world applications. In this paper, we benchmark the state-of-the-art active learning methods for logistic regression and discuss and illustrate their underlying characteristics. Experiments are carried out on three synthetic datasets and 44 real-worl…

Classifies Bitcoin addresses based on their balance functions.

Machine learning classifies complex geometric patterns with high accuracy.

PA-AMM divides reserves into active and passive parts for better liquidity provider wealth.

Active learning method optimizes seismic fragility curve estimation.

Catastrophic forgetting is a problem faced by many machine learning models and algorithms. When trained on one task, then trained on a second task, many machine learning models "forget" how to perform the first task. This is widely believed to be a serious problem for neural networks. Here, we investigate the extent to…

Deep neural networks enforce non-crossing quantile regression curves.

Several studies have established the predictive power of the yield curve in terms of real economic activity. In this paper we use data for a variety of E.U. countries: both EMU (Germany, France, Italy) and non-EMU members (Sweden and the U.K.). The data used range from 1991:Q1 to 2009:Q1. For each country, we extract t…

SAEs struggle with curved activation manifolds, revealing layer-dependent scaling laws.

In this paper we study the volatility and its probability distribution function for the cumulative production based on the experience curve hypothesis. This work presents a generalization of the study of volatility in [1], which addressed the effects of normally distributed noise in the production process. Due to its w…

Two-layer neural networks can approximate functions with fractal singularities.

We study layered neural networks of rectified linear units (ReLU) in a modelling framework for stochastic training processes. The comparison with sigmoidal activation functions is in the center of interest. We compute typical learning curves for shallow networks with K hidden units in matching student teacher scenarios…

Construct intrinsic Langevin dynamics for rigid inclusions on curved surfaces.

We introduce the functional mean-shift algorithm, an iterative algorithm for estimating the local modes of a surrogate density from functional data. We show that the algorithm can be used for cluster analysis of functional data. We propose a test based on the bootstrap for the significance of the estimated local modes …

New dynamic curves improve cryptocurrency exchange liquidity.

Model shows loss curve with two distinct exponents due to sparse activations.

Many neural network architectures rely on the choice of the activation function for each hidden layer. Given the activation function, the neural network is trained over the bias and the weight parameters. The bias catches the center of the activation, and the weights capture the scale. Here we propose to train the netw…

We propose to describe the variety of galaxies from SDSS by using only one affine parameter. To this aim, we build the Principal Curve (P-curve) passing through the spine of the data point cloud, considering the eigenspace derived from Principal Component Analysis of morphological, physical and photometric galaxy prope…

Bayesian approach improves uncertainty in deep learning models.

The choice of activation function can have a large effect on the performance of a neural network. While there have been some attempts to hand-engineer novel activation functions, the Rectified Linear Unit (ReLU) remains the most commonly-used in practice. This paper shows that evolutionary algorithms can discover novel…

Survey of trainable activation functions in neural networks.

Many activation functions have been proposed in the past, but selecting an adequate one requires trial and error. We propose a new methodology of designing activation functions within a neural network at each layer. We call this technique an "activation ensemble" because it allows the use of multiple activation functio…

Data-aware activation function customization reduces neural network error.

This paper provides an overview of activation functions in neural networks.

There has been a growing interest in expressivity of deep neural networks. However, most of the existing work about this topic focuses only on the specific activation function such as ReLU or sigmoid. In this paper, we investigate the approximation ability of deep neural networks with a broad class of activation functi…

We present the first evidence that adaptive learning techniques can boost the discovery of unusual objects within astronomical light curve data sets. Our method follows an active learning strategy where the learning algorithm chooses objects which can potentially improve the learner if additional information about them…

Large deviation principle for deep neural networks with ReLU activation.

The most widely used activation functions in current deep feed-forward neural networks are rectified linear units (ReLU), and many alternatives have been successfully applied, as well. However, none of the alternatives have managed to consistently outperform the rest and there is no unified theory connecting properties…

Study embeddings between Barron spaces with various activation functions, focusing on RePU.

Automatically discovers effective activation functions for deep learning.

Study disproves conjecture about metric completion of curve spaces.

Study shows how activation functions impact the storage capacity of treelike neural networks.

Deep Neural Networks have been shown to be beneficial for a variety of tasks, in particular allowing for end-to-end learning and reducing the requirement for manual design decisions. However, still many parameters have to be chosen in advance, also raising the need to optimize them. One important, but often ignored sys…

A new active learning method for skewed data sets.

Paper examines power consumption in neural networks using various activation functions.

Deep networks can approximate various activation functions with modest adjustments.