A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
HyCNNs improve convex function learning and optimal transport.
Maxout networks study gradients and propose initialization strategies.
We present a probabilistic variant of the recently introduced maxout unit. The success of deep neural networks utilizing maxout can partly be attributed to favorable performance under dropout, when compared to rectified linear units. It however also depends on the fact that each maxout unit performs a pooling operation…
Expands learning paradigm to stochastic orders using Choquet-Toland distance and Variational Dominance Criterion.
Study deep maxout networks and their equivalence to Gaussian processes.
We consider the problem of designing models to leverage a recently introduced approximate model averaging technique called dropout. We define a simple new model called maxout (so named because its output is the max of a set of inputs, and because it is a natural companion to dropout) designed to both facilitate optimiz…
Maxout networks show similar complexity issues as ReLU networks.
In this paper, we propose and study random maxout features, which are constructed by first projecting the input data onto sets of randomly generated vectors with Gaussian elements, and then outputing the maximum projection value for each set. We show that the resulting random feature map, when used in conjunction with …
Motivated by an important insight from neural science, we propose a new framework for understanding the success of the recently proposed "maxout" networks. The framework is based on encoding information on sparse pathways and recognizing the correct pathway at inference time. Elaborating further on this insight, we pro…
Several machine learning models, including neural networks, consistently misclassify adversarial examples---inputs formed by applying small but intentionally worst-case perturbations to examples from the dataset, such that the perturbed input results in the model outputting an incorrect answer with high confidence. Ear…
We investigate the complexity of deep neural networks (DNN) that represent piecewise linear (PWL) functions. In particular, we study the number of linear regions, i.e. pieces, that a PWL function represented by a DNN can attain, both theoretically and empirically. We present (i) tighter upper and lower bounds for the m…
We study the complexity of functions computable by deep feedforward neural networks with piecewise linear activations in terms of the symmetries and the number of linear regions that they have. Deep networks are able to sequentially map portions of each layer's input-space to the same output. In this way, deep models c…
Methods from convex optimization are widely used as building blocks for deep learning algorithms. However, the reasons for their empirical success are unclear, since modern convolutional networks (convnets), incorporating rectifier units and max-pooling, are neither smooth nor convex. Standard guarantees therefore do n…
Catastrophic forgetting of connectionist neural networks is caused by the global sharing of parameters among all training examples. In this study, we analyze parameter sharing under the conditional computation framework where the parameters of a neural network are conditioned on each input example. At one extreme, if e…
Deep neural-kernel models combine neural networks and kernel machines for scalable large datasets.
IENs reduce neural network variance without increasing complexity.
This work generalizes bounds on the number of linear regions in CPWL NNs.
We present a new, unifying approach following some recent developments on the complexity of neural networks with piecewise linear activations. We treat neural network layers with piecewise linear activations as tropical polynomials, which generalize polynomials in the so-called or tropical algebra, with pos…
The back-propagation algorithm is widely used for learning in artificial neural networks. A challenge in machine learning is to create models that generalize to new data samples not seen in the training data. Recently, a common flaw in several machine learning algorithms was discovered: small perturbations added to the…
New neural network approximates convex option prices.
Presented are two neural network architectures for convex functions, demonstrating competitive performance.
Paper introduces ICGNs to model convex gradients.
New weight initialisation for ICNNs accelerates learning and improves generalization.
New method improves neural network verification by considering multivariate input space of ReLU neurons.
Wide and shallow networks approximate convex functions well.
Adversarial examples are augmented data points generated by imperceptible perturbation of input samples. They have recently drawn much attention with the machine learning and data mining community. Being difficult to distinguish from real examples, such adversarial examples could change the prediction of many of the be…
Convex neural networks enforce convex constraints on weights and activations, improving generalization.
Following recent advances in morphological neural networks, we propose to study in more depth how Max-plus operators can be exploited to define morphological units and how they behave when incorporated in layers of conventional neural networks. Besides showing that they can be easily implemented with modern machine lea…
Proposes a differentiable LSE-ICNN for modeling multi-well potentials.
In this paper, we present a novel and principled approach to learn the optimal transport between two distributions, from samples. Guided by the optimal transport theory, we learn the optimal Kantorovich potential which induces the optimal transport map. This involves learning two convex functions, by solving a novel mi…
A scalable algorithm approximates Wasserstein Barycenters using neural networks.
New method certifies neural network robustness under random input noise.
We propose to optimize the activation functions of a deep neural network by adding a corresponding functional regularization to the cost function. We justify the use of a second-order total-variation criterion. This allows us to derive a general representer theorem for deep neural networks that makes a direct connectio…
Paper proposes a new model for better engine control.
The paper develops mixed-integer formulations for neural networks using partitioning.
A new method lifts training of input-convex neural networks to avoid dead weights and plateaued loss.
Paper develops exact convex optimization for neural networks with polynomial activations.
SOC-ICNN expands neural network representational capacity by using conic optimization.
Safe offline RL for chemical reactors using input convex neural networks.
We give a new algorithm for learning a two-layer neural network under a general class of input distributions. Assuming there is a ground-truth two-layer network where are weight matrices, represents noise, and the number of neurons in the hidden layer is no larger than the input or outp…
We develop a convex relaxation method for analyzing neural network generalization.
Scalable algorithm for computing Wasserstein-2 barycenters without bias.
We develop a fast, tractable technique called Net-Trim for simplifying a trained neural network. The method is a convex post-processing module, which prunes (sparsifies) a trained network layer by layer, while preserving the internal responses. We present a comprehensive analysis of Net-Trim from both the algorithmic a…
Optimizes functionals on probability space using ICNNs.
New method approximates neural network training for robustness.
Recently, Convolution Neural Networks (CNNs) obtained huge success in numerous vision tasks. In particular, DenseNets have demonstrated that feature reuse via dense skip connections can effectively alleviate the difficulty of training very deep networks and that reusing features generated by the initial layers in all s…
Neural networks solve the Dirichlet problem for Monge-Ampère equations.
We consider deep linear networks with arbitrary convex differentiable loss. We provide a short and elementary proof of the fact that all local minima are global minima if the hidden layers are either 1) at least as wide as the input layer, or 2) at least as wide as the output layer. This result is the strongest possibl…