arXiv research

Fast BATLLNN speeds up verification of TLL NNs by 400x.

Polynomial-time reachability for LTI systems with TLL NN controllers is achieved.

The paper designs neural networks with assurance for controlling nonlinear systems.

New NN design for nonlinear systems control with guarantees.

We propose a simple approach which, given distributed computing resources, can nearly achieve the accuracy of $k$-NN prediction, while matching (or improving) the faster prediction time of $1$-NN. The approach consists of aggregating denoised $1$-NN predictors over a small number of distributed subsamples. We show, bot…

Randomly trained neural networks can generalize well if there's a simpler underlying teacher model.

Study bridges GARCH and NN models for volatility forecasting.

This paper analyzes deep Stable neural networks, showing convergence rates under different growth settings.

A fast method for LOOCV in k-NN regression reduces computation time.

Adds layers to NNs to protect them from reverse engineering.

Artificial neural networks (NN) are instrumental in realizing highly-automated driving functionality. An overarching challenge is to identify best safety engineering practices for NN and other learning-enabled components. In particular, there is an urgent need for an adequate set of metrics for measuring all-important …

Prototype rules simplify multiclass classification in metric spaces, achieving consistency and reduced complexity.

Neural Networks (NN) have recently emerged as backbone of several sensitive applications like automobile, medical image, security, etc. NNs inherently offer Partial Fault Tolerance (PFT) in their architecture; however, the biased PFT of NNs can lead to severe consequences in applications like cryptography and security …

This work establishes the equivalence between neural networks and support vector machines.

In the $k$-nearest neighborhood model ($k$-NN), we are given a set of points $P$, and we shall answer queries $q$ by returning the $k$ nearest neighbors of $q$ in $P$ according to some metric. This concept is crucial in many areas of data analysis and data processing, e.g., computer vision, document retrieval and machi…

We focus on estimating \emph{a priori} generalization error of two-layer ReLU neural networks (NNs) trained by mean squared error, which only depends on initial parameters and the target function, through the following research line. We first estimate \emph{a priori} generalization error of finite-width two-layer ReLU …

Machine Learning (ML) is making a strong resurgence in tune with the massive generation of unstructured data which in turn requires massive computational resources. Due to the inherently compute- and power-intensive structure of Neural Networks (NNs), hardware accelerators emerge as a promising solution. However, with …

This paper studies large-width asymptotics for ReLU neural networks with α-Stable initializations.

Qualitative analysis of MC dropout for NN model uncertainty.

Training a Neural Network (NN) with lots of parameters or intricate architectures creates undesired phenomena that complicate the optimization process. To address this issue we propose a first modular approach to NN design, wherein the NN is decomposed into a control module and several functional modules, implementing …

The paper optimizes k-NN for distributed learning with minimax optimal performance.

Despite the success of neural networks (NNs), there is still a concern among many over their "black box" nature. Why do they work? Here we present a simple analytic argument that NNs are in fact essentially polynomial regression models. This view will have various implications for NNs, e.g. providing an explanation for…

Neural Networks (NNs) have been extensively used for a wide spectrum of real-world regression tasks, where the goal is to predict a numerical outcome such as revenue, effectiveness, or a quantitative result. In many such tasks, the point prediction is not enough: the uncertainty (i.e. risk or confidence) of that predic…

Study on dropout in neural networks using percolation theory.

$k$-nearest neighbour ($k$-NN) is one of the simplest and most widely-used methods for supervised classification, that predicts a query's label by taking weighted ratio of observed labels of $k$ objects nearest to the query. The weights and the parameter $k \in \mathbb{N}$ regulate its bias-variance trade-off, and the …

We derive high-probability finite-sample uniform rates of consistency for $k$-NN regression that are optimal up to logarithmic factors under mild assumptions. We moreover show that $k$-NN regression adapts to an unknown lower intrinsic dimension automatically. We then apply the $k$-NN regression rates to establish new …

Ensembles of neural networks (NNs) have long been used to estimate predictive uncertainty; a small number of NNs are trained from different initialisations and sometimes on differing versions of the dataset. The variance of the ensemble's predictions is interpreted as its epistemic uncertainty. The appeal of ensembling…

A garland based on a manifold $P$ is a finite set of manifolds homeomorphic to $P$ with some of them glued together at marked points. Fix a manifold $M$ and consider a space $\NN$ of all smooth mappings of garlands based on $P$ into $M$. We construct operations $\bullet$ and $[-,-]$ on the bordism groups $\bor_*(\NN)$ …

Exact bounds derived for neural network outputs with noisy inputs.

The $k$-nearest neighbor classification method ($k$-NNC) is one of the simplest nonparametric classification methods. The mutual $k$-NN classification method (M$k$NNC) is a variant of $k$-NNC based on mutual neighborship. We propose another variant of $k$-NNC, the symmetric $k$-NN classification method (S$k$NNC) based …

Implementing $k$-NN classification using Gromov--Wasserstein distances

The paper certifies neural network-based control barrier functions efficiently.

NNs accurately predict energy eigenvalues and other physical phenomena in 1D quantum mechanics.

We present a formal measure-theoretical theory of neural networks (NN) built on probability coupling theory. Our main contributions are summarized as follows. * Built on the formalism of probability coupling theory, we derive an algorithm framework, named Hierarchical Measure Group and Approximate System (HMGAS), nickn…

This paper uses second-order Poincaré inequalities to establish quantitative central limit theorems for Gaussian neural networks.

The $k$-nearest neighbour ($k$-NN) classifier is one of the oldest and most important supervised learning algorithms for classifying datasets. Traditionally the Euclidean norm is used as the distance for the $k$-NN classifier. In this thesis we investigate the use of alternative distances for the $k$-NN classifier. We …

Adaptive NN method improves matrix completion for non-smooth data.

This paper extends stability and generalization analysis of GD for multi-layer NNs.

Large learning rates cause oscillations in NN weights that improve generalization.

Deep learning models can adaptively estimate functions with varying smoothness using regularization.

The paper proposes a method to train NNs with a small Lipschitz constant to improve robustness.

Develops deep learning for optimizing 5G radio resource allocation.

The paper trains neural networks with robustness guarantees using semidefinite constraints.

Expressiveness and generalization of deep models was recently addressed via the connection between neural networks (NNs) and kernel learning, where first-order dynamics of NN during a gradient-descent (GD) optimization were related to gradient similarity kernel, also known as Neural Tangent Kernel (NTK). In the majorit…

We prove overfitting in minimal and random NNs, tempering the effect.

This work explores using deep NNs to learn quantum systems from probability distributions.

NNs can learn efficient algorithms for certain problems.

A current research trend in neurocomputing involves the design of novel artificial neural networks incorporating the concept of time into their operating model. In this paper, a novel architecture that employs stigmergy is proposed. Computational stigmergy is used to dynamically increase (or decrease) the strength of a…