We propose a simple approach which, given distributed computing resources, can nearly achieve the accuracy of -NN prediction, while matching (or improving) the faster prediction time of -NN. The approach consists of aggregating denoised -NN predictors over a small number of distributed subsamples. We show, bot…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “NNs”
Randomly trained neural networks can generalize well if there's a simpler underlying teacher model.
problem Why randomly trained neural networks generalize well despite interpolating training data.
method Examined a random neural network that interpolates training data and showed it generalizes well if there's a simpler underlying teacher model.
result Randomly trained neural networks can generalize well if there's a simpler underlying teacher model.
Study bridges GARCH and NN models for volatility forecasting.
problem Lack of interaction between GARCH and NN approaches for volatility forecasting.
method Established equivalence between GARCH and NN models, introduced GARCH-NN approach.
result GARCH-NN approach enhances volatility forecasting compared to standalone models.
This paper analyzes deep Stable neural networks, showing convergence rates under different growth settings.
problem Analyzing the behavior of deep Stable neural networks as width increases.
method Large-width asymptotic analysis and convergence rates for fully connected feed-forward deep Stable NNs.
result The rescaled deep Stable NN converges weakly to a Stable SP under joint growth, with sup-norm convergence rates established.
Paper proposes MS-k-NN for improved convergence rate in k-NN classification.
problem Improving convergence rate of k-NN classification methods.
method Proposes MS-k-NN that extrapolates unweighted k-NN estimators to k=0.
result MS-k-NN achieves improved convergence rate under certain conditions.
A fast method for LOOCV in k-NN regression reduces computation time.
problem Efficient computation of LOOCV for k-NN regression.
method Identical LOOCV estimate to (k+1)-NN MSE on training data.
result LOOCV computation can be done with (k+1)-NN regression once.
Adds layers to NNs to protect them from reverse engineering.
problem Extracting the underlying model of a Neural Network.
method Introducing parasitic layers that approximate a noisy identity mapping with a Convolutional NN.
result The protected NN's predictions remain mostly unchanged while making reverse-engineering more complex.
Artificial neural networks (NN) are instrumental in realizing highly-automated driving functionality. An overarching challenge is to identify best safety engineering practices for NN and other learning-enabled components. In particular, there is an urgent need for an adequate set of metrics for measuring all-important …
Universal consistency and rates of convergence of multiclass prototype algorithms in metric spacescs.LG
Prototype rules simplify multiclass classification in metric spaces, achieving consistency and reduced complexity.
problem Multiclass classification in metric spaces, focusing on universal consistency and convergence rates.
method Novel Proto-NN and hybrid rules for multiclass classification in metric spaces, analyzing convergence rates.
result Proto-NN is universally consistent and simpler to implement, with similar computational advantages.
Neural Networks (NN) have recently emerged as backbone of several sensitive applications like automobile, medical image, security, etc. NNs inherently offer Partial Fault Tolerance (PFT) in their architecture; however, the biased PFT of NNs can lead to severe consequences in applications like cryptography and security …
This work establishes the equivalence between neural networks and support vector machines.
problem Establishing the equivalence between neural networks and support vector machines.
method Proposed a method to establish the equivalence between infinitely wide neural networks trained by soft margin loss and standard soft margin SVMs with NTK trained by subgradient descent.
result The equivalence between NN and SVM is established, enabling practical applications such as non-vacuous generalization bounds and robustness certificates.
In the -nearest neighborhood model (-NN), we are given a set of points , and we shall answer queries by returning the nearest neighbors of in according to some metric. This concept is crucial in many areas of data analysis and data processing, e.g., computer vision, document retrieval and machi…
We focus on estimating \emph{a priori} generalization error of two-layer ReLU neural networks (NNs) trained by mean squared error, which only depends on initial parameters and the target function, through the following research line. We first estimate \emph{a priori} generalization error of finite-width two-layer ReLU …
Machine Learning (ML) is making a strong resurgence in tune with the massive generation of unstructured data which in turn requires massive computational resources. Due to the inherently compute- and power-intensive structure of Neural Networks (NNs), hardware accelerators emerge as a promising solution. However, with …
This paper studies large-width asymptotics for ReLU neural networks with α-Stable initializations.
problem Characterizing the large-width behavior of ReLU neural networks with α-Stable initializations.
method Analysis of the large-width distributions and training dynamics of ReLU neural networks initialized with α-Stable distributions.
result For ReLU neural networks with α-Stable initializations, the large-width training dynamics achieve zero training error at a linear rate, characterized by a random kernel.
Qualitative analysis of MC dropout for NN model uncertainty.
problem Measuring uncertainty in neural network models.
method Mathematical formulation of Monte Carlo dropout and its benefits/costs in NN models.
result Potential benefits and associated costs of using MC dropout in NN models.
Training a Neural Network (NN) with lots of parameters or intricate architectures creates undesired phenomena that complicate the optimization process. To address this issue we propose a first modular approach to NN design, wherein the NN is decomposed into a control module and several functional modules, implementing …
The paper optimizes k-NN for distributed learning with minimax optimal performance.
problem Minimizing error rates in classification, regression, and density estimation.
method Optimal aggregation of fixed-k nearest neighbors from multiple subsets of data.
result Achieves minimax optimal error rates up to a logarithmic factor.
Despite the success of neural networks (NNs), there is still a concern among many over their "black box" nature. Why do they work? Here we present a simple analytic argument that NNs are in fact essentially polynomial regression models. This view will have various implications for NNs, e.g. providing an explanation for…
Neural Networks (NNs) have been extensively used for a wide spectrum of real-world regression tasks, where the goal is to predict a numerical outcome such as revenue, effectiveness, or a quantitative result. In many such tasks, the point prediction is not enough: the uncertainty (i.e. risk or confidence) of that predic…
Study on dropout in neural networks using percolation theory.
problem Understanding dropout's effect on neural network training.
method Investigates percolation models mimicking dropout in neural networks.
result Dropout can cause a breakdown in neural networks.
We derive high-probability finite-sample uniform rates of consistency for -NN regression that are optimal up to logarithmic factors under mild assumptions. We moreover show that -NN regression adapts to an unknown lower intrinsic dimension automatically. We then apply the -NN regression rates to establish new …
Ensembles of neural networks (NNs) have long been used to estimate predictive uncertainty; a small number of NNs are trained from different initialisations and sometimes on differing versions of the dataset. The variance of the ensemble's predictions is interpreted as its epistemic uncertainty. The appeal of ensembling…
A garland based on a manifold is a finite set of manifolds homeomorphic to with some of them glued together at marked points. Fix a manifold and consider a space $\NN$ of all smooth mappings of garlands based on into . We construct operations and on the bordism groups $\bor_*(\NN)$ …
Exact bounds derived for neural network outputs with noisy inputs.
problem Bounding the output distribution of neural networks with random inputs.
method Applying ReLU NNs to derive bounds for general NNs, then using these to find exact error guarantees.
result Exact upper and lower bounds for the output distribution of neural networks with random inputs.
The -nearest neighbor classification method (-NNC) is one of the simplest nonparametric classification methods. The mutual -NN classification method (MNNC) is a variant of -NNC based on mutual neighborship. We propose another variant of -NNC, the symmetric -NN classification method (SNNC) based …
Implementing -NN classification using Gromov--Wasserstein distances
problem Comparing metric measure spaces
method Gromov--Wasserstein and fused Gromov--Wasserstein distances
result Universal consistency of -NN classifiers
The paper certifies neural network-based control barrier functions efficiently.
problem Certifying neural network-based barrier functions for safety in autonomous systems.
method Combines NN reachability and hyperplane arrangement enumeration for efficient certification.
result Soundly finds regions where neural networks are certified as barrier functions.
Observing how deep neural networks understand physics through the energy spectrum of one-dimensional quantum mechanicsphysics.comp-ph
NNs accurately predict energy eigenvalues and other physical phenomena in 1D quantum mechanics.
problem Understanding how neural networks interpret physics.
method Training NNs to predict energy eigenvalues from potentials and testing their ability to generalize.
result NNs can predict physical phenomena not learned during training, indicating a new way of understanding physics.
We present a formal measure-theoretical theory of neural networks (NN) built on probability coupling theory. Our main contributions are summarized as follows. * Built on the formalism of probability coupling theory, we derive an algorithm framework, named Hierarchical Measure Group and Approximate System (HMGAS), nickn…
Non-asymptotic approximations of Gaussian neural networks via second-order Poincaré inequalitiescs.LG
This paper uses second-order Poincaré inequalities to establish quantitative central limit theorems for Gaussian neural networks.
problem Establishing quantitative central limit theorems for Gaussian neural networks.
method Using second-order Poincaré inequalities to reduce the problem to computing the gradient and Hessian of the NN's output.
result Suboptimal rates of convergence for the NN's output due to the use of second-order Poincaré inequalities.
Fast BATLLNN speeds up verification of TLL NNs by 400x.
problem Verifying output constraints for TLL NNs.
method Uses TLL architecture and decoupled box constraints to improve verification performance.
result 400x faster than state-of-the-art verifiers.
The -nearest neighbour (-NN) classifier is one of the oldest and most important supervised learning algorithms for classifying datasets. Traditionally the Euclidean norm is used as the distance for the -NN classifier. In this thesis we investigate the use of alternative distances for the -NN classifier. We …
Adaptive NN method improves matrix completion for non-smooth data.
problem Matrix completion with non-smooth non-linear functions under high missingness.
method Two-sided nearest neighbors with \Holder function class non-linearity.
result NN error rate matches oracle's for latent factors, non-trivial for wide range of missingness.
This paper extends stability and generalization analysis of GD for multi-layer NNs.
problem Understanding the generalization of multi-layer neural networks trained by GD.
method Comprehensive stability and generalization analysis of GD for multi-layer NNs, focusing on two-layer and three-layer networks.
result Derives excess risk rates of for GD in two-layer and three-layer NNs under specific conditions.
Large learning rates cause oscillations in NN weights that improve generalization.
problem Improving generalization of neural networks trained with large learning rates.
method Theoretical analysis and feature-noise data generation model.
result Oscillating SGD with large learning rates benefits NN generalization by effectively learning weak features.
New NN design for nonlinear systems control with guarantees.
problem Designing NN architectures for nonlinear system control with guarantees.
method Exploits system model to design NN architecture, uses TLL NN for approximation.
result Guaranteed NN architecture sufficient for implementing a controller.
Deep learning models can adaptively estimate functions with varying smoothness using regularization.
problem Estimating functions with heterogeneous smoothness in Besov or BV classes.
method Introduced a Parallel NN variant of deep ReLU networks with regularization equivalent to promoting -sparsity.
result Achieves minimax rates for Besov and BV classes with exponentially closer performance as depth increases.
The paper proposes a method to train NNs with a small Lipschitz constant to improve robustness.
problem Neural networks' susceptibility to adversarial perturbations in safety-critical applications.
method The paper introduces a framework to train multi-layer NNs by minimizing their Lipschitz constant, using an optimization scheme based on the Alternating Direction Method of Multipliers.
result The proposed training procedure successfully increases the robustness of neural networks.
Deep Learning for Radio Resource Allocation with Diverse Quality-of-Service Requirements in 5Geess.SP
Develops deep learning for optimizing 5G radio resource allocation.
problem Optimizing 5G base station radio resources for diverse QoS requirements.
method Cascaded neural network structure with deep transfer learning for non-stationary conditions.
result Cascaded neural networks outperform fully connected neural networks in QoS guarantee.
The paper trains neural networks with robustness guarantees using semidefinite constraints.
problem Training neural networks with robustness and stability guarantees.
method Exploiting the banded structure of semidefinite constraints, an efficient and scalable training scheme based on interior point methods is set up.
result The method allows for enforcing Lipschitz constraints in large-scale deep neural networks, as demonstrated in numerical examples.
Expressiveness and generalization of deep models was recently addressed via the connection between neural networks (NNs) and kernel learning, where first-order dynamics of NN during a gradient-descent (GD) optimization were related to gradient similarity kernel, also known as Neural Tangent Kernel (NTK). In the majorit…
We prove overfitting in minimal and random NNs, tempering the effect.
problem Overfitting in minimal and random neural networks.
method Analyzing binary weight fitting to noisy data, proving overfitting is tempered.
result The overfitting of minimal and random neural networks is tempered.
This work explores using deep NNs to learn quantum systems from probability distributions.
problem Learning quantum systems from limited probability distribution data.
method Using deep neural networks to reconstruct quantum Hamiltonian from probability distributions.
result Deep neural networks can learn quantum Hamiltonians from probability distributions.
NNs can learn efficient algorithms for certain problems.
problem Learning efficient algorithms for specific problems.
method Recurrent Convolutional Neural Networks (RCNNs) that learn efficiently.
result RCNNs can learn as well as efficient algorithms described by a constant-sized program.
A current research trend in neurocomputing involves the design of novel artificial neural networks incorporating the concept of time into their operating model. In this paper, a novel architecture that employs stigmergy is proposed. Computational stigmergy is used to dynamically increase (or decrease) the strength of a…
Study shows neural networks learn low frequencies first, proposing solutions.
problem Frequency bias in neural network learning process.
method Developed a PDE to unravel frequency dynamics, used Fourier Features model.
result Appropriate weight initialization can eliminate or control frequency bias.
There has been a recent surge of interest in modeling neural networks (NNs) as Gaussian processes. In the limit of a NN of infinite width the NN becomes equivalent to a Gaussian process. Here we demonstrate that for an ensemble of large, finite, fully connected networks with a single hidden layer the distribution of ou…