arXiv research

Paper introduces MNNs for efficient neural network training.

We propose a novel approach for nonlinear regression using a two-layer neural network (NN) model structure with sparsity-favoring hierarchical priors on the network weights. We present an expectation propagation (EP) approach for approximate integration over the posterior distribution of the weights, the hierarchical s…

Meta-ticket finds optimal sparse subnetworks for few-shot learning in randomly initialized neural networks.

This paper studies large-width asymptotics for ReLU neural networks with α-Stable initializations.

In this paper, we present some theoretical work to explain why simple gradient descent methods are so successful in solving non-convex optimization problems in learning large-scale neural networks (NN). After introducing a mathematical tool called canonical space, we have proved that the objective functions in learning…

The $k$-NN graph has played a central role in increasingly popular data-driven techniques for various learning and vision tasks; yet, finding an efficient and effective way to construct $k$-NN graphs remains a challenge, especially for large-scale high-dimensional data. In this paper, we propose a new approach to const…

We describe tests validating progress made toward acceleration and automation of hydrodynamic codes in the regime of developed turbulence by three Deep Learning (DL) Neural Network (NN) schemes trained on Direct Numerical Simulations of turbulence. Even the bare DL solutions, which do not take into account any physics …

SOLAR improves search efficiency and accuracy with sparse, orthogonal embeddings.

A new method for Bayesian neural networks using probabilistic backpropagation.

The paper trains neural networks with robustness guarantees using semidefinite constraints.

Unified framework evaluates different nearest neighbor classification methods.

Study shows SNN graph Laplacians converge to k-NN graph Laplacians under large scale asymptotics.

Sig-PCA integrates model outputs and observations to correct model biases.

An ensemble of neural networks is known to be more robust and accurate than an individual network, however usually with linearly-increased cost in both training and testing. In this work, we propose a two-stage method to learn Sparse Structured Ensembles (SSEs) for neural networks. In the first stage, we run SG-MCMC wi…

Wide neural networks can outperform kernel methods in certain tasks.

Neural networks compress and sample WDN contamination dynamics efficiently.

Paper reveals hidden convexities in deep learning models using sparse signal processing.

New insights into why neural networks generalize well.

Study of deep Stable neural networks with various activation functions.

Finite-width neural networks are approximated by Gaussian processes with finite size corrections.

PAC-Bayesian bounds improve understanding of K-NN classifier performance.

A novel k-NN method estimates conditional mean and variance efficiently.

RIO estimates uncertainty in NN predictions without changing model architecture.

RS-NN predicts belief functions for classification, improving accuracy and uncertainty estimation.

The eigendeomposition of nearest-neighbor (NN) graph Laplacian matrices is the main computational bottleneck in spectral clustering. In this work, we introduce a highly-scalable, spectrum-preserving graph sparsification algorithm that enables to build ultra-sparse NN (u-NN) graphs with guaranteed preservation of the or…

NN-GPR improves climate model predictions by preserving fine-scale spatial information.

Paper models entropy-based impact of soft errors on neural network inference.

SpArX creates faithful explanations of neural networks' decision-making.

Online method learns sparse models efficiently in large scale settings.

Correspondence analysis (CA) is a multivariate statistical tool used to visualize and interpret data dependencies. CA has found applications in fields ranging from epidemiology to social sciences. However, current methods used to perform CA do not scale to large, high-dimensional datasets. By re-interpreting the object…

Recommender systems leverage product and community information to target products to consumers. Researchers have developed collaborative recommenders, content-based recommenders, and (largely ad-hoc) hybrid systems. We propose a unified probabilistic framework for merging collaborative and content-based recommendations…

This paper extends stability and generalization analysis of GD for multi-layer NNs.

We present a formal measure-theoretical theory of neural networks (NN) built on probability coupling theory. Our main contributions are summarized as follows. * Built on the formalism of probability coupling theory, we derive an algorithm framework, named Hierarchical Measure Group and Approximate System (HMGAS), nickn…

Adaptive regularization prevents overfitting in large-scale sparse feature models.

Method converts neural networks to function space for better uncertainty quantification.

Unions of subspaces provide a powerful generalization to linear subspace models for collections of high-dimensional data. To learn a union of subspaces from a collection of data, sets of signals in the collection that belong to the same subspace must be identified in order to obtain accurate estimates of the subspace s…

Large learning rates cause oscillations in NN weights that improve generalization.

Improved neural network models predict molecular and material properties efficiently.

Many modern big data applications feature large scale in both numbers of responses and predictors. Better statistical efficiency and scientific insights can be enabled by understanding the large-scale response-predictor association network structures via layers of sparse latent factors ranked by importance. Yet sparsit…

Proposes a sparse classifier for discriminative Gaussian Mixture Models.

DES training speeds up large-scale recommender systems convergence.

This paper studies the large sample asymptotics of data analysis procedures based on the optimization of functionals defined on $k$-NN graphs on point clouds. The paper is framed in the context of minimization of balanced cut functionals, but our techniques, ideas and results can be adapted to other functionals of rele…

This paper analyzes deep Stable neural networks, showing convergence rates under different growth settings.

A fast method for LOOCV in k-NN regression reduces computation time.

Sparse representations using data dictionaries provide an efficient model particularly for signals that do not enjoy alternate analytic sparsifying transformations. However, solving inverse problems with sparsifying dictionaries can be computationally expensive, especially when the dictionary under consideration has a …

Deep learning using multi-layer neural networks (NNs) architecture manifests superb power in modern machine learning systems. The trained Deep Neural Networks (DNNs) are typically large. The question we would like to address is whether it is possible to simplify the NN during training process to achieve a reasonable pe…

New method reduces SBL complexity from cubic to linear, improving scalability.

Efficiently solves large-scale SVMs with sparse semismooth Newton method.