arXiv research

NADS improves OoD detection accuracy by 57%.

This work maps Boltzmann distributions to ARNNs for better physics-based model approximations.

The paper proposes a neural network architecture inspired by Langevin Monte Carlo for sampling from target distributions.

This work proposes searching for optimal operation distribution in neural architecture search.

Study reduces NAS search cost by generating multiple complex architectures in one shot.

This study evaluates different normalizing flow architectures for MCMC.

DrNAS improves neural architecture search with Dirichlet distribution and progressive learning.

In neural architecture search (NAS), the space of neural network architectures is automatically explored to maximize predictive accuracy for a given task. Despite the success of recent approaches, most existing methods cannot be directly applied to large scale problems because of their prohibitive computational complex…

In this work, we perform an exploratory study on synthesizing deep neural networks using biological synaptic strength distributions, and the potential influence of different distributions on modelling performance particularly for the scenario associated with small data sets. Surprisingly, a CNN with convolutional layer…

Paper introduces privacy-preserving deep-learning service.

To achieve a high learning accuracy, generative adversarial networks (GANs) must be fed by large datasets that adequately represent the data space. However, in many scenarios, the available datasets may be limited and distributed across multiple agents, each of which is seeking to learn the distribution of the data on …

This paper studies parallelization schemes for stochastic Vector Quantization algorithms in order to obtain time speed-ups using distributed resources. We show that the most intuitive parallelization scheme does not lead to better performances than the sequential algorithm. Another distributed scheme is therefore intro…

This paper describes a reference architecture for self-maintaining systems that can learn continually, as data arrives. In environments where data evolves, we need architectures that manage Machine Learning (ML) models in production, adapt to shifting data distributions, cope with outliers, retrain when necessary, and …

Distributed learning across a coalition of organizations allows the members of the coalition to train and share a model without sharing the data used to optimize this model. In this paper, we propose new secure architectures that guarantee preservation of data privacy, trustworthy sequence of iterative learning and equ…

Conventional Neural Architecture Search (NAS) aims at finding a single architecture that achieves the best performance, which usually optimizes task related learning objectives such as accuracy. However, a single architecture may not be representative enough for the whole dataset with high diversity and variety. Intuit…

Wide and Deep GNN learns from distributed graphs and retrain online.

The emergence of neural architecture search (NAS) has greatly advanced the research on network design. Recent proposals such as gradient-based methods or one-shot approaches significantly boost the efficiency of NAS. In this paper, we formulate the NAS problem from a Bayesian perspective. We propose explicitly estimati…

Sparse transformer architecture improves accuracy and speed in generative modeling and inverse problems.

We propose Stochastic Neural Architecture Search (SNAS), an economical end-to-end solution to Neural Architecture Search (NAS) that trains neural operation parameters and architecture distribution parameters in same round of back-propagation, while maintaining the completeness and differentiability of the NAS pipeline.…

Study reveals how neural network architectures bias their learning based on feature directions.

Study shows LLMs can extrapolate rules from out-of-distribution prompts.

Researchers analyze neural process architectures and their representational capacities.

This paper distills Bayesian posterior expectations for deep neural networks.

New neural network models extreme value distributions with preserved shape constraints.

Theoretical study on how model architecture affects contrastive learning performance.

FedNAS automates federated learning by searching for better architectures.

This study analyzes NAS benchmarks and finds that only a subset of operations is crucial for generating high-performing architectures.

This research proves guarantees on sequence models' generalization to longer and novel sequences.

We characterize convolutional neural networks with respect to the relative amount of features per layer. Using a skew normal distribution as a parametrized framework, we investigate the common assumption of monotonously increasing feature-counts with higher layers of architecture designs. Our evaluation on models with …

Recently much attention has been paid to deep generative models, since they have been used to great success for variational inference, generation of complex data types, and more. In most all of these settings, the goal has been to find a particular member of that model family: optimized parameters index a distribution …

In this paper we propose a Bayesian method for estimating architectural parameters of neural networks, namely layer size and network depth. We do this by learning concrete distributions over these parameters. Our results show that regular networks with a learnt structure can generalise better on small datasets, while f…

Not all neural network architectures are created equal, some perform much better than others for certain tasks. But how important are the weight parameters of a neural network compared to its architecture? In this work, we question to what extent neural network architectures alone, without learning any weight parameter…

There is a general trend towards solving problems suited to deep learning with more complex deep learning architectures trained on larger training sets. This requires longer compute times and greater data parallelization or model parallelization. Both data and model parallelism have been historically faster in paramete…

FiGS searches over a larger space of architectures for efficient mobile models.

In-network learning outperforms Federated and Split learning in wireless networks.

LeJEPA provides a scalable, theory-driven approach to self-supervised learning.

Can we use deep learning to predict when deep learning works? Our results suggest the affirmative. We created a dataset by training 13,500 neural networks with different architectures, on different variations of spiral datasets, and using different optimization parameters. We used this dataset to train task-independent…

Dida learns meta-features invariant to feature permutations.

Evidential Softmax preserves multimodality in sparse probability distributions for generative models.

Unified framework denoises data and abstains from uncertain predictions.

We study the problem of learning conditional generators from noisy labeled samples, where the labels are corrupted by random noise. A standard training of conditional GANs will not only produce samples with wrong labels, but also generate poor quality samples. We consider two scenarios, depending on whether the noise m…

Investigates deep hedging under rough volatility models.

We propose an alternative generator architecture for generative adversarial networks, borrowing from style transfer literature. The new architecture leads to an automatically learned, unsupervised separation of high-level attributes (e.g., pose and identity when trained on human faces) and stochastic variation in the g…

The effective application of neural networks in the real-world relies on proficiently detecting out-of-distribution examples. Contemporary methods seek to model the distribution of feature activations in the training data for adequately distinguishing abnormalities, and the state-of-the-art method uses Gaussian distrib…

This paper develops a new neural network architecture for modeling spatial distributions (i.e., distributions on R^d) which is computationally efficient and specifically designed to take advantage of the spatial structure of limit order books. The new architecture yields a low-dimensional model of price movements deep …

Generative adversarial networks (GANs) have proven effective in modeling distributions of high-dimensional data. However, their training instability is a well-known hindrance to convergence, which results in practical challenges in their applications to novel data. Furthermore, even when convergence is reached, GANs ca…

Transformers show better in-context learning resilience under distribution shifts than simple MLPs.

When using reinforcement learning (RL) algorithms to evaluate a policy it is common, given a large state space, to introduce some form of approximation architecture for the value function (VF). The exact form of this architecture can have a significant effect on the accuracy of the VF estimate, however, and determining…