arXiv research

A new method for target propagation using iterative approximations converges fast and is more biologically plausible.

GAIT-prop derives a biologically plausible learning rule from backpropagation.

Theoretical framework for target propagation shows differences from backpropagation.

Uniform-in-time analysis for Stein Variational Gradient Descent across various metrics.

Finding biologically plausible alternatives to back-propagation of errors is a fundamentally important challenge in artificial neural network research. In this paper, we propose a learning algorithm called error-driven Local Representation Alignment (LRA-E), which has strong connections to predictive coding, a theory t…

NoProp learns neural networks without full back-propagation or forward-propagation.

Proposes a label propagation framework for domain adaptation.

A novel method to propagate uncertainty through the soft-thresholding nonlinearity is proposed in this paper. At every layer the current distribution of the target vector is represented as a spike and slab distribution, which represents the probabilities of each variable being zero, or Gaussian-distributed. Using the p…

Paper proposes methods to reduce financial contagion by targeted cash injections.

We study the propagation of bosonic strings in singular target space-times. For describing this, we assume this target space to be the quotient of a smooth manifold $M$ by a singular foliation ${\cal F}$ on it. Using the technical tool of a gauge theory, we propose a smooth functional for this scenario, such that the p…

Defines a calculus for integrating Moreau envelopes in differentiable programming.

Proposes a new method to estimate individual treatment effects using unlabeled data.

We present the Network-based Biased Tree Ensembles (NetBiTE) method for drug sensitivity prediction and drug sensitivity biomarker identification in cancer using a combination of prior knowledge and gene expression data. Our devised method consists of a biased tree ensemble that is built according to a probabilistic bi…

Accelerates DNN robustness verification with target labels.

Study eigenvalues and eigenvectors in neural networks, focusing on signal propagation.

Multi-layered representation is believed to be the key ingredient of deep neural networks especially in cognitive tasks like computer vision. While non-differentiable models such as gradient boosting decision trees (GBDTs) are the dominant methods for modeling discrete or tabular data, they are hard to incorporate with…

Using back-propagation and its variants to train deep networks is often problematic for new users. Issues such as exploding gradients, vanishing gradients, and high sensitivity to weight initialization strategies often make networks difficult to train, especially when users are experimenting with new architectures. Her…

In this study, we propose a novel deep neural network and its supervised learning method that uses a feedforward supervisory signal. The method is inspired by the human visual system and performs human-like association-based learning without any backward error propagation. The feedforward supervisory signal that produc…

Transfer learning is a very important tool in deep learning as it allows propagating information from one "source dataset" to another "target dataset", especially in the case of a small number of training examples in the latter. Yet, discrepancies between the underlying distributions of the source and target data are c…

Deep networks have been successfully applied to learn transferable features for adapting models from a source domain to a different target domain. In this paper, we present joint adaptation networks (JAN), which learn a transfer network by aligning the joint distributions of multiple domain-specific layers across domai…

In this paper, we introduce a novel concept for learning of the parameters in a neural network. Our idea is grounded on modeling a learning problem that addresses a trade-off between (i) satisfying local objectives at each node and (ii) achieving desired data propagation through the network under (iii) local propagatio…

Proposes a method to select features for subgroup datasets with systematic missing data.

Backpropagation is the workhorse of deep learning, however, several other biologically-motivated learning rules have been introduced, such as random feedback alignment and difference target propagation. None of these methods have produced a competitive performance against backpropagation. In this paper, we show that bi…

Bayesian inference is a popular method to build learning algorithms but it is hampered by the fact that its key object, the posterior probability distribution, is often uncomputable. Expectation Propagation (EP) (Minka (2001)) is a popular algorithm that solves this issue by computing a parametric approximation (e.g: G…

The paper studies distributions and controllability in quantum mechanical systems.

This paper makes two contributions to Bayesian machine learning algorithms. Firstly, we propose stochastic natural gradient expectation propagation (SNEP), a novel alternative to expectation propagation (EP), a popular variational inference algorithm. SNEP is a black box variational algorithm, in that it does not requi…

The backpropagation of error algorithm (BP) is impossible to implement in a real brain. The recent success of deep networks in machine learning and AI, however, has inspired proposals for understanding how the brain might learn across multiple layers, and hence how it might approximate BP. As of yet, none of these prop…

A variety of machine learning applications expect to achieve rapid learning from a limited number of labeled data. However, the success of most current models is the result of heavy training on big data. Meta-learning addresses this problem by extracting common knowledge across different tasks that can be quickly adapt…

Bandit is a framework for designing sequential experiments. In each experiment, a learner selects an arm $A \in \mathcal{A}$ and obtains an observation corresponding to $A$. Theoretically, the tight regret lower-bound for the general bandit is polynomial with respect to the number of arms $|\mathcal{A}|$. This makes ba…

Improves hyperparameter learning in GP models with non-conjugate likelihoods.

The 2008 financial crisis revealed banking consolidation paradoxically increased systemic fragility and global financial contagion with negligible spatial decay.

Adversarial examples have recently proven to be able to fool deep learning methods by adding carefully crafted small perturbation to the input space image. In this paper, we study the possibility of generating adversarial examples for code-based iris recognition systems. Since generating adversarial examples requires b…

One way to recognise an object is to study how the echo has been shaped during the interaction with the target. Wideband sonar allows the study of the energy distribution for a large range of frequencies. The frequency distribution contains information about an object, including its inner structure. This information is…

NeVI-Cut uses neural networks to efficiently propagate uncertainty without feedback.

NLE embeds labels for domain adaptation with neural networks.

New method accelerates Parallel Tempering using neural samplers.

EP learns like BPTT but with local weight updates.

Generalized belief propagation converges to optimal solutions on graphs with motifs.

Wide neural networks learn features under $μ$P, identifying weights and decomposing support.

A new MCMC method combines low and high-fidelity models to reduce computation.

The paper analyzes how synthetic data training degrades diffusion models, providing bounds and characterizing different drift regimes.

Belief propagation recovers backpropagation results.

A susceptibility propagation that is constructed by combining a belief propagation and a linear response method is used for approximate computation for Markov random fields. Herein, we formulate a new, improved susceptibility propagation by using the concept of a diagonal matching method that is based on mean-field app…

We consider probabilistic multinomial probit classification using Gaussian process (GP) priors. The challenges with the multiclass GP classification are the integration over the non-Gaussian posterior distribution, and the increase of the number of unknown latent variables as the number of target classes grows. Expecta…

ReLU networks trained with MILPs match deep learning accuracy.

Zero-Copy Architecture Detects Cross-Company Financial Signals Instantly.

WAEs offer a statistical understanding of density estimation and error bounds.

Variational inference is a powerful concept that underlies many iterative approximation algorithms; expectation propagation, mean-field methods and belief propagations were all central themes at the school that can be perceived from this unifying framework. The lectures of Manfred Opper introduce the archetypal example…