arXiv research

Inner product-based convolution has been a central component of convolutional neural networks (CNNs) and the key to learning visual representations. Inspired by the observation that CNN-learned features are naturally decoupled with the norm of features corresponding to the intra-class variation and the angle correspond…

Spectral decoupling improves neural network generalization in medical imaging.

New method combines spectral and sparse methods for Gaussian processes.

Study shows how large neural networks avoid overfitting through decoupling of feature learning and complexity growth.

This work studies how contrastive learning extracts features from unlabeled data.

Decoupled GCN is shown to be equivalent to label propagation.

Kahler geometry explains decoupling of Kerr perturbations.

A new loss function HUG decouples and generalizes neural collapse.

One question central to Reinforcement Learning is how to learn a feature representation that supports algorithm scaling and re-use of learned information from different tasks. Successor Features approach this problem by learning a feature representation that satisfies a temporal constraint. We present an implementation…

PerPCA separates unique and shared features from heterogeneous data.

FTPL policy achieves best-of-both-worlds regret in decoupled bandits with reduced computational cost.

Improved DNN calibration without sacrificing accuracy.

We present two instances, L-GAE and L-VGAE, of the variational graph auto-encoding family (VGAE) based on separating feature propagation operations from graph convolution layers typically found in graph learning methods to a single linear matrix computation made prior to input in standard auto-encoder architectures. Th…

The paper proposes methods to extract and analyze individual variable information from complex dependencies.

This paper introduces the factorial marked temporal point process model and presents efficient learning methods. In conventional (multi-dimensional) marked temporal point process models, event is often encoded by a single discrete variable i.e. a marker. In this paper, we describe the factorial marked point processes w…

Neural networks outperform kernels by learning features better.

We investigate finite-time decoupled convergence in nonlinear two-time-scale stochastic approximation.

Regularized training of an autoencoder typically results in hidden unit biases that take on large negative values. We show that negative biases are a natural result of using a hidden layer whose responsibility is to both represent the input data and act as a selection mechanism that ensures sparsity of the representati…

Paper tackles self-supervised learning for non-homophilous graphs.

Study on feature learning dynamics in infinite-depth neural networks, focusing on ResNets.

We investigate probabilistic decoupling of labels supplied for training, from the underlying classes for prediction. Decoupling enables an inference scheme general enough to implement many classification problems, including supervised, semi-supervised, positive-unlabelled, noisy-label and suggests a general solution to…

Two new methods assess feature importance for fairness in machine learning models.

The paper proposes a decoupled approach to efficiently estimate CoVaR, a measure of systemic financial risk.

This paper investigates the effectiveness of decoupled weight decay at the start of training.

Clarifies method of phase synchronization for decoupling linear differential equations.

In this paper, we present a new nonintrusive reduced basis method when a cheap low-fidelity model and expensive high-fidelity model are available. The method relies on proper orthogonal decomposition (POD) to generate the high-fidelity reduced basis and a shallow multilayer perceptron to learn the high-fidelity reduced…

AdaDEM decouples EM into two parts to improve class overlap and uncertainty.

Recent variants improve knowledge distillation performance.

New algorithms for interpreting complex multivariate functions.

Gradient Starvation causes neural networks to focus on a subset of features, leading to imbalance.

Enhances Gaussian processes with spherical features for better scalability and flexibility.

We establish decoupled functional CLTs for two-time-scale stochastic approximation.

Current reinforcement learning (RL) methods can successfully learn single tasks but often generalize poorly to modest perturbations in task domain or training procedure. In this work, we present a decoupled learning strategy for RL that creates a shared representation space where knowledge can be robustly transferred. …

Scaling end-to-end reinforcement learning to control real robots from vision presents a series of challenges, in particular in terms of sample efficiency. Against end-to-end learning, state representation learning can help learn a compact, efficient and relevant representation of states that speeds up policy learning, …

Stylized facts can be regarded as constraints for any modeling attempt of price dynamics on a financial market, in that an empirically reasonable model has to reproduce these stylized facts at least qualitatively. The dynamics of market prices is modeled on a macro-level as the result of the dynamic coupling of two dyn…

Backpropagation algorithm is indispensable for the training of feedforward neural networks. It requires propagating error gradients sequentially from the output layer all the way back to the input layer. The backward locking in backpropagation algorithm constrains us from updating network layers in parallel and fully l…

Deep learning explained through spectral filtering of hierarchical features.

A new model decouples global and local image representations without supervision.

New equations simplify gauge-theoretic Khovanov homology solutions.

Decoupled PFNs improve sequential decision-making by separating epistemic and aleatoric uncertainties.

Unified framework suppresses model bias in semi-supervised learning with decoupled sampling control.

Novel Adam-family method with decoupled weight decay for training neural networks.

DAPS++ improves diffusion-based image restoration by decoupling prior and likelihood.

New method uses diffusion models for Bayesian inverse problems.

We investigate the convergence and stability properties of the decoupled extended Kalman filter learning algorithm (DEKF) within the long-short term memory network (LSTM) based online learning framework. For this purpose, we model DEKF as a perturbed extended Kalman filter and derive sufficient conditions for its stabi…

DAPS++ improves diffusion-based image restoration by decoupling prior and likelihood.

Training a neural network using backpropagation algorithm requires passing error gradients sequentially through the network. The backward locking prevents us from updating network layers in parallel and fully leveraging the computing resources. Recently, there are several works trying to decouple and parallelize the ba…

New method detects heuristics in complex game strategies.