arXiv research

PFDL improves deep learning models' OOD generalization by decorrelating feature embeddings.

This work explores how feature decorrelation improves self-supervised learning.

A new distributed algorithm for fitting sparse additive models with feature division and decorrelation.

Introduces a new feature importance measure using Gram-Schmidt decorrelation.

Paper proposes methods to improve graph domain adaptation by decorrelating node features.

A new algorithm removes unexpected correlations in biased data for better clustering.

Learning an effective representation for high-dimensional data is a challenging problem in reinforcement learning (RL). Deep reinforcement learning (DRL) such as Deep Q networks (DQN) achieves remarkable success in computer games by learning deeply encoded representation from convolution networks. In this paper, we pro…

This paper proposes a decorrelation-based approach to test hypotheses and construct confidence intervals for the low dimensional component of high dimensional proportional hazards models. Motivated by the geometric projection principle, we propose new decorrelated score, Wald and partial likelihood ratio statistics. Wi…

Fitting statistical models is computationally challenging when the sample size or the dimension of the dataset is huge. An attractive approach for down-scaling the problem size is to first partition the dataset into subsets and then fit using distributed algorithms. The dataset can be partitioned either horizontally (i…

Losaw improves FI scores by decorrelating features in ML models.

Learning good representations is a long standing problem in reinforcement learning (RL). One of the conventional ways to achieve this goal in the supervised setting is through regularization of the parameters. Extending some of these ideas to the RL setting has not yielded similar improvements in learning. In this pape…

We introduce the blind subspace deconvolution (BSSD) problem, which is the extension of both the blind source deconvolution (BSD) and the independent subspace analysis (ISA) tasks. We examine the case of the undercomplete BSSD (uBSSD). Applying temporal concatenation we reduce this problem to ISA. The associated `high …

Sparse codes improve optimal control tasks with correlated inputs.

In this paper, we propose novel strategies for neutral vector variable decorrelation. Two fundamental invertible transformations, namely serial nonlinear transformation and parallel nonlinear transformation, are proposed to carry out the decorrelation. For a neutral vector variable, which is not multivariate Gaussian d…

Improves causal graph learning on dependent binary data.

Deep neural networks are widely used in various domains. However, the nature of computations at each layer of the deep networks is far from being well understood. Increasing the interpretability of deep neural networks is thus important. Here, we construct a mean-field framework to understand how compact representation…

Simplified non-contrastive learning avoids representation collapse.

This study shows how social insects and machine learning methods share a common mathematical framework.

Optimizes signal detection in particle physics by decorrelating classifiers.

Proposes a method to improve Lasso stability via variable decorrelation.

New theory shows how multi-head attention reduces variance and decorrelates outputs.

Machine learning models predict EUR/USD currency direction with 58.52% accuracy.

The paper develops methods for high-dimensional inference in Markov random fields.

Improves prediction stability with model misspecification and distribution shift.

Novel method decorrelates batches of triplets for active metric learning.

We describe a strategy for constructing a neural network jet substructure tagger which powerfully discriminates boosted decay signals while remaining largely uncorrelated with the jet mass. This reduces the impact of systematic uncertainties in background modeling while enhancing signal purity, resulting in improved di…

SGD reduces test error by decorrelating updates.

Unified framework for information-theoretic bounds on learning algorithms.

VAE improves anomaly detection for jet tagging at the LHC.

New framework improves multivariate time series forecasting by minimizing redundant information.

A new algorithm reduces data dimensionality and decorrelation in a distributed setting.

FreST Loss decorrelates spatio-temporal dependencies in graph signals.

Novel method decorrelates neurons for better deep learning model generalization.

Two methods use simulation to improve anomaly detection in particle physics.

Proposes SVI for covariate-shift generalization with sparse variable independence.

CW normalizes and decorrelates neural network layers for better concept understanding.

We consider the problem of uncertainty assessment for low dimensional components in high dimensional models. Specifically, we propose a decorrelated score function to handle the impact of high dimensional nuisance parameters. We consider both hypothesis tests and confidence regions for generic penalized M-estimators. U…

Batch Normalization (BN) is capable of accelerating the training of deep models by centering and scaling activations within mini-batches. In this work, we propose Decorrelated Batch Normalization (DBN), which not just centers and scales activations but whitens them. We explore multiple whitening techniques, and find th…

Adam is shown not being able to converge to the optimal solution in certain cases. Researchers recently propose several algorithms to avoid the issue of non-convergence of Adam, but their efficiency turns out to be unsatisfactory in practice. In this paper, we provide new insight into the non-convergence issue of Adam …

Predicting the patient's clinical outcome from the historical electronic medical records (EMR) is a fundamental research problem in medical informatics. Most deep learning-based solutions for EMR analysis concentrate on learning the clinical visit embedding and exploring the relations between visits. Although those wor…

A crucial problem in learning disentangled image representations is controlling the degree of disentanglement during image editing, while preserving the identity of objects. In this work, we propose a simple yet effective model with the encoder-decoder architecture to address this challenge. To encourage disentanglemen…

Estimators computed from adaptively collected data do not behave like their non-adaptive brethren. Rather, the sequential dependence of the collection policy can lead to severe distributional biases that persist even in the infinite data limit. We develop a general method -- $\mathbf{W}$-decorrelation -- for transformi…

Generalizes PCA and ICA for continuous-time signals using neural networks.

This work includes all the technical details of the Sequential Principal Curves Analysis (SPCA) in a single document. SPCA is an unsupervised nonlinear and invertible feature extraction technique. The identified curvilinear features can be interpreted as a set of nonlinear sensors: the response of each sensor is the pr…

Gradient descent on LSE objectives implicitly performs EM, leading to collapse without volume control.

We tackle linear bandits with partially observable features, achieving sublinear regret.

Based on a recent theorem due to the authors, it is shown how the extreme tail dependence between an asset and a factor or index or between two assets can be easily calibrated. Portfolios constructed with stocks with minimal tail dependence with the market exhibit a remarkable degree of decorrelation with the market at…

Paper proposes energy objective for training normalizing flows without determinants.