PFDL improves deep learning models' OOD generalization by decorrelating feature embeddings.
problem Out-of-distribution generalization in deep learning models.
method PFDL algorithm that optimizes feature decomposition network and image classification model.
result PFDL improves the accuracy of image classification models on OOD datasets.
This work explores how feature decorrelation improves self-supervised learning.
problem Complete and dimensional collapse issues in self-supervised learning.
method Study of a concise framework and connection to feature decorrelation.
result Feature decorrelation improves self-supervised learning.
A new distributed algorithm for fitting sparse additive models with feature division and decorrelation.
problem Fitting high-dimensional sparse additive models efficiently and accurately.
method Divide, decorrelate, and conquer approach.
result Effective and efficient recovery of sparsity patterns and statistical inference for each component.
Introduces a new feature importance measure using Gram-Schmidt decorrelation.
problem Determining feature influence strength and nature in datasets.
method Gram-Schmidt decorrelation and random forest regression.
result Empirical comparison of new estimators with established methods.
Paper proposes methods to improve graph domain adaptation by decorrelating node features.
problem Challenges in transferring knowledge from one graph to another.
method Proposes decorrelating node features using GCN and graph transformer layers.
result Significant performance enhancements and clear visualizations of learned representations.
A new algorithm removes unexpected correlations in biased data for better clustering.
problem Clustering with selection bias in data.
method Decorrelation regularized K-Means (DCKM) algorithm.
result DCKM achieves significant performance gains on real-world datasets.
Learning an effective representation for high-dimensional data is a challenging problem in reinforcement learning (RL). Deep reinforcement learning (DRL) such as Deep Q networks (DQN) achieves remarkable success in computer games by learning deeply encoded representation from convolution networks. In this paper, we pro…
Fitting statistical models is computationally challenging when the sample size or the dimension of the dataset is huge. An attractive approach for down-scaling the problem size is to first partition the dataset into subsets and then fit using distributed algorithms. The dataset can be partitioned either horizontally (i…
Losaw improves FI scores by decorrelating features in ML models.
problem Feature correlation distorts feature importance scores in ML models.
method Losaw uses local sample weighting to decorrelate features.
result Losaw consistently improves feature importance scores and prediction accuracy.
Learning good representations is a long standing problem in reinforcement learning (RL). One of the conventional ways to achieve this goal in the supervised setting is through regularization of the parameters. Extending some of these ideas to the RL setting has not yielded similar improvements in learning. In this pape…
We introduce the blind subspace deconvolution (BSSD) problem, which is the extension of both the blind source deconvolution (BSD) and the independent subspace analysis (ISA) tasks. We examine the case of the undercomplete BSSD (uBSSD). Applying temporal concatenation we reduce this problem to ISA. The associated `high …
Sparse codes improve optimal control tasks with correlated inputs.
problem Optimal control tasks with correlated feature inputs.
method Used a sparse code to represent natural images in an optimal control task solved with neuro-dynamic programming.
result An over-complete sparse code increases memory capacity and learning speed beyond a complete code.
In this paper, we propose novel strategies for neutral vector variable decorrelation. Two fundamental invertible transformations, namely serial nonlinear transformation and parallel nonlinear transformation, are proposed to carry out the decorrelation. For a neutral vector variable, which is not multivariate Gaussian d…
Improves causal graph learning on dependent binary data.
problem Challenges in learning causal graphical models from dependent binary data.
method Decorrelation-based approach using latent utility model and EM-like algorithm.
result Significant improvement in accuracy of causal graph learning.
Deep neural networks are widely used in various domains. However, the nature of computations at each layer of the deep networks is far from being well understood. Increasing the interpretability of deep neural networks is thus important. Here, we construct a mean-field framework to understand how compact representation…
Simplified non-contrastive learning avoids representation collapse.
problem Training failure modes in self-supervised learning.
method Hyperdimensional computing and inductive bias.
result The approach avoids representation collapses.
This study shows how social insects and machine learning methods share a common mathematical framework.
problem Understanding how decentralized systems achieve optimal decision-making.
method Developed a rigorous mathematical framework to show isomorphism between ant colonies and ensemble machine learning.
result Demonstrated that ant colony decision-making and random forest learning implement identical variance reduction strategies through decorrelation of identical units.
Optimizes signal detection in particle physics by decorrelating classifiers.
problem Systematic errors in background models can mislead signal detection.
method Use optimal transport to decorrelate classifiers from protected variables, then apply semiparametric mixture model.
result Decorrelation and signal enrichment improve the stability, robustness, and power of signal detection tests.
Proposes a method to improve Lasso stability via variable decorrelation.
problem Stability of Lasso in high-dimensional settings with correlated variables.
method Variable decorrelation before applying the Lasso.
result Improves stability of variable selection regardless of predictor correlation.
New theory shows how multi-head attention reduces variance and decorrelates outputs.
problem Understanding and optimizing multi-head attention in neural networks.
method Developed a statistical theory linking multi-head attention to ensemble Nadaraya-Watson estimators.
result MHA variance reduction depends on head decorrelation, not just head count.
Machine learning models predict EUR/USD currency direction with 58.52% accuracy.
problem Predicting the directional movement of EUR/USD in the Foreign Exchange market.
method Comparative analysis of machine learning models, including decorrelated and non-decorrelated feature sets, and meta-estimators.
result 58.52% accuracy for one-day ahead forecasts.
The paper develops methods for high-dimensional inference in Markov random fields.
problem Statistical inference for high-dimensional Markov random fields.
method Markov Chain Monte Carlo Maximum Likelihood Estimation (MCMC-MLE) with Elastic-net regularization.
result The proposed methods achieve ℓ1-consistency and false discovery rate control. Improves prediction stability with model misspecification and distribution shift.
problem Inaccurate parameter estimation and instability of prediction in real-world applications.
method Proposes Decorrelated Weighting Regression (DWR) algorithm to optimize weights for samples and variables.
result Significantly improves accuracy of parameter estimation and prediction stability.
Novel method decorrelates batches of triplets for active metric learning.
problem Correlation among triplets degrades active learning performance.
method Proposes a novel method to decorrelate batches of triplets, balancing informativeness and diversity.
result Method outperforms state-of-the-art in active metric learning.
We describe a strategy for constructing a neural network jet substructure tagger which powerfully discriminates boosted decay signals while remaining largely uncorrelated with the jet mass. This reduces the impact of systematic uncertainties in background modeling while enhancing signal purity, resulting in improved di…
SGD reduces test error by decorrelating updates.
problem Improving generalization error in machine learning models.
method Derive a formula for generalization gap change due to SGD updates, compare to GD, and show decorrelation effect.
result SGD implicitly regularizes generalization error by decorrelating updates.
Unified framework for information-theoretic bounds on learning algorithms.
problem Deriving generalization bounds for learning algorithms.
method Probabilistic decorrelation lemma, symmetrization, couplings, chaining, Young's inequality.
result New upper bounds on generalization error in expectation and high probability.
VAE improves anomaly detection for jet tagging at the LHC.
problem Anomaly detection in jet tagging at the LHC.
method Variational Autoencoder (VAE) trained on background QCD jets, with latent space learning for anomaly detection.
result Outlier Exposed VAE (OE-VAE) achieves excellent results in both sensitivity and decorrelation of jet mass.
New framework improves multivariate time series forecasting by minimizing redundant information.
problem Improving multivariate time series forecasting with deep learning techniques.
method Cross-variable Decorrelation Aware feature Modeling (CDAM) and Temporal correlation Aware Modeling (TAM) to refine Channel-mixing and exploit temporal correlations.
result Significantly surpasses existing models in comprehensive tests.
A new algorithm reduces data dimensionality and decorrelation in a distributed setting.
problem Distributed PCA for decorrelated features in big data.
method Feedforward neural network-based one time-scale algorithm for estimating eigenvectors of distributed data covariance matrix.
result DSA converges linearly to the true solution.
This paper proposes a decorrelation-based approach to test hypotheses and construct confidence intervals for the low dimensional component of high dimensional proportional hazards models. Motivated by the geometric projection principle, we propose new decorrelated score, Wald and partial likelihood ratio statistics. Wi…
FreST Loss decorrelates spatio-temporal dependencies in graph signals.
problem Complex spatio-temporal dependencies in graph-structured signals are not well captured by standard forecasting models.
method FreST Loss extends supervision to the joint spatio-temporal spectrum using Joint Fourier Transform (JFT).
result FreST Loss reduces estimation bias and improves forecasting accuracy on real-world datasets.
Novel method decorrelates neurons for better deep learning model generalization.
problem High correlations between neurons limit deep learning model generalization.
method Regularization terms from minimum spanning tree of neuron cliques, using correlation dissimilarities.
result Our regularizers outperform existing methods and minimize neuron redundancies.
Two methods use simulation to improve anomaly detection in particle physics.
problem Artificial bumps in invariant mass spectra from machine learning classifiers.
method Simulation-assisted decorrelation techniques.
result Both methods are robust to correlations in the data and improve anomaly detection.
Proposes SVI for covariate-shift generalization with sparse variable independence.
problem Covariate-shift generalization with limited data and unstable variables.
method Introduces sparsity constraint and combines reweighting and selection in an iterative way.
result Improves covariate-shift generalization performance on synthetic and real-world datasets.
CW normalizes and decorrelates neural network layers for better concept understanding.
problem Challenges in understanding neural network hidden layers.
method Introducing concept whitening (CW) to alter neural network layers.
result CW provides clearer understanding of how networks learn concepts over layers.
We consider the problem of uncertainty assessment for low dimensional components in high dimensional models. Specifically, we propose a decorrelated score function to handle the impact of high dimensional nuisance parameters. We consider both hypothesis tests and confidence regions for generic penalized M-estimators. U…
Batch Normalization (BN) is capable of accelerating the training of deep models by centering and scaling activations within mini-batches. In this work, we propose Decorrelated Batch Normalization (DBN), which not just centers and scales activations but whitens them. We explore multiple whitening techniques, and find th…
Adam is shown not being able to converge to the optimal solution in certain cases. Researchers recently propose several algorithms to avoid the issue of non-convergence of Adam, but their efficiency turns out to be unsatisfactory in practice. In this paper, we provide new insight into the non-convergence issue of Adam …
Predicting the patient's clinical outcome from the historical electronic medical records (EMR) is a fundamental research problem in medical informatics. Most deep learning-based solutions for EMR analysis concentrate on learning the clinical visit embedding and exploring the relations between visits. Although those wor…
A crucial problem in learning disentangled image representations is controlling the degree of disentanglement during image editing, while preserving the identity of objects. In this work, we propose a simple yet effective model with the encoder-decoder architecture to address this challenge. To encourage disentanglemen…
Estimators computed from adaptively collected data do not behave like their non-adaptive brethren. Rather, the sequential dependence of the collection policy can lead to severe distributional biases that persist even in the infinite data limit. We develop a general method -- W-decorrelation -- for transformi…
Generalizes PCA and ICA for continuous-time signals using neural networks.
problem Low-rank decomposition of continuous-time vector-valued signals.
method Implicit neural network framework to learn numerical approximations of PCA and ICA.
result Unified approach to PCA and ICA in continuous domain, enforcing decorrelation and independence.
This work includes all the technical details of the Sequential Principal Curves Analysis (SPCA) in a single document. SPCA is an unsupervised nonlinear and invertible feature extraction technique. The identified curvilinear features can be interpreted as a set of nonlinear sensors: the response of each sensor is the pr…
Gradient descent on LSE objectives implicitly performs EM, leading to collapse without volume control.
problem Gradient collapse in autoencoders without volume control.
method Introduced a single-layer encoder with an LSE objective and InfoMax regularization for volume control.
result Gradient--responsibility identity holds exactly; LSE alone collapses; variance prevents dead components; decorrelation prevents redundancy.
Based on a recent theorem due to the authors, it is shown how the extreme tail dependence between an asset and a factor or index or between two assets can be easily calibrated. Portfolios constructed with stocks with minimal tail dependence with the market exhibit a remarkable degree of decorrelation with the market at…
Paper proposes energy objective for training normalizing flows without determinants.
problem Challenges in training normalizing flows due to Jacobian determinants.
method Introduces energy objective based on proper scoring rules, determinant-free.
result Energy objective supports novel model families and competitive performance.
New Convolutional Unit improves Batch Whitening performance.
problem Improving the efficiency and effectiveness of Batch Whitening.
method Proposes a new Convolutional Unit that aligns with Batch Whitening theory and empirically analyzes the original Convolutional Unit.
result Significantly improved performance on multiple image classification datasets.