The paper analyzes how re-weighting helps in reducing variance in high-dimensional kernel methods under covariate shifts.
problem The challenge of high-dimensional kernel methods under covariate shifts and the role of re-weighting.
method Derives asymptotic expansion of high-dimensional kernels under covariate shifts, analyzes bias-variance decomposition, and characterizes the regularized kernel.
result Re-weighting helps in decreasing variance and can be seen as a data-dependent regularization.
Deep learning algorithms can fare poorly when the training dataset suffers from heavy class-imbalance but the testing criterion requires good generalization on less frequent classes. We design two novel methods to improve performance in such scenarios. First, we propose a theoretically-principled label-distribution-awa…
This work aims at solving the problems with intractable sparsity-inducing norms that are often encountered in various machine learning tasks, such as multi-task learning, subspace clustering, feature selection, robust principal component analysis, and so on. Specifically, an Iteratively Re-Weighted method (IRW) with so…
Generative Adversarial Network model for class-imbalanced tabular data.
problem Class imbalance in binary classification problems.
method Generative Adversarial Network (GAN) with synthetic minority class samples.
result Improves average precision compared to re-weighting and oversampling techniques.
This work proposes a new algorithm for training a re-weighted L2 Support Vector Machine (SVM), inspired on the re-weighted Lasso algorithm of Candès et al. and on the equivalence between Lasso and SVM shown recently by Jaggi. In particular, the margin required for each training vector is set independently, defining a n…
M2m method improves deep learning performance on class-imbalanced datasets.
problem Class imbalance in labeled training datasets causes deep neural networks to generalize poorly to minority classes.
method Augment less-frequent classes by translating samples from more-frequent classes.
result Significantly improves generalization on minority classes compared to existing methods.
New method makes CP intervals locally adaptive using trainable transformations.
problem Making Conformal Prediction intervals locally adaptive.
method Defining a trainable change of variables φX(A) that depends on object attributes X. result Locally adaptive prediction intervals with guaranteed marginal validity and variable sizes.
DisCor corrects reinforcement learning issues by re-weighting collected data.
problem Reinforcement learning algorithms struggle with instability and sensitivity to hyperparameters.
method DisCor reweights collected data to mitigate issues caused by the distribution of experience.
result DisCor improves reinforcement learning in challenging settings like multi-task learning and noisy reward signals.
Estimates causal contributions of multiple causes on outcome changes.
problem Quantifying the effect of multiple causes on an outcome change.
method Develops a multiply robust estimation strategy combining regression and re-weighting methods.
result The method recovers the target parameter under partial misspecification and is consistent and asymptotically normal.
Datasets often contain biases which unfairly disadvantage certain groups, and classifiers trained on such datasets can inherit these biases. In this paper, we provide a mathematical formulation of how this bias can arise. We do so by assuming the existence of underlying, unknown, and unbiased labels which are overwritt…
A new method aligns source and target distributions by tuning their weights.
problem Domain adaptation on unlabeled target datasets using labeled source datasets.
method Weighted Joint Distribution Optimal Transport (WJDOT) method that finds alignment between source and target distributions and re-weighting of source distributions.
result Achieves state-of-the-art performance on simulated and real-life datasets.
Estimates calibration error under label shift without labels.
problem Ensuring model reliability in the face of dataset shift without access to labels.
method Importance re-weighting of the labeled source distribution to estimate calibration error under label shift.
result Effective and reliable CE estimation with respect to the shifted target distribution.
Optimizes CNNs by directing gradients along output channels.
problem Improving generalization error in CNNs.
method Output-channel directed re-weighted L2 or Sobolev metrics.
result Improves generalization error by optimizing gradients.
Unsupervised Domain Adaptation aims to learn a model on a source domain with labeled data in order to perform well on unlabeled data of a target domain. Current approaches focus on learning \textit{Domain Invariant Representations}. It relies on the assumption that such representations are well-suited for learning the …
A new method generates counterfactual treatment outcomes for time-varying treatments.
problem Estimating counterfactual outcomes for time-varying treatments with high-dimensional outcomes.
method Conditional generative framework with inverse probability re-weighting.
result Our method outperforms state-of-the-art baselines in generating high-quality counterfactual samples.
OPERA blends multiple OPE estimators to evaluate new policies offline.
problem Lack of reliable offline policy evaluation methods for new policies.
method Adaptive blending of multiple OPE estimators without explicit selection.
result Consistent and reliable policy evaluation framework for offline RL.
URT layer improves few-shot image classification across diverse domains.
problem Few-shot image classification in multi-domain settings.
method Meta-learns to dynamically re-weight and compose domain-specific representations.
result Sets new state-of-the-art on Meta-Dataset.
Investigates upsampling vs. upweighting for balanced training on skewed datasets.
problem Balancing training on heavily imbalanced datasets with scarce data.
method Theoretical and empirical analysis of upsampling and upweighting strategies.
result Upsampling and upweighting diverge under stochastic gradient descent, with upsampling leading to faster convergence but higher overfitting risk.
PARIS reduces imbalanced regression datasets by pruning uninformative samples.
problem Imbalanced regression where models focus on high-frequency regions, ignoring rare but impactful events.
method PARIS uses the representer theorem to compute a closed-form representer deletion residual for iterative pruning of the training set.
result PARIS reduces training set by up to 75% while preserving or improving overall performance, outperforming other methods.
We study the use of knowledge distillation to compress the U-net architecture. We show that, while standard distillation is not sufficient to reliably train a compressed U-net, introducing other regularization methods, such as batch normalization and class re-weighting, in knowledge distillation significantly improves …
Improved DeepONets for PDE solution operators with adaptive re-weighting and new architecture.
problem Training DeepONets for PDE solution operators without paired data.
method Adaptive re-weighting of training examples and novel network architecture.
result Consistently improved predictive accuracy by a factor of 10-50x.
A new kernel improves tensor classification accuracy and reduces computation time.
problem Challenges in classifying high-dimensional tensor data.
method Proposes a weighted subspace exponential kernel based on Tucker decomposition.
result The new kernel outperforms existing methods in accuracy and computational efficiency.
Improves neural network performance by dynamically adjusting model weights based on source reliability.
problem Training neural networks on data from unreliable sources leads to poor performance.
method Dynamic re-weighting strategy using likelihood tempering to adjust model weights based on estimated source reliability.
result Significant improvement in model performance when trained on mixtures of reliable and unreliable data sources.
The paper discusses selecting predictive models for causal inference, highlighting the challenges and proposing a solution.
problem Selecting the best predictive models for causal inference from a variety of machine learning models.
method The paper proposes using Rext−risk, flexible estimators, and splitting data to compute risks for model selection. result The proposed method controls both outcome errors for treated and non-treated individuals, addressing the issue of model selection for causal inference.
A new framework SIMBA improves graph classification performance on size-imbalanced datasets.
problem Size imbalance in graph classification leads to poor model performance.
method Energy-guided structural smoothing between head and tail graphs, re-weighting based on energy propagation.
result SIMBA outperforms existing methods in size-imbalanced graph classification tasks.
Self-supervised learning performs better than supervised learning on imbalanced datasets.
problem The performance gap between balanced and imbalanced pre-training with self-supervised learning is smaller than with supervised learning.
method Systematic investigation of self-supervised learning under dataset imbalance, including experiments and theoretical analyses.
result Self-supervised representations are more robust to class imbalance than supervised representations.
One-step Bellman alignment improves online RL by reducing task mismatch.
problem Online RL struggles with task similarity defined by rewards or transitions.
method One-step Bellman alignment and re-weighted targeting (RWT) to correct task mismatch.
result Regret bounds show task shift complexity, not target MDP, affects performance.
New method enhances adversarial robustness of deep learning models.
problem Improving the robustness of deep learning models against adversarial attacks.
method Optimal transport regularized divergences applied to distributionally robust optimization.
result Improved adversarial robustness on CIFAR-10 and CIFAR-100 datasets.
A method to improve time series forecasting by dynamically adjusting weights of forecasters.
problem Challenges in time series forecasting due to evolving data distributions.
method Dynamic re-weighting of forecasters based on evolving data distributions.
result Competitive performance compared to state-of-the-art methods for combining forecasters.
FAIR method uses adversarial training to learn fair instance weights.
problem Reduces bias in machine learning predictions through fair instance weighting.
method Adversarial training to learn instance weighting function ensuring fair predictions.
result Demonstrates better trade-off between accuracy and fairness compared to other models.
This paper improves image retrieval accuracy through novel relevance feedback methods.
problem Improving image retrieval accuracy in Content-Based Image Retrieval (CBIR).
method Novel addition to feature re-weighting and classification techniques, focusing on 0-th iteration improvement.
result Significantly improved retrieval accuracy from relevance feedback.
Self-paced learning and hard example mining re-weight training instances to improve learning accuracy. This paper presents two improved alternatives based on lightweight estimates of sample uncertainty in stochastic gradient descent (SGD): the variance in predicted probability of the correct class across iterations of …
In real-world classification problems, the class balance in the training dataset does not necessarily reflect that of the test dataset, which can cause significant estimation bias. If the class ratio of the test dataset is known, instance re-weighting or resampling allows systematical bias correction. However, learning…
HappyMap improves fairness and learning across domains by generalizing multi-calibration.
problem Improving fairness and learning across different domains for predictions.
method Proposes HappyMap, a generalized approach to multi-calibration.
result Unified understanding of fairness and learning across domains.
We present our solution to the job recommendation task for RecSys Challenge 2016. The main contribution of our work is to combine temporal learning with sequence modeling to capture complex user-item activity patterns to improve job recommendations. First, we propose a time-based ranking model applied to historical obs…
Conditional generative adversarial networks (cGANs) have gained a considerable attention in recent years due to its class-wise controllability and superior quality for complex generation tasks. We introduce a simple yet effective approach to improving cGANs by measuring the discrepancy between the data distribution and…
New methods reduce bias in synthetic data for machine learning.
problem Statistical bias in synthetic data generated for privacy.
method Re-weighting strategies using privatised likelihood ratios.
result Private importance weighting enhances synthetic data utility.
The paper presents a novel approach to direct covariance function learning for Bayesian optimisation, with particular emphasis on experimental design problems where an existing corpus of condensed knowledge is present. The method presented borrows techniques from reproducing kernel Banach space theory (specifically m-k…
Maximum Likelihood Estimation (MLE) is the bread and butter of system inference for stochastic systems. In some generality, MLE will converge to the correct model in the infinite data limit. In the context of physical approaches to system inference, such as Boltzmann machines, MLE requires the arduous computation of pa…
We consider a decomposition method for compressive streaming data in the context of online compressive Robust Principle Component Analysis (RPCA). The proposed decomposition solves an n-ℓ1 cluster-weighted minimization to decompose a sequence of frames (or vectors), into sparse and low-rank components, from com…
Citizen science projects are successful at gathering rich datasets for various applications. However, the data collected by citizen scientists are often biased --- in particular, aligned more with the citizens' preferences than with scientific objectives. We propose the Shift Compensation Network (SCN), an end-to-end l…
We propose a method for nonparametric density estimation that exhibits robustness to contamination of the training sample. This method achieves robustness by combining a traditional kernel density estimator (KDE) with ideas from classical M-estimation. We interpret the KDE based on a radial, positive semi-definite ke…
In this paper we propose a novel framework for the construction of sparsity-inducing priors. In particular, we define such priors as a mixture of exponential power distributions with a generalized inverse Gaussian density (EP-GIG). EP-GIG is a variant of generalized hyperbolic distributions, and the special cases inclu…
In real world systems, the predictions of deployed Machine Learned models affect the training data available to build subsequent models. This introduces a bias in the training data that needs to be addressed. Existing solutions to this problem attempt to resolve the problem by either casting this in the reinforcement l…
We introduce a novel loss max-pooling concept for handling imbalanced training data distributions, applicable as alternative loss layer in the context of deep neural networks for semantic image segmentation. Most real-world semantic segmentation datasets exhibit long tail distributions with few object categories compri…
Stein variational gradient decent (SVGD) has been shown to be a powerful approximate inference algorithm for complex distributions. However, the standard SVGD requires calculating the gradient of the target density and cannot be applied when the gradient is unavailable. In this work, we develop a gradient-free variant …
Sparse reconstruction approaches using the re-weighted l1-penalty have been shown, both empirically and theoretically, to provide a significant improvement in recovering sparse signals in comparison to the l1-relaxation. However, numerical optimization of such penalties involves solving problems with l1-norms in the ob…
The ERI is a new index for measuring exam readiness.
problem Measuring exam readiness in a clear and actionable way.
method The ERI combines six signals derived from practice and mock tests, formalizing axioms for component maps and the composite.
result The ERI is a composite score interpretable and actionable for exam readiness.