Proposes a method to create fair, robust predictors that remain consistent across different scenarios.
problem Creating fair and robust machine learning models that behave consistently across different scenarios.
method Graphical criteria and a model-agnostic framework called CIP based on HSCIC.
result Demonstrates the effectiveness of CIP in enforcing counterfactual invariance across various datasets.
New theory validates the use of invariant predictors for OOD generalization.
problem Ensuring predictors generalize well across unseen environments.
method Developed new theoretical conditions and derived an Inter Gradient Alignment algorithm.
result Validated the necessity of invariant predictors for OOD optimality.
New method finds invariant predictors to improve model robustness.
problem Machine learning models are brittle in new environments with spurious correlations.
method Formulates invariant risk minimization as a Nash equilibrium game.
result Algorithm yields similar or better accuracy with lower variance.
Paper introduces a method to predict molecule properties from diverse data sources.
problem Limited ability to accommodate scarce or fragmented training data.
method Adaptive Invariance using invariant risk minimization to generalize beyond heterogeneous data.
result Predictor outperforms state-of-the-art transfer learning methods by significant margin.
A new IL framework estimates invariant predictors with single domain data.
problem Deep networks inherit spurious correlations and fail on unseen domains.
method Assumes multiple labeled domains for higher-level tasks, uses single domain for target task, employs cross-validation for hyperparameter selection.
result Empirically demonstrates effectiveness and correctness of hyperparameter selection.
Empirical study of IRMv1, an invariant risk minimization framework.
problem Learning predictors invariant to spurious correlations across different training environments.
method Extending ColoredMNIST experiment to various settings.
result IRMv1 performs better as spurious correlation varies more widely.
The study shows symmetry improves machine learning generalization.
problem Improving machine learning generalization through symmetry.
method Using an averaging operator to prove equivariance reduces test risk.
result Equivariant predictors reduce test risk compared to non-equivariant ones.
Estimates model performance under distribution shift using domain-invariant predictors.
problem Poor performance of models on test distributions different from training distributions.
method Uses domain-invariant predictors as a proxy for unknown target labels.
result Shows that the complexity of latent representations influences target risk.
Proposes RVP to address theoretical concerns of V-REx for OOD generalization.
problem Theoretical concerns about V-REx's motivation and utility.
method Risk Variance Penalization (RVP) modifies V-REx's regularization.
result RVP discovers a robust predictor and finds invariant predictors under certain conditions.
Causal invariance can improve finite-sample domain adaptation, but only when the target risk margins are large.
problem Finite-sample domain adaptation
method Linear regression with causal knowledge
result Adaptive aggregation can match best candidate predictor while avoiding negative transfer
New method MRI improves machine learning models' ability to generalize to unseen data.
problem Machine learning models often fail to generalize well to out-of-distribution data.
method Introduces a new notion of invariance (MRI) and a practical version (MRI-v1) to improve model generalization.
result MRI-v1 guarantees invariant predictors and outperforms IRM-v1 in various settings.
The paper tackles spurious correlations in machine learning models and introduces counterfactual invariance.
problem Spurious correlations in machine learning models that depend on irrelevant parts of input data.
method The paper uses causal inference to stress test models and introduces counterfactual invariance as a formal requirement.
result Counterfactual invariance is a requirement for models to be robust to irrelevant perturbations in input data.
A new approach to group fairness treats it as a bargaining problem.
problem Fairness in deploying predictors across subpopulations.
method Interpreting fairness as a bargaining problem and proposing relative improvement.
result Relative improvement provides axiomatic justification and finite-sample convergence guarantees.
New bounds prevent degradation in high-dimensional signal estimation.
problem Statistical learning bounds degradation with increasing dimensionality.
method Investigates linear prediction rules under structural assumptions.
result Derives upper and lower bounds on generalization error.
Bayesian Hierarchical Invariant Prediction refines ICP for better scalability and prior integration.
problem Improving computational scalability and invariance testing for causal inference.
method Bayesian Hierarchical structure to test invariance under heterogeneous data.
result Demonstrated improved scalability and potential as an alternative to ICP.
Many learning algorithms have invariances: when their training data is transformed in certain ways, the function they learn transforms in a predictable manner. Here we formalize this notion using concepts from the mathematical field of category theory. The invariances that a supervised learning algorithm possesses are …
New method learns optimal prediction strategies in adversarial games.
problem Learning optimal prediction procedures in uncertain data environments.
method Adversarial Monte Carlo approach with neural network architecture.
result Optimal strategy is equivariant and invariant to various transformations.
A method for fair representation learning through bi-level optimization and implicit differentiation.
problem Ensuring fair predictors invariant across sub-groups.
method Bi-level optimization with inner-loop for invariant predictors, implicit path alignment for efficiency.
result Consistently better trade-off in prediction performance and fairness measurement.
SFB uses stable features to adapt unstable ones for better performance.
problem Improving classifier performance on out-of-distribution data by leveraging stable features.
method SFB learns a predictor that separates stable and unstable features, then adapts unstable predictions using stable predictions.
result SFB can learn an asymptotically-optimal predictor without test-domain labels.
Harmonization schemes limit accuracy due to domain information.
problem Harmonization schemes lead to inaccurate predictions due to domain information.
method Analysis of mutual information and real label value informativeness.
result Accuracy is limited by the domain with least information.
Achieving superhuman playing level by AlphaGo corroborated the capabilities of convolutional neural architectures (CNNs) for capturing complex spatial patterns. This result was to a great extent due to several analogies between Go board states and 2D images CNNs have been designed for, in particular translational invar…
IRM fails to capture natural invariances on simple problems.
problem IRM fails to capture natural invariances on simple problems.
method IRM formulation and practical linear variant.
result IRM can lead to worse generalization than unconstrained ERM.
This paper extends stable blanket theory to models with hidden variables and causal cycles.
problem Identifying stable predictors in models with hidden variables and causal cycles.
method Use acyclic directed mixed graphs (ADMGs) and directed graphs (DGs) with m-separation and σ-separation to characterize and construct intervention-stable predictor sets. result Graphical characterizations of Markov blankets, stable frontiers, and stable blankets in models with hidden variables and cycles.
Game theory approach to predicting and responding to interventions based on causal relationships.
problem Optimizing predictions and interventions in response to observational data.
method Prediction-intervention game framework, focusing on invariant subsets of covariates.
result Stable-blanket predictors are optimal for certain follower objectives and under specific conditions.
In this paper, we consider a generalized multivariate regression problem where the responses are monotonic functions of linear transformations of predictors. We propose a semi-parametric algorithm based on the ordering of the responses which is invariant to the functional form of the transformation function. We prove t…
Paper tackles reinforcement learning generalization through invariant policy optimization.
problem Learning policies that generalize beyond training domains.
method Invariant policy optimization principle and novel learning algorithm IPO.
result Significant improvements in generalization performance on unseen domains.
The problem of forecasting conditional probabilities of the next event given the past is considered in a general probabilistic setting. Given an arbitrary (large, uncountable) set C of predictors, we would like to construct a single predictor that performs asymptotically as well as the best predictor in C, on any data.…
We consider forecasting a single time series using a large number of predictors in the presence of a possible nonlinear forecast function. Assuming that the predictors affect the response through the latent factors, we propose to first conduct factor analysis and then apply sufficient dimension reduction on the estimat…
A new method evaluates invariant performance of IRM-based representations.
problem Impact of data changes on machine learning model performance.
method Proposes a novel method to evaluate invariant performance of IRM-based representations.
result Establishes a robust criterion to assess invariant performance of various representation techniques.
Methods of transfer learning try to combine knowledge from several related tasks (or domains) to improve performance on a test task. Inspired by causal methodology, we relax the usual covariate shift assumption and assume that it holds true for a subset of predictor variables: the conditional distribution of the target…
New bounds explain deterministic non-smooth deep nets without large Lipschitz constants.
problem Challenges in explaining generalization of deterministic non-smooth deep nets.
method De-randomized PAC-Bayes margin bounds for deterministic non-convex and non-smooth predictors.
result New bounds avoid large Lipschitz constants, providing generalization guarantees.
This paper proposes a method to reduce complexity in GLMs with categorical predictors.
problem Wasteful, hard-to-interpret, and prone to overfitting of traditional one-hot encoding for high-cardinality categorical predictors.
method Clustering categories of categorical predictors through a numerical method that preserves or improves accuracy while reducing the number of coefficients.
result Clustering categories of categorical predictors reduces complexity substantially without harming accuracy.
It is widely believed that the prediction accuracy of decision tree models is invariant under any strictly monotone transformation of the individual predictor variables. However, this statement may be false when predicting new observations with values that were not seen in the training-set and are close to the location…
Prediction of dynamical time series with additive noise using support vector machines or kernel based regression has been proved to be consistent for certain classes of discrete dynamical systems. Consistency implies that these methods are effective at computing the expected value of a point at a future time given the …
New analysis identifies key factors in wildfire-generated thunderstorms.
problem Understanding the causes of pyrocumulonimbus (pyroCb) storms.
method Invariant Causal Prediction, conditional independence test, greedy-ICP search algorithm.
result Identified seven causal predictors for pyroCb formation.
The article compares predictor importance in classification problems with categorical outcomes.
problem Comparing predictor importance in classification problems with categorical response variables.
method The approach is based on the categorical Gini correlation (CGC) and tests differences in CGCs across predictor groups.
result The proposed methodology accommodates predictors of arbitrary and unequal dimensions and allows for dependence between predictor groups.
Paper proposes a sparse synthetic control method to select important predictors.
problem Choosing and weighting predictors affects synthetic control estimator performance.
method Sparse synthetic control procedure that penalizes predictors, derived in a linear factor model.
result Sparse synthetic control achieves lower bias and better post-treatment performance.
This paper presents Sparse Partitioning, a Bayesian method for identifying predictors that either individually or in combination with others affect a response variable. The method is designed for regression problems involving binary or tertiary predictors and allows the number of predictors to exceed the size of the sa…
WeakNAS uses a set of weaker predictors to find top architectures with fewer samples.
problem Finding the best neural architecture with heavy computation costs.
method Proposes a paradigm shift from fitting the whole architecture space to progressively fitting a search path through a set of weaker predictors.
result WeakNAS produces coarse-to-fine iteration to gradually refine the ranking of sampling space, requiring fewer samples to find top-performance architectures.
Regularising for invariance to data augmentation improves machine learning models.
problem Improving generalization in machine learning models through data augmentation.
method Explicit regularisation to encourage invariance at the level of individual model predictions.
result Explicit regularisation improves generalization and equalizes performance differences between objectives.
This paper continues study, both theoretical and empirical, of the method of Venn prediction, concentrating on binary prediction problems. Venn predictors produce probability-type predictions for the labels of test objects which are guaranteed to be well calibrated under the standard assumption that the observations ar…
Derives bounds for deterministic predictors using smooth loss functions.
problem Generalizing probabilistic predictors to deterministic ones.
method Exploits smoothness properties of loss and predictor classes, controlling the Jensen gap class through Rademacher complexity.
result Derives bounds for deterministic predictors involving flatness quantities from Jacobians and Hessians.
Study shows competition feedback can make ML predictors biased towards specific user groups.
problem How competition affects machine learning predictors and user prediction quality.
method Flexible model of competing ML predictors, empirical and mathematical analysis.
result Competition causes predictors to specialize for specific sub-populations at the cost of general performance.
Study on merging predictors in causal and anticausal directions using CMAXENT.
problem Comparing merging predictors in causal and anticausal directions.
method Using CMAXENT as inductive bias, study differences in merging predictors.
result CMAXENT solution reduces to logistic regression in causal direction and LDA in anticausal direction.
Paper introduces SUEL model for integrating predictors without labeled data.
problem Combining predictors with unknown accuracy and high correlation.
method Structured unsupervised ensemble learning (SUEL) with correlation-based decomposition algorithms.
result Efficient integration of dependent predictors without labeled data.
The paper develops predictors for functional data on manifolds.
problem Functional data prediction on time-varying manifolds.
method Least-squares local linear Fréchet curve predictor and weighted Fréchet mean approach.
result Asymptotical optimality of the proposed predictors.
Paper develops an efficient method for conformal prediction in sparse linear models.
problem Computing conformal prediction sets for sparse linear models is computationally infeasible.
method Numerical continuation techniques to approximate the solution path efficiently.
result The method accurately approximates conformal prediction sets for sparse linear models.
Adaptive kernels from neural networks improve model performance.
problem Improving neural network performance through adaptive kernels.
method Deriving adaptive kernels from infinite-width neural networks using feature learning and gradient flow training.
result Adaptive kernels achieve lower test loss compared to traditional kernels.