Proposes MSS to identify causal structure from heterogeneous environments.
problem Distribution shifts between environments violate i.i.d. data assumption.
method Sparse mechanism shift hypothesis, score-based approach.
result Identifies entire causal structure with high probability.
SAMS-VAE models cellular perturbations using sparse additive mechanisms.
problem Modeling effects of diverse interventions on cells.
method Sparse Additive Mechanism Shift Variational Autoencoder (SAMS-VAE).
result SAMS-VAE identifies disentangled, perturbation-specific latent subspaces.
Domain adaptation framework identifies latent variables for target distribution identifiability.
problem Unsupervised domain adaptation without identifiable joint distribution of features and labels.
method Formulated latent variable model with invariant and changing components, constrained domain shift to influence only changing components.
result Joint distribution of data and labels in target domain is identifiable under mild conditions.
In this paper we formally analyse the use of sparse filtering algorithms to perform covariate shift adaptation. We provide a theoretical analysis of sparse filtering by evaluating the conditions required to perform covariate shift adaptation. We prove that sparse filtering can perform adaptation only if the conditional…
Identifies shifts in causal mechanisms between related datasets using ANMs.
problem Estimating the full causal structure from data is challenging; focus on identifying shifts in causal mechanisms.
method Assumes nonlinear additive noise models, uses Jacobian of score function for mixture distribution to identify shifts.
result Shows applicability of the approach on synthetic and real-world data.
SJS model predicts label shifts in multinomial datasets.
problem Predicting label shifts in multinomial datasets.
method Sparse joint shift model for dataset shift.
result Valid predictions and class prior probabilities estimates.
New insights into SGD and generalization via shift-curvature and bias-curvature mechanisms.
problem Understanding the role of curvature in generalization and how SGD affects it.
method Derivation of new SGD steady-state distribution and analysis of shift-curvature and bias-curvature mechanisms.
result Shift-curvature is a significant factor in test performance, especially for small SGD noise.
Adaptive model learns from time series data with changing distributions.
problem Predicting time series data under distribution shift.
method Formulates distribution shift as weighted empirical risk minimization. Uses a gradient-based learning method for a forgetting mechanism.
result Proposes an efficient method for adaptive time series prediction.
Study investigates OOD generalization methods for mechanics problems.
problem Real-world mechanics problems with unknown test environments and data distribution shifts.
method Investigates OOD generalization methods for regression problems in mechanics.
result OOD generalization methods perform better than traditional ML methods on mechanics-specific regression problems.
We describe a mechanism by which artificial neural networks can learn rapid adaptation - the ability to adapt on the fly, with little data, to new tasks - that we call conditionally shifted neurons. We apply this mechanism in the framework of metalearning, where the aim is to replicate some of the flexibility of human …
Proposes SVI for covariate-shift generalization with sparse variable independence.
problem Covariate-shift generalization with limited data and unstable variables.
method Introduces sparsity constraint and combines reweighting and selection in an iterative way.
result Improves covariate-shift generalization performance on synthetic and real-world datasets.
CtrlNS learns latent factors and distribution shifts from sparse transitions without prior knowledge.
problem Lack of prior knowledge of domain variables limits causal temporal representation learning.
method Sparse transition assumption and identifiability results from theoretical perspective.
result Effective in identifying distribution shifts and latent factors without prior knowledge.
Paper proposes SJS model to estimate model performance under covariate and label shifts.
problem Estimating model performance when both covariates and labels shift.
method Sparse Joint Shift (SJS) model and SEES algorithm.
result SEES achieves significant shift estimation error improvements over existing approaches.
In Hamiltonian mechanics, a (continuous) symmetry leads to conserved quantity, which is a function on (extended) phase space. In Nambu mechanics, a straightforward consequence of symmetry is just a relative integral invariant, a differential form which only upon integration over a cycle provides a conserved real number…
New framework improves model robustness by focusing on stable relations across environments.
problem Standard supervised learning fails under data distribution shift.
method Gradient-based learning framework derived from the principle of independent causal mechanisms (ICM).
result Models generalize well to unseen scenarios, ignoring unstable relations.
Safe-DRFS selects features robust to covariate shifts for reliable performance.
problem Feature selection fails in diverse deployment environments.
method Safe-DRFS extends safe screening to distributionally robust settings under covariate shift.
result Safe-DRFS identifies a feature subset encompassing optimal subsets across distribution shifts.
REx tackles distributional shift by reducing risk differences across domains.
problem Tackling distributional shift when transferring machine learning systems to real-world applications.
method Risk Extrapolation (REx) assumes training domains represent test-time variations and uses extrapolated domains to minimize risk variance.
result REx reduces sensitivity to extreme distributional shifts, including causal and anti-causal inputs.
Proposes real-time risk monitoring for machine learning systems under unknown shifts.
problem Dynamic distribution shifts challenge real-world machine learning systems' risk assurances.
method Sequential hypothesis testing with 'testing by betting' to detect risk violations.
result Effective real-time risk monitoring under various unknown shifts.
Study counterfactuals in cyclic systems with shifts and scales.
problem Counterfactual inference in cyclic systems with shifts and scales.
method Shift-scale interventions in cyclic SCMs.
result Valid inference in cyclic systems with shifts and scales.
New method transfers causal mechanisms for few-shot domain adaptation.
problem Few labeled target domain data for regression problems.
method Mechanism transfer using structural equations in causal modeling.
result Method can adapt from apparently different distributions.
DRSS method identifies unnecessary samples and features in DR covariate shift.
problem Identifying unnecessary samples and features in DR covariate shift.
method Combines DR learning and safe screening techniques.
result DRSS method provides reliable identification of unnecessary samples and features under specified distribution uncertainty.
The paper explores how to extrapolate from limited data points using causal mechanisms.
problem Handling distribution shifts with limited target samples.
method Formulates the extrapolation problem with a latent-variable model embodying the minimal change principle in causal mechanisms, and identifies conditions for identification.
result Theoretical understanding and practical methods for extrapolation without requiring an on-support target distribution.
Sparse coding is an unsupervised learning algorithm that learns a succinct high-level representation of the inputs given only unlabeled data; it represents each input as a sparse linear combination of a set of basis functions. Originally applied to modeling the human visual cortex, sparse coding has also been shown to …
The paper tackles policy learning in dynamic environments using causal methods.
problem Existing reinforcement learning algorithms assume static mechanisms, but real-world systems often have changing mechanisms.
method The paper introduces multi-environment contextual bandits and policy invariance to handle environmental shifts.
result An optimal invariant policy is guaranteed to generalize across environments under suitable assumptions.
FinStressTS creates synthetic benchmarks for financial forecasting, revealing model weaknesses.
problem Limited failure attribution in real-world financial benchmarks.
method Synthetic benchmark with 30 diagnostic environments linked to six mechanism families.
result Model performance varies by mechanism type, with autoregressive models often outperforming Transformers.
OpFlow predicts robust OD flows by learning choice potentials conditioned on spatial exposures.
problem Deep models trained on raw counts are vulnerable to distribution shift.
method OpFlow learns row-centered choice potentials and reconstructs flows by combining them with a calibrated origin scale.
result OpFlow improves robustness under environment shifts, as shown by controlled synthetic shifts and a real-world experiment.
Proposes SGShift to identify shifted features causing model performance degradation under concept shift.
problem Concept shift leading to miscalibration in ML models across domains.
method SGShift method for identifying sparse set of shifted features using feature selection and statistical tools.
result SGShift identifies shifted features more accurately than baseline methods, requires few samples in the shifted domain, and is robust to complex cases.
This paper proposes a method for modeling event sequences with ambiguous timestamps, a time-discounting convolution. Unlike in ordinary time series, time intervals are not constant, small time-shifts have no significant effect, and inputting timestamps or time durations into a model is not effective. The criteria that …
Paper introduces kernel deformed exponential families for sparse continuous attention.
problem Creating efficient attention mechanisms for sparse data.
method Developed kernel deformed exponential families, theoretically and experimentally.
result Kernel deformed exponential families can attend to multiple compact regions of data.
New principle for disentangling latent factors using sparse regularization.
problem Disentangling latent factors from complex data.
method Sparse regularization of latent mechanisms to induce disentanglement.
result Recovery of latent variables up to permutation under certain conditions.
Paper extends sparse alternatives to softmax for continuous domains, enabling efficient attention mechanisms.
problem Efficiently assigning zero probability to irrelevant categories in continuous domains.
method Extend alpha-entmax to continuous domains, introducing continuous-domain attention mechanisms.
result Continuous attention allows attending to time intervals and compact regions, improving text classification, machine translation, and visual question answering.
Paper addresses covariate shift in deep learning regression models.
problem Covariate shift in dependent data from different distributions.
method Sparse-penalized deep neural network (SPDNN) estimator for nonparametric regression.
result Adaptive convergence rates for quantile and Huber regression.
Method identifies shifts leading to large model performance differences.
problem Detecting shifts in distribution that affect model performance.
method Parametric changes in causal mechanisms define robustness sets; worst-case optimization problem approximated as non-convex quadratic.
result Second-order approximation of worst-case loss for small shifts, leading to efficient algorithms.
New method detects novel node categories in graphs with distribution shifts.
problem Detecting novel node categories in graphs with distribution shifts.
method Recall-Constrained Optimization with Selective Link Prediction (RECO-SLIP).
result RECO-SLIP outperforms existing methods in detecting novel node categories.
Sparse Transformers degrade semantic information first, with early layers encoding more.
problem Understanding how sparse Transformers affect learned representations and semantic information.
method Probed Transformers with progressively pruned weights to observe changes in semantic information and model behavior.
result Complex semantic information is first to degrade in sparse Transformers, with early layers encoding more.
TILT improves target domain performance by penalizing an auxiliary component on unlabeled target inputs.
problem Improving performance on target domain under covariate shift.
method TILT uses a novel objective function to decompose the source predictor and penalize an auxiliary component on unlabeled target inputs.
result TILT improves target domain performance over source-only training and other baselines.
DeRegiME forecasts with regime structure, improving probabilistic predictions across various time series.
problem Probabilistic forecasting discards residual uncertainty, and distribution shifts are hard to capture.
method DeRegiME uses a sparse variational Gaussian process with a nonstationary regime-mixing kernel to separate latent uncertainty regimes.
result DeRegiME improves NLPD by 20.3% on average across benchmarks, with gains on CRPS and MSE.
We propose a method for solving statistical mechanics problems defined on sparse graphs. It extracts a small Feedback Vertex Set (FVS) from the sparse graph, converting the sparse system to a much smaller system with many-body and dense interactions with an effective energy on every configuration of the FVS, then learn…
New method for disentangling latent factors with sparse dependencies.
problem Disentangling latent factors from observed variables and past factors.
method Mechanism sparsity regularization and sparse causal graphical model.
result Identifiability of latent factors up to a sparse causal graph.
Blind deconvolution is a ubiquitous problem of recovering two unknown signals from their convolution. Unfortunately, this is an ill-posed problem in general. This paper focuses on the {\em short and sparse} blind deconvolution problem, where the one unknown signal is short and the other one is sparsely and randomly sup…
Predictive models can fail to generalize from training to deployment environments because of dataset shift, posing a threat to model reliability and the safety of downstream decisions made in practice. Instead of using samples from the target distribution to reactively correct dataset shift, we use graphical knowledge …
Many transformations in deep learning architectures are sparsely connected. When such transformations cannot be designed by hand, they can be learned, even through plain backpropagation, for instance in attention mechanisms. However, during learning, such sparse structures are often represented in a dense form, as we d…
This paper explores how environmental properties can simplify reinforcement learning in non-episodic settings.
problem Challenges in reinforcement learning with continuous interaction and sparse delayed rewards.
method Analysis of environment shaping and dynamism properties to simplify learning.
result Properties like environment shaping and dynamism can significantly ease learning in non-episodic, sparse reward settings.
A new hashing method improves accuracy by learning an orthogonal transform.
problem Improving hashing accuracy for similarity search.
method Procrustean Orthogonal Sparse Hashing (POSH) method.
result POSH outperforms state-of-the-art hashing methods.
This paper tackles distribution shift in model-based offline RL, proposing a shifts-aware reward method.
problem Distribution shift challenges model-based offline RL by distorting value estimation and policy optimization.
method The paper disentangles the problem into model bias and policy shift, proposing a shifts-aware reward through probabilistic inference.
result The proposed shifts-aware reward method effectively mitigates distribution shift and improves policy optimization.
Task shift from classification to regression is possible in overparameterized linear models with limited additional data.
problem Transferability of latent knowledge from classification to regression in overparameterized linear models.
method Investigation of task shift in overparameterized linear regression, zero-shot and few-shot cases, with a focus on minimum-norm interpolation.
result Minimum-norm interpolators can transfer latent knowledge from classification to regression with limited additional data.
Paper introduces DP methods for high-dimensional variable selection.
problem Sparse variable selection in high-dimensional learning.
method Pure differentially private estimators using Integer Programming.
result Achieves state-of-the-art empirical support recovery.
DRCD identifies causal direction between continuous and discrete variables using density ratio monotonicity.
problem Inferring causal direction between continuous and discrete variables from observational data.
method Density Ratio-based Causal Discovery (DRCD) method.
result DRCD identifies causal direction between continuous and discrete variables using density ratio monotonicity.