ECV method optimizes ensemble parameters for randomized ensembles.
problem Efficient tuning of ensemble parameters in randomized ensembles.
method ECV (Extrapolated Cross-Validation) method for tuning ensemble and subsample sizes.
result ECV yields δ-optimal ensembles for squared prediction risk.
Distributional shift is one of the major obstacles when transferring machine learning prediction systems from the lab to the real world. To tackle this problem, we assume that variation across training domains is representative of the variation we might encounter at test time, but also that shifts at test time may be m…
New framework shows ERM is optimal for both interpolation and extrapolation in domain generalization.
problem Formalizing and solving the challenges of domain generalization.
method Reformulated domain generalization as an online game between a risk-minimizing player and an adversary.
result ERM is minimax-optimal for both interpolation and extrapolation in domain generalization.
Emputation learns imputation models guided by missingness assumptions.
problem Learning imputation models for missing data given observed data.
method Guided by specific missingness assumptions, Emputation trains a deep generative model to learn the extrapolation distribution of missing variables.
result The population minimizer of the emputation risk recovers the target extrapolation distribution under various identification assumptions.
LLMs overestimate stock returns and are less accurate at predicting extreme outcomes.
problem Behavioral biases in LLMs' stock return forecasts.
method Comparison of LLM forecasts with crowd-sourced estimates and historical data.
result LLMs overestimate stock returns and are less accurate at predicting extreme outcomes.
New method approximates CVaR with less data for heavy-tailed risks.
problem Lack of data for accurate CVaR approximation in heavy-tailed distributions.
method Importance sampling based extrapolation for heavy-tailed distributions.
result Statistically consistent approximations with reduced data requirements.
RTE enables extrapolation to new tasks by learning task transformations.
problem Learning systems struggle to generalize to unseen tasks.
method Relational Task Extrapolator (RTE) learns task transformations to enable extrapolation.
result RTE substantially outperforms existing approaches on extrapolation tasks.
We extend nonparametric models to handle extrapolation, providing bounds for inference.
problem Challenges in nonparametric statistical inference when evaluating outside the conditioning variable's support.
method Introduced a class of extrapolation assumptions and a consistent estimation procedure to handle extrapolation.
result Validated extrapolation-aware conclusions through various applications and real-world data.
The market practice of extrapolating different term structures from different instruments lacks a rigorous justification in terms of cash flows structure and market observables. In this paper, we integrate our previous consistent theory for pricing under credit, collateral and funding risks into term structure modellin…
Binary encoding enables neural networks to extrapolate periodic functions.
problem Extrapolating periodic functions without prior knowledge of their form.
method Normalized Base-2 Encoding (NB2E) for continuous numerical values.
result MLPs using NB2E can successfully extrapolate diverse periodic signals.
Method controls extrapolation in prediction profiles for statistical and machine learning models.
problem Avoiding invalid predictions due to extrapolation in prediction profiles.
method Genetic algorithm optimization over constrained factor regions.
result Optimal factor settings without constraint are often invalid and extrapolated.
Neural networks struggle with extrapolation, but a new framework allows them to learn counterfactual invariances.
problem Neural networks' inability to extrapolate beyond training data distribution.
method Introduces a learning framework that allows neural networks to extrapolate over group transformations based on counterfactual invariances.
result Neural networks can learn counterfactual invariances from a single environment, overcoming their limitations in extrapolation.
Study shows LLMs can extrapolate rules from out-of-distribution prompts.
problem Understanding LLMs' ability to generalize from unexpected inputs.
method Formal languages and rule-based scenarios to evaluate LLMs' OOD behavior.
result LLMs can extrapolate rules from out-of-distribution prompts, even in complex scenarios.
The aim of this paper is to study the construction of prospective mortality tables from a low number of persons subjected to risk. The presented models are the Lee-Carter and log-Poisson methods respectively. The low number of people subjected to risk, particularly noticed for the persons who are getting on, implies th…
New methods reduce extrapolation errors in feature importance.
problem Flawed feature importance methods using unrestricted permutations lead to extrapolation errors.
method Three new approaches: conditional model reliance, Knockoffs with Gaussian transformation, and restricted ALE plot designs.
result Theoretical and numerical results show our strategies reduce/eliminate extrapolation.
New RL method handles diverse human operator data.
problem Handling data from multiple, unknown policies.
method Domain-Invariant Model-based Offline RL (DIMORL) with Risk Extrapolation (REx).
result Models trained with REx generalize better across different demonstrators.
Equity risk premium is a central component of every risk and return model in finance and a key input to estimate costs of equity and capital in both corporate finance and valuation. An article by Damodaran examines three broad approaches for estimating the equity risk premium. The first is survey based, it consists in …
New models extrapolate false alarms in ASV without new data.
problem Reliable extrapolation of false alarm rates in ASV without new speaker data.
method Generative models in ASV score space for arbitrary systems.
result Models accurately extrapolate false alarm rates for large speaker populations.
Neural networks extrapolate poorly in simple tasks but succeed in complex ones.
problem Understanding neural networks' extrapolation capabilities and conditions for success.
method Analyzing ReLU MLPs and GNNs, connecting to neural tangent kernel.
result ReLU MLPs learn linear functions but not most nonlinear ones, while GNNs succeed in complex tasks due to task-specific non-linearities.
A new principle for extrapolating regression outside training data.
problem Regression extrapolation when predictions are outside training data range.
method Data-adaptive marginal transformation and simple relationship assumption.
result Progression method offers guarantees on approximation error beyond training data range.
We extend return extrapolation to incorporate asymmetry and saturation, finding that asymmetric nonlinear extrapolation leads to lower welfare loss.
problem Optimal portfolio choice under stochastic volatility
method Smooth, nonlinear extrapolation function with sentiment and variance hedging
result Lower welfare loss with asymmetric nonlinear extrapolation
Logical neural networks solve mazes by filling dead ends, but not all methods generalize well.
problem Understanding how logical neural networks extrapolate solutions to mazes.
method Examined recurrent and implicit neural networks trained on maze-solving tasks.
result Models fail to generalize well to diverse maze sizes, suggesting limitations in learning scalable algorithms.
Accelerates Riemannian gradient methods with extrapolation.
problem Optimizing functions on manifolds efficiently.
method Extrapolating iterates in Riemannian gradient descent.
result Achieves optimal convergence rate and computational advantage.
The paper tackles extrapolation in extreme regions of regression problems.
problem Extrapolation on the tails of covariates in continuous regression problems.
method Statistical regression on a subsample of furthest observations, focusing on their angular components, using multivariate regular variation theory.
result Quantifies predictive performance on tail regions in terms of excess risk, presenting it as a finite sample risk bound with a bias-variance decomposition.
Improves reinforcement learning extrapolation in Gridworlds.
problem Generalizing to unseen states in reinforcement learning.
method Avoiding deterministic action choice, using ego-centric representation, incorporating symmetry, and adding an entropy term.
result Significant improvement in extrapolation performance in Gridworld environments.
New methods improve neural network extrapolation to long sequences.
problem Neural networks struggle with extrapolation to very long or adversarial sequences.
method Activation binning and localized differentiable memory architecture.
result No extrapolation errors detected within memory constraints.
We introduce a simple approach for testing the reliability of homogeneous generators and the Markov property of the stochastic processes underlying empirical time series of credit ratings. We analyze open access data provided by Moody's and show that the validity of these assumptions - existence of a homogeneous genera…
Concept modulation models unify identifiability and extrapolation in conditional latent variable models.
problem Reliable generalization in conditional latent variable models
method Concept modulation models (CMMs) with structure AoΛoCoX result Lifts identifiability to conditional settings and controls extrapolation through attribute potentials.
NSR enables neural networks to reason with continuous numbers and extrapolate.
problem Quantitative reasoning and extrapolation in neural networks.
method Proposes Neural Status Register (NSR) for continuous number reasoning.
result NSR achieves extrapolation to numbers many orders of magnitude larger than training data.
Extends return extrapolation to nonlinear, asymmetric functions under stochastic volatility.
problem Behavioral anomalies in portfolio choice under stochastic volatility.
method Smooth, nonlinear, asymmetric extrapolation function; CRRA investor; Heston stochastic volatility; Hamilton-Jacobi-Bellman equation; Numerical solutions (finite-difference ADI, deep learning-driven iterative).
result Saturation acts as an endogenous correction mechanism, reducing welfare loss.
This paper explores conditions for neural networks to extrapolate to new domains.
problem Understanding when neural networks can extrapolate to unseen domains.
method Analyzes conditions for nonlinear models to extrapolate under specific distribution shifts.
result Neural networks of the form f(x)=∑fi(xi) can extrapolate if feature covariance is well-conditioned. The paper explores how to extrapolate from limited data points using causal mechanisms.
problem Handling distribution shifts with limited target samples.
method Formulates the extrapolation problem with a latent-variable model embodying the minimal change principle in causal mechanisms, and identifies conditions for identification.
result Theoretical understanding and practical methods for extrapolation without requiring an on-support target distribution.
Proposes RVP to address theoretical concerns of V-REx for OOD generalization.
problem Theoretical concerns about V-REx's motivation and utility.
method Risk Variance Penalization (RVP) modifies V-REx's regularization.
result RVP discovers a robust predictor and finds invariant predictors under certain conditions.
Method generates plausible financial stress scenarios using large deviations.
problem Misleading risk management by overlooking or overemphasizing implausible scenarios.
method Exploits large-deviations principle to concentrate risk factors near most likely stress configurations.
result Can generate informative stress scenarios even with limited historical data.
Diverging Flows detects extrapolations in flow models, ensuring reliable predictions.
problem Flow models extrapolate into invalid data, leading to silent failures.
method Structurally enforce inefficient transport for off-manifold inputs.
result Effective detection of extrapolations without compromising predictive fidelity or inference latency.
Proposes a new method to avoid model extrapolation in Shapley values.
problem Model extrapolation in marginal Shapley values leads to unreliable explanations.
method Proposes a new approach that avoids model extrapolation using marginal averaging and causal information.
result Demonstrates the impacts of model extrapolation on Shapley values and proposes a new method to avoid it.
Proposes a new tail risk measure based on the most probable maximum risk event size.
problem Current risk measures like VaR and ES are limited in their applicability and require specifying a confidence level.
method Develops a new risk measure called MPMR that does not require a confidence level and scales with the length of the time interval.
result The new risk measure, MPMR, scales with the number of observations by a power law, allowing for reliable estimations of long-term risks based on short-term estimations.
Paper tackles intervention extrapolation using identifiable representations.
problem Predicting effects of unseen interventions on outcomes.
method Combines identifiable representation learning with autoencoders to enforce linear invariance.
result Identifiable representations enable non-linear extrapolation of interventions.
Paper improves PINNs' extrapolation by TL and adaptive AFs.
problem PINNs' poor extrapolation performance and sensitivity to AFs.
method Transfer learning within an extended domain and adaptive activation functions.
result Average 40% reduction in relative L2 error and 50% in mean absolute error in extrapolation domain.
We present a framework on how to hedge the interest rate sensitivity of liabilities discounted by an extrapolated yield curve. The framework is based on functional analysis in that we consider the extrapolated yield curve as a functional of an observed yield curve and use its Gâteaux variation to understand the sensiti…
The paper tackles extrapolation in generative models by enforcing independence of mechanisms.
problem How to make generative models extrapolate to new, unseen environments?
method Developed a theoretical framework for independence of mechanisms, demonstrated on toy examples and real-world data.
result Extrapolation capabilities of generative models can be improved by enforcing independence of mechanisms explicitly during training.
Methodology extrapolates wind fields from sparse data with uncertainty quantification.
problem Extrapolating wind fields from limited measurements with uncertainty.
method Nonparametric Bayesian dictionary learning for sparse/incomplete data.
result Enhanced extrapolation accuracy, even in high-dimensional data.
BiPE blends intra-segment and inter-segment encodings for better length extrapolation.
problem Improving length extrapolation in language models.
method Bilevel Positional Encoding (BiPE) that separates intra-segment and inter-segment encodings.
result BiPE enhances length extrapolation across various text modalities.
Neural Network based controllers hold enormous potential to learn complex, high-dimensional functions. However, they are prone to overfitting and unwarranted extrapolations. PAC Bayes is a generalized framework which is more resistant to overfitting and that yields performance bounds that hold with arbitrarily high pro…
Neural networks are surprisingly good at interpolating and perform remarkably well when the training set examples resemble those in the test set. However, they are often unable to extrapolate patterns beyond the seen data, even when the abstractions required for such patterns are simple. In this paper, we first review …
STMT predicts compounds in unknown areas with trend reflection.
problem Predicting properties of unknown compounds in the extrapolation area.
method Stochastic Threshold Model Trees (STMT) that reflects data trends.
result STMT improves prediction accuracy for one compound in the extrapolation area.
A new method improves language model extrapolation without changing training.
problem Improving language model performance on sequences not seen during training.
method Perturbation-based training procedure for language models.
result The method consistently improves out-of-support prediction.
We consider a generic convex optimization problem associated with regularized empirical risk minimization of linear predictors. The problem structure allows us to reformulate it as a convex-concave saddle point problem. We propose a stochastic primal-dual coordinate (SPDC) method, which alternates between maximizing ov…