Reply to Tetlock et al. on tail risk and probability gap.
problem Expert judgment fails to account for tail risk.
method Comparison of forecasting tournaments and extreme value theory.
result Greater gap between tail expectation and probability properties.
What do binary (or probabilistic) forecasting abilities have to do with overall performance? We map the difference between (univariate) binary predictions, bets and "beliefs" (expressed as a specific "event" will happen/will not happen) and real-world continuous payoffs (numerical benefits or harm from an event) and sh…
New acquisition function improves batch Bayesian active learning.
problem BatchBALD conflates epistemic and aleatoric uncertainty, leading to suboptimal performance.
method Focus on predictive probabilities to separate epistemic uncertainty, leading to better performance and faster evaluation.
result The new acquisition function performs better and allows for larger batches.
Signed pairwise interactions conflate uniqueness, redundancy, and synergy
problem Signed pairwise interactions conflate uniqueness, redundancy, and synergy
method Stochastic Hi-Fi
result Stochastic Hi-Fi recovers structure missed by scalar baselines
PITMonitor monitors model calibration over time with formal error guarantees.
problem Fixed-sample tests applied to models over time can lead to false alarms.
method PITMonitor uses mixture e-processes to detect distributional shifts in probability integral transforms.
result PITMonitor achieves competitive detection rates on river's FriedmanDrift benchmark.
Modern portfolio theory(MPT) addresses the problem of determining the optimum allocation of investment resources among a set of candidate assets. In the original mean-variance approach of Markowitz, volatility is taken as a proxy for risk, conflating uncertainty with risk. There have been many subsequent attempts to al…
Framework for precise recall control in spatial conflation tasks.
problem Precise recall control in large-scale spatial conflation tasks to avoid downstream analytics failures and excessive manual review.
method End-to-end framework using equigrid bounding-box filter, CSR representation, neural ranker, and inverse-variance weighted ensemble of threshold estimators.
result Achieves exact recall with sub-percent variance over tens of millions of geometry pairs, runs on a single TPU v3 core.
Accurately determining dependency structure is critical to discovering a system's causal organization. We recently showed that the transfer entropy fails in a key aspect of this---measuring information flow---due to its conflation of dyadic and polyadic relationships. We extend this observation to demonstrate that this…
A new framework separates classifier calibration and discrimination.
problem Combining reliability and resolution in probabilistic predictions.
method Manokhin Probability Matrix separates reliability and resolution using Spiegelhalter Z-statistic and AUC-ROC.
result Classifiers are categorized into four archetypes: Eagle, Bull, Sloth, and Mole.
OOD detection methods often misidentify OOD points, leading to ineffective safety improvements.
problem Improving model safety through OOD detection methods often leads to incorrect identification of out-of-distribution points.
method Re-examine popular OOD detection procedures based on predictive uncertainty or features of supervised models trained on in-distribution data.
result Popular OOD detection methods incorrectly conflate high uncertainty and far feature-space distance with being out-of-distribution.
Unified framework suppresses model bias in semi-supervised learning with decoupled sampling control.
problem Class imbalance in semi-supervised learning, especially with distributional mismatches.
method Unified framework SC-SSL with decoupled sampling control, explicit expansion capability, and adaptive sampling probabilities.
result Consistent and state-of-the-art performance across various benchmark datasets and distribution settings.
New tools for constructing disintegrations and studying their modes.
problem Difficulty in constructing disintegrations and understanding their modes.
method Developed comprehensive mathematical tools for constructing disintegrations and analyzing their modes.
result Disagreement between restricted density and disintegration density in certain cases.
New research shows calibration error is flawed when dealing with model uncertainty.
problem Current model evaluation techniques conflate model uncertainty with aleatoric uncertainty.
method Posterior predictive checks to evaluate deep learning models.
result Calibration error and variants are incorrect when model uncertainty is present.
When machine predictors can achieve higher performance than the human decision-makers they support, improving the performance of human decision-makers is often conflated with improving machine accuracy. Here we propose a framework to directly support human decision-making, in which the role of machines is to reframe pr…
Financial markets are not random, but hard to predict due to hidden causes and strategic use.
problem Hard to predict financial markets
method Disciplined thesis on the distinction between no-arbitrage, informational efficiency, and net exploitability
result Discovers that markets are hard to predict due to hidden causes and strategic use
New algorithm improves reinforcement learning from partial observations.
problem Inferior performance of algorithms in real-world reinforcement learning due to partial observability.
method Representation-based approach to POMDPs, leading to a tractable algorithm.
result Empirically demonstrates superior performance with partial observations.
Foresight Arena benchmarks AI forecasting on real-world markets, isolating predictive edge.
problem Evaluating AI forecasting ability in real-world markets is challenging due to overfitting, centralized trust, and conflated metrics.
method Permissionless, on-chain benchmark using probabilistic forecasts, commit-reveal protocol, and smart contracts.
result Demonstrates the need for 350 predictions to reliably distinguish agents of different skill levels.
A measure of relative importance of variables is often desired by researchers when the explanatory aspects of econometric methods are of interest. To this end, the author briefly reviews the limitations of conventional econometrics in constructing a reliable measure of variable importance. The author highlights the rel…
Model distinguishes homophily and triadic closure in network analysis.
problem Conflated properties of homophily and transitivity in network analysis.
method Generative model with triadic closure edges and inference procedure.
result Can identify mechanisms responsible for network edges and community structure.
New method disentangles feature importance scores in machine learning.
problem Misinterpretation of feature importance scores due to interactions and dependencies.
method Derive DIP (Disentangled Importance) decomposition of feature importance scores.
result DIP decomposition uniquely separates standalone contributions from interactions and dependencies.
Braided vector fields on spatial subdomains homeomorphic to the cylinder play a crucial role in applications such as solar and plasma physics, relativistic astrophysics, fluid and vortex dynamics, elasticity, and bio-elasticity. Often the vector field's topology -- the entanglement of its field lines -- is non-trivial,…
New method shows data-driven causal studies can be misleading.
problem Misattribution of causality in data-driven earth science studies.
method Subsample-based ensemble approach for robust causality analysis.
result Transfer entropy-based causal graphs can be spurious.
New method improves uncertainty calibration in deep learning.
problem Systematic overconfidence in EDL on out-of-distribution inputs.
method Density-Informed Pseudo-count EDL (DIP-EDL) separates class prediction from uncertainty.
result DIP-EDL achieves asymptotic concentration and enhances robustness and uncertainty calibration.
A novel disentangled graph autoencoder improves treatment effect estimation from networked observational data.
problem Treatment effect estimation from observational data is challenging due to unconfoundedness assumption and latent confounders.
method Proposes a disentangled variational graph autoencoder to disentangle latent factors and enforce factor independence.
result Extensive experiments show superior performance compared to state-of-the-art approaches.
New method reveals true causal functions in nonlinear time series, not just scores.
problem Causal discovery in nonlinear time series often uses scalar edge scores, which hide true function-valued causal influence.
method Formalized function-valued causal influence for additive, contribution-decomposable architectures. Introduced a practical framework based on ICE for estimating causal response functions directly from trained models.
result Edges with indistinguishable scalar scores can exhibit qualitatively different functional behaviors.
CAIRO separates ranking from scaling to improve robustness.
problem Conflating ranking and scaling in regression leads to model vulnerability.
method Two-stage approach: first learns a scoring function, then recovers scale.
result CAIRO recovers true regression function with auto-calibration guarantees.
CCVAE captures label characteristics in VAEs for better representation learning.
problem Capturing rich label characteristics in VAEs without conflating them with label values.
method Developed CCVAE, a novel VAE model that explicitly captures label characteristics in latent space.
result CCVAE allows for effective and general interventions like smooth traversals and diverse conditional generation.
SDE-Net quantifies uncertainty in deep nets using stochastic dynamics.
problem Uncertainty quantification in deep neural networks.
method Viewing DNN transformations as state evolution of a stochastic dynamical system, introducing a Brownian motion term for epistemic uncertainty.
result SDE-Net outperforms existing methods in uncertainty estimation across various tasks.
TSFMs embed non-stationary time series data, revealing specific types of changes.
problem Understanding non-stationarity in TSFMs' embedding spaces.
method Examined mean shifts, variance changes, linear trends, and persistence in TSFMs.
result Different TSFMs exhibit distinct failure modes in detecting non-stationarity.
LOFT separates subspace rotation and transformation for orthogonal fine-tuning.
problem Conflating subspace rotation and transformation in orthogonal fine-tuning.
method LOFT explicitly separates subspace rotation and transformation, using task-aware support selection.
result LOFT recovers principal-subspace orthogonal adaptation and improves efficiency-performance trade-off.
Paper defines feature impact and importance from data, not models.
problem Misinterpretation of feature importance as impact leads to flawed insights.
method Mathematical definitions of feature impact and importance derived from partial dependence curves.
result Feature rankings by these definitions are competitive with existing techniques.
The paper benchmarks saliency methods for time series predictions.
problem Benchmarking saliency methods for time series predictions.
method Compared saliency-based interpretability methods across diverse neural architectures.
result Saliency methods fail to reliably identify feature importance over time in time series data.
IGSD separates task-specific content channels in transformer components by comparing activation replacement with zero ablation.
problem Mechanistic interpretability of transformer components
method IGSD: paired-intervention framework for comparing activation replacement with zero ablation
result IGSD identifies an early-layer content channel in transformer components that standard importance methods underestimate.
Transformers mimic Bayesian reasoning in controlled settings, revealing geometric mechanisms.
problem Verifying if transformers perform Bayesian reasoning rigorously in natural data.
method Constructing Bayesian wind tunnels with known posteriors and proving memorization impossibility.
result Transformers achieve 10−3-10−4 bit accuracy in Bayesian posteriors, while MLPs fail. Model learns brevity by exposing to easy problems, improving efficiency without explicit length penalties.
problem Excessive verbosity in step-by-step reasoning models trained with RLVR.
method Retaining and up-weighting moderately easy problems as implicit length regularizers.
result Model generates solutions that are, on average, nearly twice as short without explicit length penalties.
A new protocol corrects confounding effects to measure alignment-induced activation shifts accurately.
problem Confounding effects in measuring alignment-induced activation shifts using naive methods.
method Introduces a four-variant decomposition to separate alignment shift from template effects.
result Correctly measures alignment-induced activation shifts, recovering behaviorally active subspace.
fiBAG integrates multiplatform genomic data to identify disease markers.
problem Understanding complex mechanisms underlying human diseases from multiplatform genomic data.
method fiBAG uses Gaussian process models and Bayes factors to identify functional evidence and guide variable selection.
result fiBAG improves detection of disease-related markers compared to non-integrative methods.
Introduces LoCA regret to evaluate model-based RL methods.
problem Lack of consistent metrics to evaluate model-based RL methods.
method Inspired by neuroscience, introduces LoCA regret to measure model-based behavior.
result LoCA regret can identify model-based behavior and assess how close methods are to optimal model-based behavior.
Quantum computing poses a threat to Bitcoin and Ethereum, but only to spending and not mining.
problem Quantum computing threat to Bitcoin and Ethereum
method Separation of Shor's and Grover's quantum algorithms
result Quantum algorithms' impact on Bitcoin and Ethereum
Introduces foundation priors for using model-generated data in empirical research.
problem Using model-generated data as real observations in empirical research.
method Introduces foundation priors as an exponential-tilted, generalized Bayesian update of the user's primitive prior.
result Synthetic data reflects both model patterns and user's priors, enabling principled use in empirical work.
ProbFM provides principled uncertainty quantification for financial forecasting.
problem Lack of principled uncertainty quantification in financial applications.
method Probabilistic Time Series Foundation Model with Uncertainty Decomposition using Deep Evidential Regression (DER).
result DER maintains competitive forecasting accuracy while providing explicit epistemic-aleatoric uncertainty decomposition.
Bayesian model clusters diverse 'omics data for disease subtyping.
problem Clustering diverse 'omics datasets conflates multiple structures.
method Multi-view Bayesian mixture model with semi-supervised learning.
result Identifies distinct clusters of patients for stratified medicine.
Bitcoin fails to function as a stable currency or store of value.
problem Bitcoin's failure to serve as a reliable currency or store of value.
method Quantitative finance methods and economic arguments applied to cryptocurrencies.
result Bitcoin does not function as a currency, store of value, or inflation hedge.
TabSODA improves imputation of surveys with skips and ordinal data.
problem Handling structural skips and ordinal responses in survey data.
method TabSODA uses an Elucidated Diffusion Model with skip pattern detection and ordinal awareness.
result TabSODA reduces ordinal missing-at-random (MACE) by up to 23.7% and improves categorical accuracy by up to 9%.
The paper distinguishes between conditional and marginal processes in language models and discusses conditions for usefulness.
problem The conditional nature of language models trained on observed sequences and the need for marginal text-only processes.
method Distinguishing between full conditional language process, marginal text-only process, and model-induced distribution; analyzing assumptions of stationarity and ergodicity.
result The marginal text-only law is useful only when the observed prefix is an approximately sufficient statistic for the latent circumstances relevant to continuation.
Method detects batch heterogeneity in genomic data.
problem Batch effects confound genomic diagnostics.
method Bayesian model evidence clustering.
result Detects batch effects without known labels.
Paper introduces MVS to detect non-Markovian observations in reinforcement learning.
problem Real-world sensors violate Markov property, leading to suboptimal reinforcement learning performance.
method Uses prediction-based Markov Violation Score (MVS) combining random forest and ridge regression.
result MVS detects non-Markovian structure in observation trajectories, quantifying its impact.
The paper explores how AI trading agents' similar information representation can cause financial market instability.
problem Systemic instability in AI-dominated financial markets due to similar information representation.
method Structural multi-agent market model with two-layer decision architecture for AI agents.
result Representation homogeneity can lead to systemic instability in financial markets.