Recent years have seen much research on fairness in machine learning. Here, mean difference (MD) or demographic parity is one of the most popular measures of fairness. However, MD quantifies not only discrimination but also explanatory bias which is the difference of outcomes justified by explanatory features. In this …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,694 papers · 148 categories
Trend · papers per month
48 results for “Explanatory Model Analysis”
Style Miner generates stable and significant style factors for time series analysis.
problem Finding significant and stable explanatory factors in high-dimensional time series data.
method Proposes a reinforcement learning method to balance explanatory power and stability constraints.
result Outperforms existing methods by a large margin and achieves a 10% gain in R-squared explanatory power.
PROD method improves high-dimensional regression by handling strong correlations.
problem Violation of Irrepresentable Condition in LASSO for high-dimensional data.
method PROD procedure based on orthogonal decomposition of design matrix.
result PROD enhances performance of high-dimensional penalized regression.
Principal component regression (PCR) is a two-stage procedure: the first stage performs principal component analysis (PCA) and the second stage constructs a regression model whose explanatory variables are replaced by principal components obtained by the first stage. Since PCA is performed by using only explanatory var…
Enhances patient failure prediction using dynamic survival models.
problem Lack of precise individual level prediction in conventional models.
method Developed counterfactual dynamic survival model (CDSM).
result Inflection point of estimated survival curves predicts patient failure time.
A new framework for time series analysis using state-space learning.
problem Ineffectiveness of traditional Kalman filtering in handling big data and multiple explanatory variables.
method State Space Learning (SSL) framework using statistical learning for high-dimensional regression.
result SSL outperforms traditional methods in subset selection and forecasting accuracy.
Paper uses interbank contagion to predict U.S. bank defaults, finding it highly explanatory.
problem Predicting U.S. bank defaults using interbank contagion.
method Regression and neural network models were used to analyze U.S. commercial bank data.
result Interbank contagion is highly explanatory in default prediction, often outperforming established metrics.
This paper compares two stock factor models in China's A-share market.
problem Contradicting results in existing research on stock factor models.
method Empirical analysis using China's A-share data from 2005-2020, orthogonalizing redundant factors, and 25-group portfolio returns calculation.
result The five-factor model outperforms the three-factor model in explaining excess return rates.
Bayesian framework explains diverse explanatory values.
problem Understanding and predicting human preferences for explanations.
method Developed a Bayesian account to integrate various explanatory values.
result Core values from psychology, statistics, and philosophy emerge from a common framework.
Feature-free regression krigingphysics.soc-ph
FFRK automatically extracts features for spatial interpolation without external variables.
problem Spatial interpolation challenges, especially nonstationarity and lack of explanatory variables.
method Feature-Free Regression Kriging (FFRK) method that extracts geospatial features.
result FFRK outperforms classical methods in predicting heavy metal concentrations.
Interactive EMA combines multiple explainability methods to improve model understanding.
problem Isolated explanations of machine learning models lead to misunderstandings and wrong reasoning.
method Interactive EMA (IEMA) combines multiple explainability methods sequentially.
result Interactive EMA increases the performance and confidence of human decision making.
The paper develops a method to model high-dimensional data with many variables and weak signals.
problem Modeling high-dimensional dependent data with many explanatory variables and low signal-to-noise ratio.
method Penalized regression for high-dimensional data, factor modeling of residuals, high-dimensional white noise testing, projected Principal Component Analysis.
result Established asymptotic properties of the proposed method for high-dimensional data.
Improved neural network model for predicting latent budgets in compositional data.
problem Predicting response variables in compositional data with non-negativity constraints.
method LBA-NN, a feed forward neural network model that incorporates K-means clustering for interpretation.
result LBA-NN outperforms traditional LBA in prediction accuracy, specificity, recall, and mean square error.
XGL uses global explanations to guide human supervision in machine learning.
problem Improving model quality through human-machine interaction.
method XGL employs global explanations to guide human selection of informative examples.
result XGL avoids overselling the model's quality and performs comparably to other strategies.
Realized moments of higher order computed from intraday returns are introduced in recent years. The literature indicates that realized skewness is an important factor in explaining future asset returns. However, the literature mainly focuses on the whole market and on the monthly or weekly scale. In this paper, we cond…
A measure of relative importance of variables is often desired by researchers when the explanatory aspects of econometric methods are of interest. To this end, the author briefly reviews the limitations of conventional econometrics in constructing a reliable measure of variable importance. The author highlights the rel…
Paper improves deep learning convergence rates for low-dimensional data.
problem Sub-optimal rates in deep learning due to unrealistic assumptions on intrinsic dimension.
method Introduced an entropic notion of intrinsic dimension for exponential families and demonstrated improved convergence rates.
result Test error scales as , improving on best-known rates.
Probabilistic learning for binary classification with categorical variables.
problem Binary classification with categorical covariates.
method Probabilistic analysis and two algorithms for learning boolean functions.
result Effective learning of boolean functions from binary data.
We propose a procedure for assigning a relevance measure to each explanatory variable in a complex predictive model. We assume that we have a training set to fit the model and a test set to check the out of sample performance. First, the individual relevance of each variable is computed by comparing the predictions in …
RelatIF selects more intuitive training examples for explaining model predictions.
problem Influence functions identify outliers as explanatory examples, leading to poor explanations.
method RelatIF separates global and local influence, optimizing for local relative to global effects.
result Examples selected by RelatIF are more intuitive than those from influence functions.
Root Cause Analysis for Anomalies is challenging because of the trade-off between the accuracy and its explanatory friendliness, required for industrial applications. In this paper we propose a framework for simple and friendly RCA within the Bayesian regime under certain restrictions (that Hessian at the mode is diago…
Disentangling by Partitioning: A Representation Learning Framework for Multimodal Sensory Datastat.ML
Multimodal sensory data resembles the form of information perceived by humans for learning, and are easy to obtain in large quantities. Compared to unimodal data, synchronization of concepts between modalities in such data provides supervision for disentangling the underlying explanatory factors of each modality. Previ…
Study predicts customer data sharing in Open Banking and explains key factors.
problem Predicting and explaining customer data sharing in Open Banking environments.
method Hybrid data balancing strategy with ADASYN and NEARMISS, XGBoost models, SHAP, CART.
result 91.39% accuracy for inflow and 91.53% for outflow predictions, revealing influential features.
Interpretable representations improve explainable AI by translating complex data into understandable concepts.
problem Many explainers use interpretable representations but overlook their full potential and assumptions.
method An in-depth analysis of interpretable representations for tabular, image, and text data, identifying strengths, weaknesses, and desiderata.
result Linear model quantifies interpretable concepts' influence on black-box predictions, revealing their explanatory properties and manipulability.
Proposes counterfactual explainability for causal attribution, extending variance analysis methods.
problem Lack of mechanistic understanding in existing tools for explaining complex models.
method Extends global sensitivity analysis methods to causal explanations using directed acyclic graphs.
result Developed methods to estimate counterfactual explainability and applied to income inequality analysis.
Study tests if equity factors explain Bitcoin's risk and returns.
problem Explaining Bitcoin's risk and return with equity factors.
method Applied statistical methods to test Fama-French factors on Bitcoin's excess returns.
result Fama-French factors have explanatory power on Bitcoin's risk and returns.
This text discusses several popular explanatory methods that go beyond the error measurements and plots traditionally used to assess machine learning models. Some of the explanatory methods are accepted tools of the trade while others are rigorously derived and backed by long-standing theory. The methods, decision tree…
Motivated by the practical challenge in monitoring the performance of a large number of algorithmic trading orders, this paper provides a methodology that leads to automatic discovery of the causes that lie behind a poor trading performance. It also gives theoretical foundations to a generic framework for real-time tra…
101 Formulaic Alphasq-fin.PM
We present explicit formulas - that are also computer code - for 101 real-life quantitative trading alphas. Their average holding period approximately ranges 0.6-6.4 days. The average pair-wise correlation of these alphas is low, 15.9%. The returns are strongly correlated with volatility, but have no significant depend…
Study uses geometric algebra to analyze credit cycles, revealing dangerous feedback loops.
problem Understanding and predicting dangerous feedback loops in credit cycles.
method Represent economic states as multi-vectors in Clifford algebra, focusing on bivector elements for rotational coupling.
result Geometric relationship between unemployment and credit contraction shifts from simple correlation to dangerous rotational dynamics during crises.
A new baseline for Shapley values in MLPs considers model use.
problem Lack of a robust baseline for Shapley values in neural networks.
method Proposes a neutrality-based baseline for Shapley values.
result Empirically validated the proposed baseline for binary classification tasks.
Financial factors selection with knockoffs: fund replication, explanatory and prediction networksq-fin.ST
Knockoffs method selects financial factors, controlling false discoveries.
problem Controlling false discoveries in financial factor selection.
method Apply knockoff procedure to build fake factors.
result Shows versatility in fund replication and network inference.
Proposes models to better represent ordinal data with non-unimodal distributions.
problem Real-world ordinal data often have non-unimodal conditional probability distributions.
method Develops approximately unimodal likelihood models to better represent non-unimodal CPDs.
result Proposed models can effectively represent both unimodal and nearly unimodal CPDs.
We propose a K-sparse exhaustive search (ES-K) method and a K-sparse approximate exhaustive search method (AES-K) for selecting variables in linear regression. With these methods, K-sparse combinations of variables are tested exhaustively assuming that the optimal combination of explanatory variables is K-sparse. By co…
Ensembling classification models based on phalanxes of variables with applications in drug discoverystat.ML
Statistical detection of a rare class of objects in a two-class classification problem can pose several challenges. Because the class of interest is rare in the training data, there is relatively little information in the known class response labels for model building. At the same time the available explanatory variabl…
Develops method to assess feature importance in black-box models for unconditional distribution.
problem Lack of methods to analyze feature importance in black-box models for unconditional distribution.
method Approximation method to compute feature importance curves for unconditional distribution.
result Produces sparse and faithful results, computationally efficient.
Principal component regression (PCR) is a widely used two-stage procedure: principal component analysis (PCA), followed by regression in which the selected principal components are regarded as new explanatory variables in the model. Note that PCA is based only on the explanatory variables, so the principal components a…
There has recently been a surge of work in explanatory artificial intelligence (XAI). This research area tackles the important problem that complex machines and algorithms often cannot provide insights into their behavior and thought processes. XAI allows users and parts of the internal system to be more transparent, p…
"Why Should I Trust Interactive Learners?" Explaining Interactive Queries of Classifiers to Usersstat.ML
Although interactive learning puts the user into the loop, the learner remains mostly a black box for the user. Understanding the reasons behind queries and predictions is important when assessing how the learner works and, in turn, trust. Consequently, we propose the novel framework of explanatory interactive learning…
SAR evaluates ML-based linear regression models for statistical significance.
problem Lack of formal statistical significance in ML-based regression models.
method Statistical Agnostic Regression (SAR) using concentration inequalities and worst-case scenario analysis.
result SAR provides a threshold for statistical significance without assuming underlying assumptions.
CD-RCA method identifies causal relationships in prediction errors without predefined graphs.
problem Challenges in diagnosing prediction errors due to lack of transparency in black-box models.
method Causal-Discovery-based Root-Cause Analysis (CD-RCA) method that estimates causal relationships without predefined causal graphs.
result CD-RCA outperforms heuristic attribution methods in identifying variable contributions to prediction errors.
Principal component regression (PCR) is a two-stage procedure that selects some principal components and then constructs a regression model regarding them as new explanatory variables. Note that the principal components are obtained from only explanatory variables and not considered with the response variable. To addre…
Study improves understanding of network degree distributions using non-linear ERGs.
problem Lack of models capable of accounting for the variance of empirical degree distributions.
method Defined a fitness-induced variant of the two-star model to reproduce sample variance.
result Non-linear ERGs can reproduce the sample variance of empirical degree distributions.
Satellite imagery and remote sensing provide explanatory variables at relatively high resolutions for modeling geospatial phenomena, yet regional summaries are often desirable for analysis and actionable insight. In this paper, we propose a novel method of inducing spatial aggregations as a component of the machine lea…
Nonlinear kernel regression models are often used in statistics and machine learning because they are more accurate than linear models. Variable selection for kernel regression models is a challenge partly because, unlike the linear regression setting, there is no clear concept of an effect size for regression coeffici…
Enhancing Variable Importance in Random Forests: A Novel Application of Global Sensitivity Analysisstat.ML
Global Sensitivity Analysis improves feature importance ranking in Random Forests.
problem Improving feature importance ranking in Random Forests.
method Applying Global Sensitivity Analysis to Random Forests for feature ranking.
result Our method provides a novel way to rank features based on their importance.
The problem of inferring the direct causal parents of a response variable among a large set of explanatory variables is of high practical importance in many disciplines. Recent work exploits stability of regression coefficients or invariance properties of models across different experimental conditions for reconstructi…
We investigate entropy as a financial risk measure. Entropy explains the equity premium of securities and portfolios in a simpler way and, at the same time, with higher explanatory power than the beta parameter of the capital asset pricing model. For asset pricing we define the continuous entropy as an alternative meas…