Neural-ANOVA breaks down neural networks into simpler models.
problem Understanding complex neural network decision-making processes.
method Formulates a learning problem to decompose neural networks into lower-order models using ANOVA.
result Demonstrates improved approximation properties compared to other regression methods.
We solve the ANOVA decomposition for categorical inputs.
problem Lack of a closed-form expression for ANOVA decomposition with categorical dependent variables.
method Bridge functional analysis with discrete Fourier analysis to derive a closed-form decomposition.
result Closed-form decomposition for categorical inputs without assumptions.
Proposes ANOVA-TPNN for stable interpretation of complex functions.
problem Stability issues in estimating components of functional ANOVA models.
method Introduces ANOVA-TPNN based on tensor product basis expansion.
result ANOVA-TPNN provides stable estimation of components.
Paper applies ANOVA decomposition for interpretable data approximation.
problem High-dimensional data interpretation and dimensionality reduction.
method ANOVA decomposition and Grouped Transformations for interpretability.
result Ability to rank variable interactions and unimportant variables.
Two ANOVA-based algorithms boost random Fourier feature models for function approximation.
problem Approximating high-dimensional functions with low-order interactions.
method Utilizes ANOVA decomposition to learn low-order functions and index sets of important variables.
result Significantly reduces approximation error compared to existing methods.
We provide a unified view of additive explanations for dependent inputs.
problem Challenges in obtaining a tractable representation and estimating the decomposition for dependent inputs.
method Combining Hilbert space methods with generalized functional ANOVA, we build an explicit decomposition Riesz Basis.
result Proposed a simple yet powerful algorithm to estimate the decomposition from data.
Bayesian-TPNN improves ANOVA-TPNN for detecting higher-order components.
problem Difficulty in incorporating higher-order components in ANOVA-TPNN due to computational and memory constraints.
method Bayesian inference procedure for functional ANOVA model with TPNN basis functions.
result Bayesian-TPNN detects higher-order components with reduced computational cost.
Neural Decomposition breaks down VAE latent structure for better interpretability.
problem Limited interpretability of VAE latent representations.
method Adapted functional ANOVA to VAEs, applying constraints for identifiability.
result Decomposes data variation into latent and fixed input effects.
We describe convolutional networks using harmonic functions.
problem Understanding the function space and smoothness of convolutional networks.
method Using reproducing kernel Hilbert spaces and functional ANOVA decomposition.
result Convolutional networks can be decomposed into a sum of elementary functions.
Given a reproducing kernel Hilbert space H of real-valued functions and a suitable measure mu over the source space D (subset of R), we decompose H as the sum of a subspace of centered functions for mu and its orthogonal in H. This decomposition leads to a special case of ANOVA kernels, for which the functional ANOVA r…
The paper explores SHAP scores and their connection to functional ANOVA, highlighting challenges in approximations.
problem Estimating SHAP scores and understanding their limitations.
method Using the connection to functional ANOVA, the paper outlines the challenges in SHAP approximations and their relation to feature distribution and ANOVA terms.
result Challenges in SHAP approximations are primarily due to feature distribution and the number of ANOVA terms estimated.
Unified framework for feature-based explanations using ANOVA and game theory.
problem Differences between feature-based explanations methods limit their applicability.
method Introduces a unified framework combining fANOVA and cooperative game theory.
result Uncovered similarities and differences between various explanation techniques.
Improved SVM classification with interpretable features from scattered data.
problem Classification of scattered data points in high-dimensional spaces.
method Truncated ANOVA decomposition for sparse feature selection; use of trigonometric or wavelet feature maps.
result Better classification accuracy and interpretability with ℓ1-norm regularization. New Fourier analysis method for non-uniform Boolean hypercube.
problem Non-uniform probability measures on the Boolean hypercube.
method ANOVA-based decomposition, explicit basis, least squares problem.
result Generalization of Fourier analysis for arbitrary probability measures.
New decompositions misattribute differences between populations, even when outcomes are identical.
problem Misattribution of differences between populations using common functional decompositions.
method Extending the Kitagawa-Oaxaca-Blinder decomposition to nonlinear functional decompositions.
result Functional ANOVA and Accumulated Local Effects can misattribute differences even when outcomes are identical in two populations.
Models which estimate main effects of individual variables alongside interaction effects have an identifiability challenge: effects can be freely moved between main effects and interaction effects without changing the model prediction. This is a critical problem for interpretability because it permits "contradictory" m…
Study federates measurement of demographic disparities from quantile sketches.
problem Misalignment of fairness goals with siloed data collection and privacy regulations.
method Federated auditing of demographic parity through score distributions, using Wasserstein--Frechet variance and quantile summaries.
result Proposes a one-shot, communication-efficient protocol to estimate global disparity and its decomposition.
This work uses ANOVA to understand how different factors contribute to test error in machine learning models.
problem Understanding why overparametrized models generalize well despite potentially fitting noise.
method Analysis of variance (ANOVA) to decompose test error into components of variance.
result The interaction between training samples and initialization can dominate variance, and there are phase transitions in variance behavior.
Meta-ANOVA simplifies complex models for better interpretability.
problem Complex models are hard to interpret, limiting their use in fields needing accountability.
method Transforms black-box models into interpretable ANOVA models by screening unnecessary interactions.
result Meta-ANOVA provides an interpretable model for any prediction model, proving asymptotic consistency.
Improved Gaussian process models for interpretable predictions.
problem Complex responses require high-dimensional interaction terms in additive Gaussian processes.
method Orthogonal additive kernel (OAK) with orthogonality constraint on additive functions.
result OAK models achieve similar or better predictive performance with fewer terms, retaining interpretability.
PED-ANOVA efficiently calculates HP importance in arbitrary subspaces.
problem Understanding the role of different hyperparameters in arbitrary subspaces.
method Derive a novel f-ANOVA formulation for arbitrary subspaces and use Pearson divergence (PED) for a closed-form calculation of HP importance.
result Demonstrates successful identification of important HPs in different subspaces.
A new method quickly identifies key variables and interactions.
problem Identifying key variables and interactions in high-dimensional data.
method Kernel trick for sparse orthogonal decomposition in O(# covariates) time.
result Outperforms existing methods for large, high-dimensional data sets.
The paper uses transformed ANOVA to identify important fire detection variables.
problem Identifying key variables for forest fire detection.
method Developed a complete orthonormal system for standard normal distribution, applied Z-score transformation, and used ANOVA approximation.
result Attribute ranking reveals important variables for fire detection.
The focus of this paper is the efficient computation of counterparty credit risk exposure on portfolio level. Here, the large number of risk factors rules out traditional PDE-based techniques and allows only a relatively small number of paths for nested Monte Carlo simulations, resulting in large variances of estimator…
TreeHFD algorithm explains tree ensemble models through hierarchical orthogonality.
problem Difficulty in explaining black-box tree ensemble models.
method TreeHFD algorithm using hierarchical orthogonality constraints.
result TreeHFD estimates Hoeffding decomposition from data samples.
Proposes FOAGP for efficient orthogonal effect decomposition of black-box computer experiments.
problem Challenges in sensitivity analysis of black-box computer experiments with complex, nonlinear functional outputs.
method Functional-output orthogonal additive Gaussian process (FOAGP) with conditional orthogonality constraint.
result Demonstrates effectiveness in orthogonal effect decomposition and variance decomposition through simulations and real-world application.
A new method selects variables efficiently for fast and accurate dynamic system identification.
problem Efficiently selecting variables for scalable Gaussian processes.
method Forward variable selection using Karhunen-Loève decomposition and Gibbs sampling.
result Method yields competitive accuracies and inference times for dynamic systems.
TSL learns separable models to avoid signal cancellation and off-support extrapolation.
problem Signal cancellation and off-support extrapolation in additive models.
method Tensor Separation Learning (TSL) via stagewise greedy procedure with orthogonal refitting.
result TSL avoids information loss caused by marginalizing higher-order interactions.
Paper introduces efficient methods for estimating cross-partial derivatives and sensitivity indices.
problem Efficiently estimating cross-partial derivatives and sensitivity indices in complex models.
method Using randomized points and constraints, the paper develops estimators with optimal convergence rates and low bias.
result The estimators achieve optimal rates of convergence and do not suffer from the curse of dimensionality.
GAMI-Tree uses model-based trees to fit low-order fANOVA models.
problem Fitting interpretable fANOVA models with low-order interactions.
method GAMI-Tree uses model-based trees and a new interaction filtering method.
result GAMI-Tree outperforms EBM and GAMI-Net in predictive performance and interpretability.
Among interpretable machine learning methods, the class of Generalised Additive Neural Networks (GANNs) is referred to as Self-Explaining Neural Networks (SENN) because of the linear dependence on explicit functions of the inputs. In binary classification this shows the precise weight that each input contributes toward…
New ML algorithms improve model interpretability without sacrificing performance.
problem Lack of interpretability in complex machine learning models.
method Developed new algorithms based on fANOVA framework, including GAMI-Lin-T and GAMI-Net.
result GAMI-Lin-T and GAMI-Net perform comparably to EBM and better in interpretability.
Itô processes are the most common form of continuous semimartingales, and include diffusion processes. This paper is concerned with the nonparametric regression relationship between two such Itô processes. We are interested in the quadratic variation (integrated volatility) of the residual in this regression, over a un…
In this article, we propose a new numerical approach to high-dimensional partial differential equations (PDEs) arising in the valuation of exotic derivative securities. The proposed method is extended from Reisinger and Wittum (2007) and uses principal component analysis (PCA) of the underlying process in combination w…
Hypothesis testing is one of the most common types of data analysis and forms the backbone of scientific research in many disciplines. Analysis of variance (ANOVA) in particular is used to detect dependence between a categorical and a numerical variable. Here we show how one can carry out this hypothesis test under the…
New algorithms for hypothesis testing in high-dimensional data are shown to be effective under various noisy conditions.
problem Testing high-dimensional probability measures under noisy conditions.
method Low coordinate degree functions (LCDF) using Efron-Stein decomposition.
result LCDF can effectively test high-dimensional probability measures under noisy channels, with efficacy depending on scalar Fisher information.
Shallow trees in ensemble models make models more interpretable and sometimes better.
problem Lack of transparency in high-performing tree ensemble models.
method Developed an interpretation algorithm to convert tree ensembles into functional ANOVA representations. Proposed strategies to enhance interpretability.
result Shallow trees in ensemble models can lead to better generalization performance and improved interpretability.
LLM generates coherent macroeconomic stress scenarios for portfolio risk assessment.
problem Macro-financial stress testing and portfolio risk assessment using traditional methods.
method Hybrid prompt-RAG pipeline combining structured prompting and retrieval of country fundamentals and news.
result LLM-generated scenarios yield stable tail-risk amplification with limited sensitivity to retrieval choices.
A new measure of causal influence quantifies intrinsic contributions in DAGs.
problem Quantifying intrinsic causal contributions in Directed Acyclic Graphs (DAGs).
method Recursive decomposition of node contributions, structure-preserving interventions, Shapley symmetrization.
result A measure of intrinsic causal contribution that is invariant to node relabeling.
Proposes FARM model combining latent factor and sparse regression.
problem Testing adequacy of latent factor and sparse regression models.
method Factor Augmented sparse linear Regression Model (FARM) with FabTest and ANOVA type tests.
result Model robustness and effectiveness validated through experiments.
DFI maps covariates to latent representations for feature importance.
problem Feature importance when predictors are statistically dependent.
method Disentangled Feature Importance (DFI) using entropic optimal transport.
result DFI yields stable, interpretable, uncertainty-quantified attributions of shared predictive signal.
Efficiently explains model outputs using HSIC, a dependence measure.
problem Efficiently explain model outputs for various architectures.
method HSIC, RKHS, Reproducing Kernel Hilbert Spaces, black-box attribution.
result Up to 8 times faster than previous methods while maintaining fidelity.
Develops monotone tree-based GAMI models using XGBoost.
problem Incorporating monotonicity into GAMI models based on boosted trees.
method Adapting XGBoost algorithm to fit monotone GAMI-Tree models, filtering interactions, and parsing results.
result Fits monotone GAMI-Tree models that are directly interpretable and visualizable.
Unified framework for linear attribution methods in deep learning.
problem Separate theoretical foundations of XAI attribution methods.
method GRALIS (Gradient-Riesz Averaged Locally-Integrated Shapley) framework.
result Unified representation theory for linear attribution methods.
We extend multi-way, multivariate ANOVA-type analysis to cases where one covariate is the view, with features of each view coming from different, high-dimensional domains. The different views are assumed to be connected by having paired samples; this is a common setup in recent bioinformatics experiments, of which we a…
We introduce a semi-parametric Bayesian model for survival analysis. The model is centred on a parametric baseline hazard, and uses a Gaussian process to model variations away from it nonparametrically, as well as dependence on covariates. As opposed to many other methods in survival analysis, our framework does not im…
Neuroimaging research has predominantly drawn conclusions based on classical statistics, including null-hypothesis testing, t-tests, and ANOVA. Throughout recent years, statistical learning methods enjoy increasing popularity, including cross-validation, pattern classification, and sparsity-inducing regression. These t…
Bagging is a device intended for reducing the prediction error of learning algorithms. In its simplest form, bagging draws bootstrap samples from the training sample, applies the learning algorithm to each bootstrap sample, and then averages the resulting prediction rules. We extend the definition of bagging from stati…