The paper explains DNNs by quantifying interactions among input variables.
problem Understanding and explaining the complex behavior of deep neural networks.
method The paper defines and quantifies the significance of interactions among multiple input variables using the Shapley value.
result The proposed method effectively explains the behavior of DNNs by assigning attribution values to input variables.
We develop a pivotal test to assess the statistical significance of the feature variables in a single-layer feedforward neural network regression model. We propose a gradient-based test statistic and study its asymptotics using nonparametric techniques. Under technical conditions, the limiting distribution is given by …
An AI approach selects variables in linear models.
problem Selecting significant variables in linear regression models.
method Artificial Neural Network trained to determine variable significance based on OLS estimates.
result The AI approach outperforms traditional methods in accuracy and variable selection.
Generative models with both discrete and continuous latent variables are highly motivated by the structure of many real-world data sets. They present, however, subtleties in training often manifesting in the discrete latent being under leveraged. In this paper, we show that such models are more amenable to training whe…
A new method tests variable significance without assuming model correctness.
problem Testing variable significance in the presence of complex interactions.
method Flexible nonparametric or machine learning methods to estimate conditional mean independence.
result Achieves minimax optimal rate in nonparametric testing problem.
TimeCNN improves forecasting by refining cross-variable interactions over time.
problem Multivariate time series forecasting struggles with dynamic and multifaceted cross-variable correlations.
method TimeCNN uses timepoint-independent convolution kernels to capture evolving relationships among variables.
result TimeCNN outperforms state-of-the-art models in real-world datasets with significant computational and speed advantages.
SAR evaluates ML-based linear regression models for statistical significance.
problem Lack of formal statistical significance in ML-based regression models.
method Statistical Agnostic Regression (SAR) using concentration inequalities and worst-case scenario analysis.
result SAR provides a threshold for statistical significance without assuming underlying assumptions.
We develop a simple and computationally efficient significance test for the features of a machine learning model. Our forward-selection approach applies to any model specification, learning task and variable type. The test is non-asymptotic, straightforward to implement, and does not require model refitting. It identif…
New method estimates variable importance for large models efficiently.
problem Estimating variable importance for large, opaque models is computationally challenging.
method Combining early stopping and warm-start techniques for scalable variable importance estimation.
result The method provides theoretical guarantees and demonstrates improved accuracy and computational efficiency.
HNet detects significant associations in mixed data types efficiently.
problem Complex relationships between variables in mixed data types.
method Graphical hypergeometric networks (HNet) using statistical inference.
result HNet outperforms Bayesian structure learning in detecting node links.
Variable selection is of significant importance for classification and regression tasks in machine learning and statistical applications where both predictability and explainability are needed. In this paper, a Copula Entropy (CE) based method for variable selection which use CE based ranks to select variables is propo…
A new catnat function improves gradient descent for categorical variables.
problem Gradient descent challenges with discrete latent categorical variables.
method Replaced softmax with catnat, a hierarchical binary split function.
result Catnat function offers significant advantages in gradient descent.
We investigate the problem of testing whether d random variables, which may or may not be continuous, are jointly (or mutually) independent. Our method builds on ideas of the two variable Hilbert-Schmidt independence criterion (HSIC) but allows for an arbitrary number of variables. We embed the d-dimensional joint …
Study finds dividend policy has no significant effect on IPO stock prices.
problem Impact of dividend policy on IPO price performance.
method Long-run performance statistics and GARCH model, dummy variable used.
result Dividend policy has no significant effect on IPO stock prices.
New method uses SVD entropy to price artworks.
problem Lack of fine measurements in traditional art pricing models.
method SVD entropy of painting images for content measurement.
result SVD entropy positively affects sales price at 1% significance level.
We propose a technique for increasing the efficiency of gradient-based inference and learning in Bayesian networks with multiple layers of continuous latent vari- ables. We show that, in many cases, it is possible to express such models in an auxiliary form, where continuous latent variables are conditionally determini…
The paper shows how neural networks with less decision boundary variability generalize better.
problem Improving neural network generalizability by reducing decision boundary variability.
method Introduces new measures (algorithm DB variability and (ε,η)-data DB variability) to quantify decision boundary variability and proves theoretical bounds on generalizability. result Neural networks with lower decision boundary variability have better generalizability, as shown by extensive experiments and theoretical bounds.
Investigates the number of experiments needed for statistical significance in medication testing.
problem Determining the number of experiments needed for a statistically significant result.
method Examines binomial and general probability distributions, considering placebo efficacy and varying distributions.
result The number of experiments needed can be significantly higher when placebo efficacy is considered.
Bayesian neural network improves feature selection and prediction.
problem Improving feature selection and prediction accuracy in neural networks.
method BNN-ARD with l2-norm feature importance measure.
result Improves variable selection and predictive performance on real-world data.
We summarize our recent findings, where we proposed a framework for learning a Kolmogorov model, for a collection of binary random variables. More specifically, we derive conditions that link outcomes of specific random variables, and extract valuable relations from the data. We also propose an algorithm for computing …
Study on CEF discount in Bangladesh, finds size and maturity impact, turnover negative.
problem Exploring the discount puzzle in closed-end mutual funds in Bangladesh.
method Fixed effects panel regression with diagnostic tests.
result Fund size and maturity positively impact CEF discount, turnover negatively impacts.
Conditional independence tests (CI tests) have received special attention lately in Machine Learning and Computational Intelligence related literature as an important indicator of the relationship among the variables used by their models. In the field of Probabilistic Graphical Models (PGM)--which includes Bayesian Net…
Deep neural networks (DNNs) are famous for their high prediction accuracy, but they are also known for their black-box nature and poor interpretability. We consider the problem of variable selection, that is, selecting the input variables that have significant predictive power on the output, in DNNs. We propose a backw…
Knoop enhances variable selection with over-parameterization and knockoffs.
problem Challenges of variable selection in high-dimensional datasets.
method Generates knockoff variables, integrates them into an over-parameterized model, and uses anomaly-based significance tests.
result Superior performance in variable selection compared to existing methods.
New method for fitting graphical models with latent variables using regularized conditional likelihood.
problem Graphical modeling with latent variables and confounding dependencies.
method Regularized conditional likelihood for exponential family graphical models.
result Framework applicable to broader settings without knowing latent variables' distribution.
Paper proposes a method to identify key variables in thick data.
problem Selecting significant variables for response prediction.
method Permutation tests on candidate variables for feature selection.
result The approach outperforms Lasso in feature selection.
New method controls FDR for sparse GLMs, identifying positive and negative relationships.
problem Sparse GLMs with high-dimensional data and varying sample size.
method Debiased-Lasso estimator and CLIME method for precision matrix estimation.
result Asymptotically controls directional FDR and FDV for sparse GLMs.
A novel non-supervised method detects anomalies in multivariate time series.
problem Detecting anomalies in multivariate time series data.
method Partitioning based on clustering of correlation coefficients.
result Significant improvement in anomaly detection performance.
Adaptive feature normalization improves model robustness to extraneous variables.
problem Degrading model performance due to extraneous variables in deep learning.
method Adaptive feature normalization using instance normalization instead of batch normalization.
result Adaptive normalization leads to significant performance gains across different datasets and architectures.
A new method combines predictors and their lags using supervised PCA for dynamic forecasting.
problem Dynamic forecasting with many predictors.
method Supervised PCA with re-scaling and penalized methods.
result The method outperforms traditional PCA and diffusion-index approaches in prediction.
We propose a procedure for assigning a relevance measure to each explanatory variable in a complex predictive model. We assume that we have a training set to fit the model and a test set to check the out of sample performance. First, the individual relevance of each variable is computed by comparing the predictions in …
Paper proposes a QUBO formulation that reduces binary variables in Bayesian network learning.
problem Reducing the number of binary variables in QUBO formulations for Bayesian network learning.
method Proposes a new QUBO formulation that minimizes binary variables.
result Significantly reduces the number of binary variables required for Bayesian network structure learning.
Recently there has been a significant interest in learning disentangled representations, as they promise increased interpretability, generalization to unseen scenarios and faster learning on downstream tasks. In this paper, we investigate the usefulness of different notions of disentanglement for improving the fairness…
Develops fully Bayesian LVGP for better uncertainty quantification.
problem Uncertainty in qualitative inputs for GP models.
method Maps qualitative inputs to latent variables, uses standard GP over LVs, estimates LVs through ML, develops fully Bayesian approach.
result Significant improvements in prediction accuracy and uncertainty quantification over plug-in approach.
Latent variable models improve RL by facilitating efficient learning and exploration.
problem Improving sample efficiency in reinforcement learning.
method Representation view of latent variable models for state-action value functions, incorporating kernel embeddings and UCB exploration.
result Established sample complexity of the proposed approach in online and offline settings, demonstrated superior performance in benchmarks.
Study on how intraclass variability affects Temporal Ensembling accuracy.
problem Effect of intraclass variability on Temporal Ensembling accuracy.
method Investigated through experiments with varying seed sizes and types on different datasets.
result Significant drop in accuracy with high intraclass variability datasets, more seed images improve accuracy, and seed type impacts overall efficiency.
Many real-world optimization problems require significant resources for objective function evaluations. This is a challenge to evolutionary algorithms, as it limits the number of available evaluations. One solution are surrogate models, which replace the expensive objective. A particular issue in this context are hiera…
Proposes a new method for variable importance using targeted learning.
problem Uncertainty quantification in variable importance metrics.
method Employing the targeted learning (TL) framework for conditional permutation variable importance.
result Improved accuracy in finite sample contexts compared to traditional methods.
In many data exploration tasks it is meaningful to identify groups of attribute interactions that are specific to a variable of interest. For instance, in a dataset where the attributes are medical markers and the variable of interest (class variable) is binary indicating presence/absence of disease, we would like to k…
The problem of inferring the direct causal parents of a response variable among a large set of explanatory variables is of high practical importance in many disciplines. Recent work exploits stability of regression coefficients or invariance properties of models across different experimental conditions for reconstructi…
For statistical learning, categorical variables in a table are usually considered as discrete entities and encoded separately to feature vectors, e.g., with one-hot encoding. "Dirty" non-curated data gives rise to categorical variables with a very high cardinality but redundancy: several categories reflect the same ent…
Study examines downsizing impact on Indian construction firms' profitability.
problem Impact of downsizing layoffs on construction firms' profitability in India.
method Used Co-integration test, OLS, and VAR models on secondary data of 15 companies.
result Employee Expenses and Number of Employees have significant impact on profitability.
CtrlNS learns latent factors and distribution shifts from sparse transitions without prior knowledge.
problem Lack of prior knowledge of domain variables limits causal temporal representation learning.
method Sparse transition assumption and identifiability results from theoretical perspective.
result Effective in identifying distribution shifts and latent factors without prior knowledge.
RISE learns decisions with sensitive variables, improving worst-case outcomes.
problem Uncertainty and bias in decisions due to delayed sensitive variable data.
method Incorporates sensitive variables offline but not at deployment, using quantile or infimum optimization.
result Improves worst-case outcomes for individuals affected by unavailable sensitive variables.
New method selects direct causal parents from large sets of variables.
problem Inferring direct causal parents from many variables, especially nonlinear and cyclic.
method One-vs.-the-rest feature selection approach with theoretical guarantees.
result Significant improvements over existing methods.
Supervised topic models with a logistic likelihood have two issues that potentially limit their practical use: 1) response variables are usually over-weighted by document word counts; and 2) existing variational inference methods make strict mean-field assumptions. We address these issues by: 1) introducing a regulariz…
Study examines credit risk's impact on Vietnamese banks' financial performance.
problem Impact of credit risk on commercial banks' financial performance in Vietnam.
method Dynamic Difference Generalized Method of Moments (dynamic Difference GMM) approach to address autocorrelation, non-constant variance, and endogeneity issues.
result ROE and NIM persist from one year to the next, while NPLR negatively affects ROA and ROE.
Researchers relax the CVF's smoothness requirement to create more flexible flow models.
problem Challenges in constructing flexible density models due to the CVF's smoothness requirement.
method Introduce L-diffeomorphisms as generalized transformations that may violate smoothness on zero Lebesgue-measure sets. result The relaxation allows for the use of non-smooth activation functions like ReLU in residual flows.