Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

2795588361,115 · Jun 202019922001200920172026
48 results for opaque data

TaylorPODA uses Taylor expansions to improve feature attributions for opaque models.

problem Lack of systematic framework for quantifying feature contributions in opaque models.
method Taylor expansion framework with postulates (precision, federation, zero-discrepancy, adaptation).
result TaylorPODA achieves competitive results and provides principled explanations.

A Bloom filter approach combined with Transformer models improves accuracy for machine learning tasks on opaque IDs.

problem Improving accuracy for machine learning tasks on opaque IDs with large vocabulary sizes.
method Applying hash functions to map opaque IDs to multiple hash tokens, similar to a Bloom filter, and using a multi-layer Transformer to process these digests.
result Models outperform those without hashing and sampled softmax, achieving high accuracy with a smaller computational budget.

New method refines model-free evaluation of complex machine learning models.

problem Evaluating the excess risk of opaque machine learning predictors.
method Perturbing derivatives to create pseudo-outcomes and refitting the model twice.
result Upper bound on excess risk derived efficiently without prior function class knowledge.

Paper shows how to quantify uncertainty in medical ML models.

problem Uncertainty in opaque ML models can lead to safety risks in medical applications.
method Introduces Uncertainty Wrapper to quantify uncertainty transparently.
result Demonstrates practical utility of Uncertainty Wrapper in flow cytometry.

The paper tests semantic importance in opaque models using betting.

problem Precise statistical guarantees for semantic concepts in black-box models.
method Formalizes global and local statistical importance via conditional independence and SKIT.
result Shows effectiveness and flexibility of the framework on various models.

Active learning has long been a topic of study in machine learning. However, as increasingly complex and opaque models have become standard practice, the process of active learning, too, has become more opaque. There has been little investigation into interpreting what specific trends and patterns an active learning st…

2017-07-31abs ↗pdf ↗

Representations learnt through deep neural networks tend to be highly informative, but opaque in terms of what information they learn to encode. We introduce an approach to probabilistic modelling that learns to represent data with two separate deep representations: an invariant representation that encodes the informat…

2019-02-08abs ↗pdf ↗

FAST-DAD distills complex ensemble models into faster, more accurate individual models.

problem Deploying complex AutoML ensemble predictors on tabular data is slow, large, and opaque.
method Data augmentation strategy based on Gibbs sampling from a self-attention pseudolikelihood estimator.
result FAST-DAD distillation produces significantly better individual models than standard training.

SMILE improves explainability of machine learning models.

problem Difficulty in understanding and trusting the conclusions of black-box machine learning models.
method Statistical Model-agnostic Interpretability with Local Explanations (SMILE).
result SMILE makes machine learning models more interpretable.

Complex black-box predictive models may have high accuracy, but opacity causes problems like lack of trust, lack of stability, sensitivity to concept drift. On the other hand, interpretable models require more work related to feature engineering, which is very time consuming. Can we train interpretable and accurate mod…

2019-02-28abs ↗pdf ↗

Thanks to the availability of large scale digital datasets and massive amounts of computational power, deep learning algorithms can learn representations of data by exploiting multiple levels of abstraction. These machine learning methods have greatly improved the state-of-the-art in many challenging cognitive tasks, s…

2018-09-28abs ↗pdf ↗

A new method uses conformal prediction to create reliable confidence masks for image super-resolution.

problem Uncertainty quantification in image super-resolution using generative models.
method Conformal prediction techniques applied to a confidence mask for reliable uncertainty communication.
result Strong theoretical guarantees and empirical solid performance in image super-resolution.

KaCGM models provide transparent causal inference from tabular data.

problem Limited auditability in deep causal models for tabular data.
method KaCGM uses Kolmogorov-Arnold Networks to parameterize structural equations, enabling direct inspection and visualization of causal mechanisms.
result KaCGM achieves competitive performance and interpretable causal effects in real-world applications.

Cameras are an essential part of sensor suite in autonomous driving. Surround-view cameras are directly exposed to external environment and are vulnerable to get soiled. Cameras have a much higher degradation in performance due to soiling compared to other sensors. Thus it is critical to accurately detect soiling on th…

2019-05-04abs ↗pdf ↗

Survey connects XAI and surrogate modeling for better understanding of complex systems.

problem Opaque surrogate models hide input-output relationships, hindering decision-making.
method Synthesizes XAI techniques with surrogate modeling workflows.
result Surrogate models can be made more interpretable using XAI methods.

Recently, deep learning becomes the main focus of machine learning research and has greatly impacted many important fields. However, deep learning is criticized for lack of interpretability. As a successful unsupervised model in deep learning, the autoencoder embraces a wide spectrum of applications, yet it suffers fro…

2018-12-31abs ↗pdf ↗

This paper studies generic properties of connections on vector bundles, solving cohomological equations and proving opaque connections.

problem Generic properties of unitary connections on vector bundles over Riemannian manifolds.
method Introduction of operators of uniform divergence type and perturbative arguments from spectral theory.
result The existence of twisted Conformal Killing Tensors (CKTs) is generically solved, and connections are generically opaque.

Explainable Artificial Intelligence (XAI)has received a great deal of attention recently. Explainability is being presented as a remedy for the distrust of complex and opaque models. Model agnostic methods such as LIME, SHAP, or Break Down promise instance-level interpretability for any complex machine learning model. …

2019-03-27abs ↗pdf ↗

Bayesian framework improves survival prediction accuracy and uncertainty quantification.

problem Inaccurate uncertainty estimates in survival models.
method Bayesian framework combining variational inference, neural multi-task logistic regression, and sparsity-inducing prior.
result Better quantification of survival uncertainty and more accurate predictions.

DORA analyzes deep neural networks' internal representations to detect spurious correlations.

problem Detecting spurious correlations in deep neural networks' internal representations.
method DORA uses Extreme-Activation (EA) distance measure to assess representation similarities.
result Identifies internal representations capable of detecting spurious correlations.

AI-Interpret transforms opaque policies into simple, interpretable decision rules.

problem Designing effective decision aids for professionals to mitigate decision-making biases.
method Combining imitation learning, program induction, and clustering to transform learned policies into interpretable descriptions.
result Providing interpretable decision rules as flowcharts significantly improves people's planning strategies and decisions.

dalex simplifies model exploration and fairness for Python developers.

problem Model black-box nature and risks of discrimination, lack of reproducibility, and data drift.
method Model-agnostic interface for interactive model exploration.
result Enhances model transparency and accountability through interactive explainability and fairness.

Due to the capability of deep learning to perform well in high dimensional problems, deep reinforcement learning agents perform well in challenging tasks such as Atari 2600 games. However, clearly explaining why a certain action is taken by the agent can be as important as the decision itself. Deep reinforcement learni…

2019-02-01abs ↗pdf ↗

Tree-Query uses LLMs to discover causal relationships in a transparent, interpretable manner.

problem Error propagation in classical causal discovery methods and opaque, confidence-free behavior of recent LLM-based causal oracles.
method Tree-Query is a tree-structured, multi-expert LLM framework that reduces causal discovery to queries about backdoor paths and dependencies.
result Tree-Query provides interpretable judgments with robustness-aware confidence scores and improves structural metrics over LLM baselines.

The thesis tackles two stochastic control problems in capital structure and portfolio choice.

problem Optimizing banks' dividend and recapitalization policies and individual's life-cycle portfolio choice.
method Developed stochastic control models to calibrate and analyze U.S. banks' asset values and optimal portfolio selection models.
result Calibrated model reveals that noise in reported asset values can hide up to one-third of true asset return volatility and increase banks' market equity value by 7.8%.

Although deep learning models have been successfully applied to a variety of tasks, due to the millions of parameters, they are becoming increasingly opaque and complex. In order to establish trust for their widespread commercial use, it is important to formalize a principled framework to reason over these models. In t…

2018-11-11abs ↗pdf ↗

This paper provides a guide to feature importance methods for better scientific inference.

problem Limited understanding of data-generating process due to opaque ML model mechanisms.
method Comprehensive review and new proofs of global feature importance methods.
result Facilitates a thorough understanding and concrete recommendations for FI methods.

Although neural networks can achieve very high predictive performance on various different tasks such as image recognition or natural language processing, they are often considered as opaque "black boxes". The difficulty of interpreting the predictions of a neural network often prevents its use in fields where explaina…

2018-12-03abs ↗pdf ↗

Automated feature engineering improves interpretable models without manual work.

problem Lack of interpretability in complex models causes trust and stability issues.
method Use elastic black-box models to create simpler, interpretable glass-box models.
result Extracted features from complex models improve linear model performance.

Study highlights how lazy data practices in fair ML research can unfairly impact minority groups.

problem Lazy data practices in fair ML research can unfairly impact minority groups.
method Systematic study of 280 experiments across 142 publications on 142 datasets.
result Unreflective data practices lead to biased findings and unfair treatment of minorities.

The paper advocates for interpretable, accountable, reproducible machine learning in medicine.

problem Black box models in medicine lack transparency and regulatory approval.
method Intrinsically interpretable modeling approaches and collaborative learning paradigms.
result Interpretable machine learning models can support clinical decisions and gain regulatory approval.