Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,694 papers · 148 categories

Trend · papers per month

106211317422 · Jun 202019922001200920172026
48 results for Evaluation Methodology

A new network-based method for high-level data classification without normalization.

problem Evaluation and classification of data with hidden attribute interactions.
method A new network-building methodology based on attribute-attribute interactions, avoiding normalization.
result Improvement in accuracy of high-level classification using betweenness centrality.

This paper critiques flawed MVTS anomaly detection evaluation methods and proposes a simple baseline.

problem Flawed evaluation methods in MVTS anomaly detection research.
method Robust evaluation protocols, including PCA-based baseline.
result Simple PCA-based baseline outperforms many DL approaches.

The study addresses biases in evaluating molecular optimization methods and proposes methods to reduce these biases.

problem Biases in in silico evaluation of molecular optimization methods.
method Discussion and empirical investigation of bias reduction methods for predictor misspecification and sample reuse.
result Empirical investigation of bias reduction methods for predictor misspecification and sample reuse.

Proposes a method to choose thresholds for LLM evaluation metrics.

problem Ensuring reliable large language models (LLMs) with correct threshold selection.
method Identify risks, stakeholders' risk tolerance, and use ground-truth data to determine thresholds.
result Demonstrates a concrete example with the Faithfulness metric and HaluBench dataset.

We introduce a methodology for efficient monitoring of processes running on hosts in a corporate network. The methodology is based on collecting streams of system calls produced by all or selected processes on the hosts, and sending them over the network to a monitoring server, where machine learning algorithms are use…

2017-07-12abs ↗pdf ↗

This paper compares and evaluates methods for evaluating statistical models using benchmarking data and simulations.

problem Choosing between benchmarking data sets and simulation studies for method comparison studies.
method Borrowing ideas from mixed methods research and Clinical Scenario Evaluation, the paper investigates and develops new approaches to evaluate methods.
result Develops new approaches to evaluate methods by combining the strengths of benchmarking data sets and simulation studies.

A new test evaluates risk estimation accuracy using probability integral transform.

problem Measuring the accuracy of financial market risk estimations.
method Probability Integral Transform (PIT) of ex post realized returns against ex ante probability distributions.
result The new test shows the importance of capturing the dynamic of financial markets.

Proposes a new method to rank risky investments based on Omega measure.

problem Evaluating and ranking risky investment projects.
method Introduces an investment certainty equivalence approach and uses the Omega measure.
result Proposed method ranks projects differently from conventional risk-adjusted discount rate (RADR) approach.

This work compares OmniAnomaly with PCA for MTSAD, finding PCA can match or outperform OmniAnomaly.

problem Comparing deep learning models with classical methods in MTSAD under fair evaluation protocols.
method Systematic comparison of OmniAnomaly and PCA on SMD, using identical thresholding and evaluation procedures.
result PCA can achieve performance comparable to OmniAnomaly and even outperform it under certain conditions.

New method builds complex networks from attribute interactions without normalization.

problem Improving high-level classification algorithms by capturing hidden attribute interactions.
method Proposes a new complex network building methodology based on attribute-attribute interactions, avoiding normalization.
result Demonstrates improved performance in high-level classification techniques.

Bitcoin price prediction models fail to outperform a simple 'today's price' baseline, especially at longer horizons.

problem Lack of robust models that consistently outperform a naive price predictor at various horizons.
method Surveyed peer-reviewed papers, categorized by evaluation methodology, contrasted with social media discourse, and proposed methodological standards.
result No peer-reviewed study has shown robust superiority over the naive baseline across multiple market regimes at short-to-medium horizons.

Graph representations offer powerful and intuitive ways to describe data in a multitude of application domains. Here, we consider stochastic processes generating graphs and propose a methodology for detecting changes in stationarity of such processes. The methodology is general and considers a process generating attrib…

2017-06-21abs ↗pdf ↗

In this paper, we propose a design methodology for one-class classifiers using an ensemble-of-classifiers approach. The objective is to select the best structures created during the training phase using an ensemble of spanning trees. It takes the best classifier, partitioning the area near a pattern into γγ2γ^{γ-2} sub-…

2019-09-09abs ↗pdf ↗

We define a methodology to quantify market activity on a 24 hour basis by defining a scale, the so-called scale of market quakes (SMQ). The SMQ is designed within a framework where we analyse the dynamics of excess price moves from one directional change of price to the next. We use the SMQ to quantify the FX market an…

2009-09-09abs ↗pdf ↗

Study categorizes time series anomaly detection metrics based on evaluation challenges.

problem Challenges in evaluating time series anomaly detection due to diverse application objectives and metric assumptions.
method Problem-oriented framework categorizing metrics into six dimensions based on evaluation challenges.
result Quantifies each metric's discriminative ability and reveals limitations of widely used metrics.

A new causal deepset framework improves off-policy evaluation under complex interference.

problem Handling spatio-temporal interference in off-policy evaluation.
method Permutation invariance assumption and novel algorithms incorporating it.
result Significantly more precise estimations than existing methods.

New method improves consistency of reinforcement learning performance evaluations.

problem Inconsistent performance results in reinforcement learning due to flawed evaluation metrics.
method Proposes a new comprehensive evaluation methodology for reinforcement learning algorithms.
result Demonstrates improved reliability of performance measurements for reinforcement learning algorithms.

Correctly evaluating defenses against adversarial examples has proven to be extremely difficult. Despite the significant amount of recent work attempting to design defenses that withstand adaptive attacks, few have succeeded; most papers that propose defenses are quickly shown to be incorrect. We believe a large contri…

2019-02-18abs ↗pdf ↗

Data-driven method for option pricing using historical asset prices.

problem Tackling the gap between historical asset prices and risk-neutral option pricing.
method Identifying a pricing kernel process, solving utility maximization and functional optimization problems using deep learning.
result Demonstrated the efficiency of the data-driven option pricing methodology.

In the current era of worldwide stock market interdependencies, the global financial village has become increasingly vulnerable to systemic collapse. The recent global financial crisis has highlighted the necessity of understanding and quantifying interdependencies among the world's economies, developing new effective …

2014-08-03abs ↗pdf ↗

The increasing availability of individual-level data has led to numerous applications of individualized (or personalized) treatment rules (ITRs). Policy makers often wish to empirically evaluate ITRs and compare their relative performance before implementing them in a target population. We propose a new evaluation metr…

2019-05-14abs ↗pdf ↗

Paper proposes efficient method for evaluating Bayesian models in imaging.

problem Evaluation of Bayesian models in imaging when ground truth is unavailable.
method Novel combination of Bayesian cross-validation and data fission for unsupervised model selection and misspecification detection.
result Achieved excellent selection and detection accuracy with low computational cost.

Study reveals gaps between simulated and real-world treatment effect evaluation metrics.

problem Evaluation of treatment effect estimation models differs between academic and practical settings.
method Comprehensive empirical study comparing semi-simulated benchmarks and real-world datasets.
result Counterfactual metrics do not reliably predict observable metrics, and rankings from simulated benchmarks do not generalize to real-world data.

The paper evaluates index-based allocation policies using data from randomized control trials.

problem Evaluating index-based allocation policies in resource-scarce scenarios.
method Using data from randomized control trials, the paper introduces an efficient estimator and methods for computing asymptotically correct confidence intervals.
result Valid statistical conclusions can be drawn for index-based allocation policies.

We study the effectiveness of non-uniform randomized feature selection in decision tree classification. We experimentally evaluate two feature selection methodologies, based on information extracted from the provided dataset: (i)(i) \emph{leverage scores-based} and (ii)(ii) \emph{norm-based} feature selection. Experimenta…

2014-03-24abs ↗pdf ↗

Adopting a zonal structure of electricity market requires specification of zones' borders. In this paper we use social welfare as the measure to assess quality of various zonal divisions. The social welfare is calculated by Market Coupling algorithm. The analyzed divisions are found by the usage of extended Locational …

2014-05-05abs ↗pdf ↗

Study evaluates two-sample tests for validating generative models in high dimensions.

problem Validating the performance and efficiency of non-parametric two-sample tests for high-dimensional generative models.
method Proposes and evaluates the sliced Wasserstein distance, mean of Kolmogorov-Smirnov statistics, and novel sliced Kolmogorov-Smirnov statistic.
result One-dimensional-based tests provide comparable sensitivity to other multivariate metrics but with lower computational cost.

New method evaluates AI stock prediction systems based on decision-making processes.

problem Lack of evaluation for AI systems' decision-making processes.
method Scores intermediate decision process using large language models and closed-loop reinforcement learning feedback.
result Composite behavioral score correlates with Sharpe ratio and reduces prediction error.