Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,742 papers · 148 categories

Trend · papers per month

13274053 · Jun 202019922001200920172026
48 results for expert judgment

Framework uses human judgment to distinguish algorithmically indistinguishable cases.

problem Clarifying human-AI collaboration in prediction and decision tasks.
method Integrates human judgment to distinguish algorithmically indistinguishable cases.
result Improves performance of any feasible algorithmic predictor.

Approach for assessing supply chain cyber risks using expert judgment and forecasting.

problem Supply chain managers face challenges in assessing cyber risks affecting business factors.
method Structured expert judgment and forecasting models to assess various attack techniques and impacts.
result Facilitates implementation of risk management activities and decision-making processes.

Graphical models improve actuarial judgment in insurance claims analysis.

problem Improving actuarial judgment in insurance claims analysis.
method Using graphical models to represent complex inter-dependencies and incorporate qualitative knowledge.
result Graphical models can be used to express and analyze non-life insurance claims data.

Paper proposes government indemnification for AI risks to solve judgment-proof problem.

problem Uninsurable risks from AI, especially existential risks, create a judgment-proof problem.
method A government-provided, mandatory indemnification program using risk-priced fees and Bayesian Truth Serum.
result The approach better leverages private information and signals risk mitigation efforts.

Enhances BO with expert preferences about abstract properties.

problem Lack of expert knowledge in BO for black-box experimental design.
method Human-AI collaboration to incorporate expert preferences into surrogate modeling.
result Superior performance compared to baselines in synthetic and real-world datasets.

A method for eliciting expert beliefs using preferential questions and normalizing flows.

problem Eliciting high-dimensional probability distributions from noisy judgments.
method Normalizing flows based on preferential questions with a novel functional prior.
result The method allows for the inference of arbitrarily flexible densities from preferential judgments.

This paper considers the challenge of evaluating a set of classifiers, as done in shared task evaluations like the KDD Cup or NIST TREC, without expert labels. While expert labels provide the traditional cornerstone for evaluating statistical learners, limited or expensive access to experts represents a practical bottl…

2012-12-05abs ↗pdf ↗

Potential violent criminals will often need to go through a sequence of preparatory steps before they can execute their plans. During this escalation process police have the opportunity to evaluate the threat posed by such people through what they know, observe and learn from intelligence reports about their activities…

2019-07-03abs ↗pdf ↗

Develops Austen plots for assessing bias from unobserved confounding in observational studies.

problem Bias in causal estimates due to unobserved confounding.
method Formalizes confounding strength, uses Austen plots to visualize and quantify bias.
result Allows domain experts to assess the plausibility of strong confounders.

Proposes a new bankruptcy prediction model using Bayesian framework with expert knowledge.

problem Bankruptcy prediction using accounting ratios and statistical models.
method Generative modeling with Expert Bayesian framework.
result Model is either comparable or superior to existing methods with lower False Positive Rate.

Language-based methods improve human similarity approximations without requiring many human judgments.

problem Approximating human similarity judgments using pre-trained deep neural networks (DNNs) is challenging and expensive.
method Developed language-based methods to approximate human similarity judgments, validated with adaptive tag collection pipeline.
result Language-based methods significantly improve performance over DNN-based methods with fewer human judgments.

Large language models predict human sensory judgments across multiple modalities.

problem Determining the extent of perceptual information in language.
method State-of-the-art large language models were used to predict sensory judgments across six psychophysical datasets.
result Large language models can predict human sensory judgments across multiple modalities with significant correlation to human data.

To study how mental object representations are related to behavior, we estimated sparse, non-negative representations of objects using human behavioral judgments on images representative of 1,854 object categories. These representations predicted a latent similarity structure between objects, which captured most of the…

2019-01-09abs ↗pdf ↗

Paper improves PBO using Skew Gaussian Processes for better optimization.

problem Optimizing with preference judgments, especially in A/B tests and recommender systems.
method Uses Skew Gaussian Processes to model preference function and exact posterior inference.
result Exact SkewGP posterior leads to better optimization results than Laplace approximation.

Topic models are typically evaluated with respect to the global topic distributions that they generate, using metrics such as coherence, but without regard to local (token-level) topic assignments. Token-level assignments are important for downstream tasks such as classification. Even recent models, which aim to improv…

2019-05-18abs ↗pdf ↗

It is inconceivable how chaotic the world would look to humans, faced with innumerable decisions a day to be made under uncertainty, had they been lacking the capacity to distinguish the relevant from the irrelevant---a capacity which computationally amounts to handling probabilistic independence relations. The highly …

2018-01-30abs ↗pdf ↗

Machine learning predicts ECHR judgments on human rights violations.

problem Predicting the outcome of ECHR judgments on human rights violations.
method Auto-sklearn for model selection, N-grams, word embeddings, doc2vec, echr2vec for feature extraction, cross-validation for accuracy assessment.
result Features from echr2vec embedding provided the highest cross-validation accuracy for 5 Articles, overall test accuracy was 68.83%.

Tree-Query uses LLMs to discover causal relationships in a transparent, interpretable manner.

problem Error propagation in classical causal discovery methods and opaque, confidence-free behavior of recent LLM-based causal oracles.
method Tree-Query is a tree-structured, multi-expert LLM framework that reduces causal discovery to queries about backdoor paths and dependencies.
result Tree-Query provides interpretable judgments with robustness-aware confidence scores and improves structural metrics over LLM baselines.

The goal of ordinal embedding is to represent items as points in a low-dimensional Euclidean space given a set of constraints in the form of distance comparisons like "item ii is closer to item jj than item kk". Ordinal constraints like this often come from human judgments. To account for errors and variation in jud…

2016-06-22abs ↗pdf ↗

Comparative study of neural networks for short-term FOREX forecasting.

problem Simulating expert judgment in foreign exchange market forecasting.
method Implemented and compared LSTM and ANN architectures for short-term FOREX forecasting.
result ANN custom architecture outperforms LSTM in prediction quality and resource efficiency.

Accurate prediction of suicide risk in mental health patients remains an open problem. Existing methods including clinician judgments have acceptable sensitivity, but yield many false positives. Exploiting administrative data has a great potential, but the data has high dimensionality and redundancies in the recording …

2016-05-03abs ↗pdf ↗

This paper proposes a simple technical approach for the analytical derivation of Point-in-Time PD (probability of default) forecasts, with minimal data requirements. The inputs required are the current and future Through-the-Cycle PDs of the obligors, their last known default rates, and a measurement of the systematic …

2015-07-20abs ↗pdf ↗

In this paper, we address the problem of measuring and analysing sensation, the subjective magnitude of one's experience. We do this in the context of the method of triads: the sensation of the stimulus is evaluated via relative judgments of the form: "Is stimulus S_i more similar to stimulus S_j or to stimulus S_k?". …

2019-08-21abs ↗pdf ↗

CJE calibrates cheap LLM judges against an oracle, achieving high accuracy at a fraction of the cost.

problem Inexpensive LLM judges can produce biased rankings, leading to unreliable outcomes.
method CJE uses a small oracle to calibrate cheap scores, then evaluates at scale with valid uncertainty.
result CJE achieves 99% pairwise ranking accuracy at 14x lower cost compared to a 16x oracle/judge cost ratio.

LLM evaluation suffers from systematic biases and lacks reliable positive judgments.

problem LLM evaluation suffers from systematic biases and lacks reliable positive judgments.
method Formulate LLM evaluation as a positive-unlabelled learning problem and propose a geometric auditing framework based on Partial Optimal Transport.
result Improved alignment with human preferences, increased robustness to presentation biases, and interpretable confidence estimates.

E-Commerce (E-Com) search is an emerging important new application of information retrieval. Learning to Rank (LETOR) is a general effective strategy for optimizing search engines, and is thus also a key technology for E-Com search. While the use of LETOR for web search has been well studied, its use for E-Com search h…

2019-03-01abs ↗pdf ↗

Proposes RDASS for better Korean text summarization evaluation.

problem ROUGE scores fail to capture semantic meaning in Korean text summarization.
method Introduces RDASS metrics and a method to improve their correlation with human judgment.
result RDASS metrics correlate better with human judgment than ROUGE scores.

AI assistants often give convincing but incorrect responses to match user beliefs.

problem Sycophancy in AI assistants that use human feedback.
method Examined five AI assistants across four tasks, analyzed human preference data, and compared model outputs against preference models.
result Sycophancy is a general behavior of AI assistants, driven in part by human preference judgments.

Multi-expert L2D underfits more severely, requiring new methods.

problem Underfitting in multi-expert L2D settings.
method PiCCE (Pick the Confident and Correct Expert), a surrogate-based method.
result PiCCE effectively reduces multi-expert L2D to a single-expert-like problem, resolving underfitting.

TENP prunes experts and neurons in Mixture-of-Experts models for efficient deployment.

problem Efficient deployment of large language models constrained by static parameter footprint.
method Structured Trapezoidal ExpertNeuron Pruning (TENP) identifies and retains important experts and neurons.
result DeepSeek model achieves 10% better performance on code generation tasks with 40% expert sparsity.