A property, or statistical functional, is said to be elicitable if it minimizes expected loss for some loss function. The study of which properties are elicitable sheds light on the capabilities and limitations of point estimation and empirical risk minimization. While recent work asks which properties are elicitable, …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “property elicitation”
Study generalizes property elicitation to imprecise probabilities.
problem Minimizing risk over imprecise probability distributions.
method Maximin risk minimization over a set of imprecise probabilities.
result Conditions for elicitability of IP-properties.
New algorithm achieves faster multicalibration in online settings.
problem Achieving fair predictions in online settings with limited interaction.
method Generalized multicalibration to arbitrary bounded hypothesis classes and introduced swap multicalibration. Proposed an oracle-efficient algorithm achieving faster multicalibration error.
result Oracle-efficient algorithm achieves -swap multicalibration error with high probability, improving on previous bounds.
Constructs new elicitable risk measures with multiplicative scoring functions.
problem Defining new elicitable risk measures with specific properties.
method Constructs new elicitable risk measures using a multiplicative scoring function.
result Encompasses and allows construction of novel elicitable risk measures.
Study uses property elicitation to understand how fairness regularizers affect optimal decisions.
problem Understanding how fairness regularizers change the optimal decision in predictive algorithms.
method Property elicitation to analyze the relationship between loss, regularization, and optimal decision.
result Necessary and sufficient condition for when a property changes with the addition of a regularizer.
Robustifies elicitable functionals to handle small distribution misspecifications.
problem Determining uniquely optimal forecasts under distributional misspecification.
method Integrates statistical robustness into elicitable functionals using Kullback-Leibler divergence.
result Robust elicitable functionals admit unique solutions at the boundary of uncertainty regions.
Given a binary prediction problem, which performance metric should the classifier optimize? We address this question by formalizing the problem of Metric Elicitation. The goal of metric elicitation is to discover the performance metric of a practitioner, which reflects her innate rewards (costs) for correct (incorrect)…
The paper analyzes elicitability of return risk measures and their scoring functions.
problem Elicitability of return risk measures and their scoring functions.
method Dual representation results for convex and geometrically convex return risk measures, axiomatic characterizations of Orlicz premia, and construction of strictly consistent scoring functions.
result Orlicz premia are the only elicitable return risk measures under different sets of conditions.
This paper explores how to choose scoring rules for estimating properties with parametric assumptions.
problem Indirect elicitation of properties with parametric assumptions.
method Developed a framework for choosing proper scoring rules for indirect elicitation, considering constraints and optimal solutions.
result The optimal estimation of the target property changes monotonically with the increase of each weight, and often setting some weights as zero yields the best configuration.
Recently, financial industry and regulators have enhanced the debate on the good properties of a risk measure. A fundamental issue is the evaluation of the quality of a risk estimation. On the one hand, a backtesting procedure is desirable for assessing the accuracy of such an estimation and this can be naturally achie…
This paper develops a new method for eliciting more flexible metrics, improving fairness and applicability.
problem Limited flexibility in existing metric elicitation strategies for reflecting user preferences.
method Develops a strategy for eliciting quadratic metrics based on predictive rates, requiring only relative preference feedback.
result Achieves near-optimal query complexity and broadens the use cases for metric elicitation.
Paper proposes incentives for federated learning to ensure truthful contributions.
problem Ensuring truthful contributions from decentralized users in federated learning.
method Introduces a scoring rule based framework to incentivize truthful reporting of local hypotheses at a Bayesian Nash Equilibrium.
result Proposed solution verified using MNIST and CIFAR-10 datasets, showing decreasing scores for low-quality hypotheses.
Proposes a method to select fair performance metrics through metric elicitation.
problem Choosing fair performance metrics in multiclass classification with multiple sensitive groups.
method Metric elicitation strategy that requires only relative preference feedback and is robust to noise.
result Elicits group-fair performance metrics for multiclass classification problems.
We discuss equivalent axiomatic characterizations of distortion risk measures, and give a novel and concise proof of the characterization of elicitable distortion risk measures. Elicitability has recently been discussed as a desirable criterion for risk measures, motivated by statistical considerations of forecasting. …
A statistical functional, such as the mean or the median, is called elicitable if there is a scoring function or loss function such that the correct forecast of the functional is the unique minimizer of the expected score. Such scoring functions are called strictly consistent for the functional. The elicitability of a …
Study creates web interface to elicit user-preferred metrics.
problem Eliciting classification metrics that align with user preferences.
method Developed a web-based interface and conducted a user study.
result Users preferred metrics that align with their task and context.
Sample Elicitationcs.LG
It is important to collect credible training samples for building data-intensive learning systems (e.g., a deep learning system). Asking people to report complex distribution , though theoretically viable, is challenging in practice. This is primarily due to the cognitive loads required for human agents t…
In the present contribution we characterize law determined convex risk measures that have convex level sets at the level of distributions. By relaxing the assumptions in Weber (2006), we show that these risk measures can be identified with a class of generalized shortfall risk measures. As a direct consequence, we are …
A method for eliciting expert beliefs using preferential questions and normalizing flows.
problem Eliciting high-dimensional probability distributions from noisy judgments.
method Normalizing flows based on preferential questions with a novel functional prior.
result The method allows for the inference of arbitrarily flexible densities from preferential judgments.
Proposes method for eliciting non-parametric joint priors using normalizing flows.
problem Learning complex non-parametric joint priors for model parameters.
method Expert elicitation combined with normalizing flows for generative modeling.
result Framework supports elicitation of both parametric and non-parametric priors.
This thesis formalizes metric selection for machine learning applications.
problem Selecting the right performance metric for machine learning models.
method Metric Elicitation framework using pairwise preference feedback.
result Novel strategies for eliciting various classification metrics robust to noise.
Coherence and elicitabilityq-fin.RM
The risk of a financial position is usually summarized by a risk measure. As this risk measure has to be estimated from historical data, it is important to be able to verify and compare competing estimation procedures. In statistical decision theory, risk measures for which such verification and comparison is possible,…
Method combines deep learning and elicitability for solving complex stochastic equations.
problem Solving McKean-Vlasov FBSDEs with common noise.
method Combines Picard iterations, elicitability, and deep learning.
result Validated on systemic-risk model and extended to quantile-mediated interactions.
Expected Shortfall (ES) has been widely accepted as a risk measure that is conceptually superior to Value-at-Risk (VaR). At the same time, however, it has been criticised for issues relating to backtesting. In particular, ES has been found not to be elicitable which means that backtesting for ES is less straightforward…
Develops a simulation-based method to translate expert knowledge into prior distributions for Bayesian models.
problem Effective incorporation of expert knowledge into prior distributions for diverse model structures.
method Simulation-based stochastic gradient descent to learn hyperparameters of parametric priors from expert knowledge.
result Method is adaptable to various elicitation techniques and independent of model structure.
Paper establishes identifiability and elicitability of tail risk measures.
problem Identifying and measuring tail risk measures accurately.
method Establishes identifiability and elicitability of tail risk measures using generators and quantiles.
result Joint identifiability and elicitability of tail risk measures and quantiles.
In this note, we comment on the relevance of elicitability for backtesting risk measure estimates. In particular, we propose the use of Diebold-Mariano tests, and show how they can be implemented for Expected Shortfall (ES), based on the recent result of Fissler and Ziegel (2015) that ES is jointly elicitable with Valu…
We consider settings in which the right notion of fairness is not captured by simple mathematical definitions (such as equality of error rates across groups), but might be more complex and nuanced and thus require elicitation from individual or collective stakeholders. We introduce a framework in which pairs of individ…
This paper attempts to provide a decision-theoretic foundation for the measurement of economic tail risk, which is not only closely related to utility theory but also relevant to statistical model uncertainty. The main result is that the only risk measures that satisfy a set of economic axioms for the Choquet expected …
Framework uses IRL and RL to elicit and optimize risk preferences robustly to noise.
problem Eliciting and optimizing risk preferences in noisy environments.
method Adaptive Bayesian IRL for elicitation, model-free RL for optimization, using quantile networks.
result Framework achieves convergence rate of .
New method allows backtesting of systemic risk forecasts.
problem Systemic risk measures are not elitable and identifiable, making backtesting impossible.
method Introduces multi-objective elicitability and Diebold--Mariano type tests.
result Proposes a traffic-light approach for backtesting.
We propose a cost-effective framework for preference elicitation and aggregation under the Plackett-Luce model with features. Given a budget, our framework iteratively computes the most cost-effective elicitation questions in order to help the agents make a better group decision. We illustrate the viability of the fram…
Generalized Orlicz premiaq-fin.RM
Introduces generalized Orlicz premia for broader applicability.
problem Developing a flexible framework for insurance premium calculation.
method Introduces a generalized Orlicz premium definition using non-convex loss functions.
result Generalized Orlicz premia encompass various specific cases and maintain key properties.
Formulates a Dueling Bandits problem for eliciting Kemeny rankings.
problem Eliciting preferences to find a Kemeny ranking.
method Formulates the problem as a Dueling Bandits problem, considering sampling with and without replacement.
result Approximation bounds and algorithms for finding PAC solutions with sample complexity.
A framework for eliciting utility functions from investor preferences.
problem Hard elicitation of specific utility functions in portfolio selection.
method Preference-fitting method using probability-wealth pairs and PHARA approximation.
result Fitted utility function converges to the optimal one as more data is used.
Learning predictive models from small high-dimensional data sets is a key problem in high-dimensional statistics. Expert knowledge elicitation can help, and a strong line of work focuses on directly eliciting informative prior distributions for parameters. This either requires considerable statistical expertise or is l…
Inverse classification uses an induced classifier as a queryable oracle to guide test instances towards a preferred posterior class label. The result produced from the process is a set of instance-specific feature perturbations, or recommendations, that optimally improve the probability of the class label. In this work…
Requirements elicitation can be very challenging in projects that require deep domain knowledge about the system at hand. As analysts have the full control over the elicitation process, their lack of knowledge about the system under study inhibits them from asking related questions and reduces the accuracy of requireme…
Interactive Elicitation of Knowledge on Feature Relevance Improves Predictions in Small Data Setscs.AI
Providing accurate predictions is challenging for machine learning algorithms when the number of features is larger than the number of samples in the data. Prior knowledge can improve machine learning models by indicating relevant variables and parameter values. Yet, this prior knowledge is often tacit and only availab…
Elicitation without Backpropagation: Steering Model Behavior by Optimizing the Latent Posteriorcs.LG
PPT optimizes transformer behavior by steering its latent posterior using prior samples.
problem Eliciting desired behavior from transformers without backpropagation.
method Posterior Prefix Tuning (PPT) uses predictive Monte Carlo (PMC) samples and importance sampling to optimize the latent posterior.
result PPT optimizes transformer behavior without backpropagation, achieving high utility across different utility functions.
In this paper we propose an approach to preference elicitation that is suitable to large configuration spaces beyond the reach of existing state-of-the-art approaches. Our setwise max-margin method can be viewed as a generalization of max-margin learning to sets, and can produce a set of "diverse" items that can be use…
Crowdsourced wisdom improves causal learning.
problem Improving causal learning through collective intelligence.
method Crowdsourcing, expert knowledge elicitation, aggregation techniques, and LLMs.
result Collective contributions enhance global causal structure.
CAESar improves risk forecasting by combining VaR and ES estimates.
problem Lack of tail risk measures in financial risk management.
method Conditional Autoregressive Expected Shortfall model, combining VaR and ES estimates.
result CAESar outperforms existing methods in risk forecasting.
We characterize when the level sets of a continuous quasi-monotone functional defined on a suitable convex subset of a normed space can be uniquely represented by a family of bounded continuous functionals. Furthermore, we investigate how regularly these functionals depend on the parameterizing level. Finally, we show …
Generative Adversarial Regression (GAR) learns risk scenarios robustly across policies.
problem Learning risk scenarios for conditional risk objectives.
method Generative adversarial framework for risk matching.
result GAR produces more stable and risk-preserving scenarios than baselines.
Platform uses queries to elicit investor preferences for portfolio trades, improving allocation efficiency.
problem Hidden-information problem in institutional crossing markets where investors value trades as portfolios but liquidity discovery is organized by individual securities.
method Modeling portfolio crossing as preference elicitation, using price-directed demand queries and value queries to verify selected packages.
result Hybrid procedure using demand and value queries recovers 88-95% of full-information welfare with a limited query budget.
Requirements elicitation requires extensive knowledge and deep understanding of the problem domain where the final system will be situated. However, in many software development projects, analysts are required to elicit the requirements from an unfamiliar domain, which often causes communication barriers between analys…
ContextBench benchmarks methods for generating linguistically fluent inputs that activate specific latent features in language models.
problem Identifying inputs that trigger specific behaviours or latent features in language models.
method Context modification and benchmarking methods like Evolutionary Prompt Optimisation (EPO) with LLM-assistance and diffusion model inpainting.
result Enhanced methods achieve state-of-the-art performance in balancing elicitation effectiveness and fluency.