Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

105211316421 · Jun 202019922001200920172026
48 results for subjective evaluations

Optimizes crowdsourced preference-based subjective evaluation with online learning.

problem Large-scale evaluation of generative media using crowdsourcing due to combinatorial explosion.
method Automatic optimization of pair combination selections and evaluation volumes with online learning.
result Optimizes evaluation by reducing pair combinations and allocating optimal evaluation volumes.

Pairwise ranking aligns subjective clinical evaluations with objective indicators.

problem Aligning subjective clinical evaluations with objective indicators for improved diagnosis.
method Pairwise ranking methods to align subjective evaluations with objective indicators.
result The resulting score improves classification accuracy and provides a nuanced severity assessment.

Bayesian framework evaluates predictors of subjective visual tasks.

problem Evaluating uncertainty in machine learning predictors for tasks with subjective annotations.
method Bayesian framework to estimate epistemic uncertainty from human labels.
result Framework successfully applied to four image classification tasks.

Paper introduces ρρ-Perfect to estimate model-human correlation in subjective datasets.

problem Inherent noise in subjective ratings limits model-human correlation quantification.
method Defines ρρ-Perfect as highest achievable correlation between perfect predictor and human ratings. Estimates based on heteroscedastic noise scenarios.
result Demonstrates ρρ-Perfect can distinguish model limitations from data quality issues.

K-fold Cross Validation is commonly used to evaluate classifiers and tune their hyperparameters. However, it assumes that data points are Independent and Identically Distributed (i.i.d.) so that samples used in the training and test sets can be selected randomly and uniformly. In Human Activity Recognition datasets, we…

2019-04-04abs ↗pdf ↗

How does one objectively measure the performance of an individual offensive lineman in the NFL? The existing literature proposes various measures that rely on subjective assessments of game film, but has yet to develop an objective methodology to evaluate performance. Using a variety of statistics related to an offensi…

2016-03-24abs ↗pdf ↗

Paper uses subjective logic to estimate uncertainty in multi-armed bandit problems.

problem Estimating uncertainty in multi-armed bandit problems.
method Formalism of subjective logic applied to multi-armed bandits, proposing new algorithms.
result Subjective logic quantities enable useful assessment of uncertainty.

A novel validation method improves feature importance analysis in subject-specific ML models.

problem Limited effectiveness of general ML models trained on large datasets for individual patient outcomes.
method A novel validation approach using a general ML model with repeated trials and random seed variation.
result Consistent identification of key features at the subject level and improved group-level feature importance analysis.

Proposes a framework for extracting consistent physiological features across users.

problem Variability of biosignals across different users and tasks.
method Adversarial feature extractor for disentangled universal representations.
result Up to 8.8% improvement in average accuracy of classification.

This paper describes audEERING's submissions as well as additional evaluations for the One-Minute-Gradual (OMG) emotion recognition challenge. We provide the results for audio and video processing on subject (in)dependent evaluations. On the provided Development set, we achieved 0.343 Concordance Correlation Coefficien…

2018-05-03abs ↗pdf ↗

The article reviews scoring rules for estimating and evaluating forecasts.

problem Evaluating probabilistic forecasts and estimating probability distributions.
method Mathematical foundations and characterization of scoring rules.
result Important families of scoring rules and their applications in statistics and machine learning.

In this paper, we consider the problem of black box continuous submodular maximization where we only have access to the function values and no information about the derivatives is provided. For a monotone and continuous DR-submodular function, and subject to a bounded convex body constraint, we propose Black-box Contin…

2019-01-28abs ↗pdf ↗

A significant challenge for a supervised learning approach to inertial human activity recognition is the heterogeneity of data between individual users, resulting in very poor performance of impersonal algorithms for some subjects. We present an approach to personalized activity recognition based on deep embeddings der…

2020-01-15abs ↗pdf ↗

AI benchmarks evaluate football team performance using generative models.

problem Evaluating human performance in complex interactive tasks is error-prone and unreliable.
method Trained Conditional VRNN Model on player and ball tracking data to imitate and predict team interactions.
result Trained model as a useful benchmark for evaluating team performance in football.

We present a practical health-theme machine learning (ML) application concerning `AI for social good' domain for `Producing Good Outcomes' track. In particular, the solution is concerning the problem of a potential elderly adult dementia onset prediction in aging societies. The paper discusses our attempt and encouragi…

2019-11-25abs ↗pdf ↗

Overlap between treatment groups is required for non-parametric estimation of causal effects. If a subgroup of subjects always receives the same intervention, we cannot estimate the effect of intervention changes on that subgroup without further assumptions. When overlap does not hold globally, characterizing local reg…

2019-07-09abs ↗pdf ↗

Fast algorithms developed for adaptive and fully adaptive submodular maximization problems.

problem Maximizing submodular functions subject to constraints in linear time.
method Developed linear-time algorithms for two submodular maximization problems: adaptive and fully adaptive.
result Achieved (11/eε)(1-1/e-ε) approximation ratio for adaptive submodular maximization and $ rac{1-1/e-ε}{4-2/e-2ε}$ for fully adaptive submodular maximization.

Recent developments in differentially private (DP) machine learning and DP Bayesian learning have enabled learning under strong privacy guarantees for the training data subjects. In this paper, we further extend the applicability of DP Bayesian learning by presenting the first general DP Markov chain Monte Carlo (MCMC)…

2019-01-29abs ↗pdf ↗

Competency questions help experts select best clustering for energy data.

problem Ad hoc and subjective selection of clustering structures by domain experts.
method Formalize expert knowledge and requirements with competency questions.
result Competency questions improve reproducibility and evaluation of clustering applications.

This paper reviews PU learning evaluation methods and provides practical recommendations.

problem Evaluating PU learning methods when only positive and unlabelled data are available.
method Critical review of 51 articles proposing PU classifiers and alternative predictive accuracy measures.
result Practical recommendations for improving PU learning evaluation.

Unified model combines neural networks and dictionary learning for clinical predictions from brain data.

problem Predicting clinical severity from brain imaging data.
method Combines neural networks with dictionary learning to model patient-specific and shared features.
result Unified model outperforms state-of-the-art methods in predicting clinical severity.

Recent work on Bayesian optimization has shown its effectiveness in global optimization of difficult black-box objective functions. Many real-world optimization problems of interest also have constraints which are unknown a priori. In this paper, we study Bayesian optimization for constrained problems in the general ca…

2014-03-22abs ↗pdf ↗

The need for diversification of recommendation lists manifests in a number of recommender systems use cases. However, an increase in diversity may undermine the utility of the recommendations, as relevant items in the list may be replaced by more diverse ones. In this work we propose a novel method for maximizing the u…

2014-11-13abs ↗pdf ↗

A new method for efficient inference in sequential latent-variable models.

problem Computational challenges in integrating subject-specific random effects.
method Anchored variational inference framework to approximate posterior distributions.
result The method achieves accurate estimation with significant computational gains.

This paper provides lower bounds on the convergence rate of Derivative Free Optimization (DFO) with noisy function evaluations, exposing a fundamental and unavoidable gap between the performance of algorithms with access to gradients and those with access to only function evaluations. However, there are situations in w…

2012-09-11abs ↗pdf ↗

In this work we are concerned with valuing optionalities associated to invest or to delay investment in a project when the available information provided to the manager comes from simulated data of cash flows under historical (or subjective) measure in a possibly incomplete market. Our approach is suitable also to inco…

2015-09-11abs ↗pdf ↗