The paper shows how expert knowledge can improve treatment effect estimation.
problem Lack of leveraging expert knowledge in treatment effect estimation.
method Formally defining two types of expertise (predictive and prognostic) and demonstrating their influence on treatment effect estimation methods.
result Expertise type significantly influences treatment effect estimation methods, and can be predicted from a dataset.
A new method calibrates forecasts without sacrificing expertise.
problem Forecasters' calibration scores can be manipulated to appear expert.
method Deterministic and stochastic online procedures to calibrate forecasts.
result Calibration can be achieved without losing expertise.
FinHEAR combines LLMs with human expertise for better financial decision-making.
problem Challenges in financial decision-making for language models.
method Multi-agent framework with specialized LLMs for historical analysis, event interpretation, and expert retrieval.
result FinHEAR outperforms baselines in financial tasks with higher accuracy and risk-adjusted returns.
Study shows human advisors use context to improve student outcomes in algorithm-assisted advising.
problem How human advisors use context to guide interventions in algorithm-assisted advising.
method Mixed-methods approach combining quantitative and qualitative data from a randomized controlled trial.
result 2 out of 3 interventions by advisors were plausibly 'expertly targeted' to students using non-algorithmic context.
This paper presents a generic Bayesian framework that enables any deep learning model to actively learn from targeted crowds. Our framework inherits from recent advances in Bayesian deep learning, and extends existing work by considering the targeted crowdsourcing approach, where multiple annotators with unknown expert…
Novel CNN-based gaze scanpath comparison distinguishes experts from novices in dental radiograph interpretation.
problem Distinguishing expertise in dental radiograph interpretation based on gaze behavior.
method Convolutional neural networks (CNN) process scene information at the fixation level, using image patches as input to compare gaze scanpaths.
result 93% accuracy in distinguishing experts from novices using image patch features.
PTBCC improves accuracy in multi-class annotation aggregation by learning from prototype confusion matrices.
problem Inaccurate and insufficient confusion matrices for annotators in multi-class classification tasks.
method PTBCC (ProtoType learning-driven Bayesian Classifier Combination) uses prototype confusion matrices to capture annotator expertise.
result PTBCC achieves up to 15% accuracy improvement and 3% higher average accuracy compared to existing methods.
We present an interactive version of an evidence-driven state-merging (EDSM) algorithm for learning variants of finite state automata. Learning these automata often amounts to recovering or reverse engineering the model generating the data despite noisy, incomplete, or imperfectly sampled data sources rather than optim…
UCFE benchmarks LLMs in financial tasks with human feedback.
problem Evaluating LLMs' financial task performance and user satisfaction.
method Hybrid approach combining human expert evaluations and dynamic interactions.
result Significant alignment between benchmark scores and human preferences (Pearson correlation coefficient of 0.78).
Fund2Persona creates personalized financial advisor personas from fund data, improving investment advice.
problem Lack of consistent advisor expertise and difficulty in encoding it in LLM systems.
method Grounds financial advisor personas in fund disclosures, market context, and manager commentary through an agentic actor--scorer--patcher loop.
result Personas better recover portfolio decisions and manager interpretation than generic baselines.
A popular approach for large scale data annotation tasks is crowdsourcing, wherein each data point is labeled by multiple noisy annotators. We consider the problem of inferring ground truth from noisy ordinal labels obtained from multiple annotators of varying and unknown expertise levels. Annotation models for ordinal…
Develops a test to assess if human experts add value to predictions.
problem Detecting if human expertise adds value to predictions.
method Statistical framework and hypothesis test to assess independence of expert predictions from outcomes.
result Physicians' decisions for AGIB patients incorporate information not available to a screening tool.
Many efficient algorithms with strong theoretical guarantees have been proposed for the contextual multi-armed bandit problem. However, applying these algorithms in practice can be difficult because they require domain expertise to build appropriate features and to tune their parameters. We propose a new method for the…
DPBD simplifies labeling functions through interactive demonstrations.
problem Difficulty in writing labeling functions for large-scale labeled training data.
method Data Programming by Demonstration (DPBD) framework using interactive demonstrations.
result Ruler system generates labeling rules more easily and with higher user satisfaction.
Advocates for user-friendly RL problem descriptions to improve usability and generalization.
problem Usability and generalization challenges in RL for non-engineers.
method Development of user-friendly description languages for RL problems.
result Improved ability of RL algorithms to generalize to new problems.
Paper proposes learnable topological features for efficient phylogenetic inference.
problem Finding appropriate topological structures for phylogenetic inference tasks requires significant design effort and domain expertise.
method Combines raw node features with graph neural networks to automatically adapt to different tasks.
result Demonstrates effectiveness and efficiency on simulated and real data phylogenetic inference tasks.
Algorithm improves learning by integrating diverse agents' behaviors.
problem Lack of social learning in reinforcement learning algorithms.
method Free energy approach for social bandit learning.
result Algorithm converges to optimal policy and enhances learning.
RL algorithms with medical integration improve personalized treatment recommendations.
problem Developing effective personalized treatment strategies for chronic diseases.
method Integrating medical knowledge into RL algorithms for DTR.
result Enhanced treatment recommendations with increased confidence.
Study identifies Bitcoin arbitrageurs and their trading strategies.
problem Detecting and understanding Bitcoin arbitrageurs on Mt. Gox.
method Analyzing historical trade data from Mt. Gox (2011-2014) to identify and categorize arbitrageurs.
result Expert arbitrageurs have a positive profit margin, while novice users do not.
A new method combines experts' opinions to train regression models with noisy labels.
problem Training regression models with noisy labels from multiple experts.
method Estimate each labeler's expertise and combine opinions using learned weights.
result Empirically outperforms existing techniques on simulated and real data.
The goal of imitation learning (IL) is to learn a good policy from high-quality demonstrations. However, the quality of demonstrations in reality can be diverse, since it is easier and cheaper to collect demonstrations from a mix of experts and amateurs. IL in such situations can be challenging, especially when the lev…
New method learns robot skills from data, matching or outperforming existing methods.
problem Learning robot skills from fixed datasets.
method Offline Reinforcement Learning via Supervised Learning using implicit models.
result Implicit models can match or outperform explicit models in acquiring robotic skills.
Machine learning on graph structured data has attracted much research interest due to its ubiquity in real world data. However, how to efficiently represent graph data in a general way is still an open problem. Traditional methods use handcraft graph features in a tabular form but suffer from the defects of domain expe…
We propose a probabilistic model to aggregate the answers of respondents answering multiple-choice questions. The model does not assume that everyone has access to the same information, and so does not assume that the consensus answer is correct. Instead, it infers the most probable world state, even if only a minority…
We present a learning-based system for rapid mass-scale material synthesis that is useful for novice and expert users alike. The user preferences are learned via Gaussian Process Regression and can be easily sampled for new recommendations. Typically, each recommendation takes 40-60 seconds to render with global illumi…
Data Science is currently a popular field of science attracting expertise from very diverse backgrounds. Current learning practices need to acknowledge this and adapt to it. This paper summarises some experiences relating to such learning approaches from teaching a postgraduate Data Science module, and draws some learn…
Neural Architecture Search (NAS) represents a class of methods to generate the optimal neural network architecture and typically iterate over candidate architectures till convergence over some particular metric like validation loss. They are constrained by the available computation resources, especially in enterprise e…
Learning algorithms normally assume that there is at most one annotation or label per data point. However, in some scenarios, such as medical diagnosis and on-line collaboration,multiple annotations may be available. In either case, obtaining labels for data points can be expensive and time-consuming (in some circumsta…
Paper tackles unobserved confounding in human-AI collaborations.
problem Unobserved confounding undermines human-AI collaboration effectiveness.
method Combines sensitivity analysis from causal inference with AI-driven statistical modeling.
result Enhances robustness and reliability of collaborative outcomes.
Fund2Persona creates personalized financial advisor personas from fund data, improving investment advice and manager interpretation.
problem Lack of consistent and specific financial advisor expertise in personalized investment advice.
method Grounds financial advisor personas in fund disclosures, holdings transitions, market context, and manager commentary through an agentic actor--scorer--patcher loop.
result Personas better recover portfolio decisions and grounded manager interpretation than generic baselines.
Media seems to have become more partisan, often providing a biased coverage of news catering to the interest of specific groups. It is therefore essential to identify credible information content that provides an objective narrative of an event. News communities such as digg, reddit, or newstrust offer recommendations,…
Enhances model compression with multi-teacher knowledge distillation.
problem Uncertainty evaluation and diverse teacher expertise in model deployment.
method Bayesian inference and teacher-informed prior with entropy-based weighting.
result Improved predictive accuracy and robust uncertainty quantification.
Deep neural network architectures have traditionally been designed and explored with human expertise in a long-lasting trial-and-error process. This process requires huge amount of time, expertise, and resources. To address this tedious problem, we propose a novel algorithm to optimally find hyperparameters of a deep n…
Online reviews provided by consumers are a valuable asset for e-Commerce platforms, influencing potential consumers in making purchasing decisions. However, these reviews are of varying quality, with the useful ones buried deep within a heap of non-informative reviews. In this work, we attempt to automatically identify…
Survey on two non-Kähler geometry conjectures.
problem Constant holomorphic sectional curvature and Fino-Vezzoni conjectures in non-Kähler geometry.
method Survey and discussion of historical and recent developments.
result Discussion of conjectures without new results.
mNARX+ creates accurate surrogate models for complex systems without requiring domain expertise.
problem Creating accurate surrogate models for complex dynamical systems without extensive domain knowledge.
method Data-driven, recursive algorithm that automatically selects temporal features and their causal ordering.
result Automatically identifies critical auxiliary quantities and their order for accurate modeling.
New framework assesses LLMs' expertise using nonparametric ranking and confidence diagrams.
problem Evaluating and ranking large language models (LLMs) for alignment and performance.
method Nonparametric contextual ranking, confidence diagram, Gaussian multiplier bootstrap.
result Validated confidence diagram for assessing LLMs' domain-specific expertise.
The main objective of exams consists in performing an assessment of students' expertise on a specific subject. Such expertise, also referred to as skill or knowledge level, can then be leveraged in different ways (e.g., to assign a grade to the students, to understand whether a student might need some support, etc.). S…
Bayesian model transfers knowledge across different engineering fleets.
problem Data sparsity in predictive models for engineering infrastructure.
method Hierarchical Bayesian approach with multitask learning.
result Improves survival analysis and power prediction in truck fleets and wind farms.
Improved power arbitrage through domain-adapted reinforcement learning.
problem Optimizing profit in the Dutch power market through arbitrage opportunities.
method Dual-agent reinforcement learning with imitation of power traders' behaviors.
result Significant improvement in cumulative profit and loss (P&L) with a three-fold increase.
Competency questions help experts select best clustering for energy data.
problem Ad hoc and subjective selection of clustering structures by domain experts.
method Formalize expert knowledge and requirements with competency questions.
result Competency questions improve reproducibility and evaluation of clustering applications.
Paper proves convergence of warped product manifolds to a nonnegative scalar curvature limit.
problem Proving convergence of sequences of manifolds with nonnegative scalar curvature.
method Warped product manifolds with diverging circular fibers, proving convergence in W1,p sense. result Sequence converges to an extreme limit space with nonnegative scalar curvature.
Introduces BCVI, a Bayesian cluster validity index for better cluster selection.
problem Selecting the optimal number of clusters in clustering algorithms.
method Develops BCVI using Dirichlet or generalized Dirichlet priors, evaluated against existing indices.
result BCVI offers clear advantages in scenarios where user expertise is valuable.
KEEL improves causal discovery with fuzzy knowledge and complex data.
problem Challenges in causal discovery due to prior knowledge, domain inconsistencies, and small sample sizes.
method Weakly-supervised fuzzy knowledge and data co-driven causal discovery method (KEEL).
result KEEL outperforms state-of-the-art methods in accuracy, robustness, and computational efficiency.
Synapse arbitrates TSFMs to improve time series forecasting performance.
problem TSFMs vary in performance across different forecasting tasks, domains, and horizons.
method Synapse dynamically assigns and adjusts predictive weights based on TSFM performance.
result Synapse consistently outperforms other ensembling techniques and individual TSFMs.
Research identifies risks in selecting project managers for civil engineering projects.
problem Lack of awareness of project manager selection criteria and associated risks.
method Combined ANP-FMEA approach for risk analysis.
result ANP-FMEA model identifies more significant risks than traditional FMEA.
We propose the nuclear norm penalty as an alternative to the ridge penalty for regularized multinomial regression. This convex relaxation of reduced-rank multinomial regression has the advantage of leveraging underlying structure among the response categories to make better predictions. We apply our method, nuclear pen…
Study shows LLM-advisors match human performance in eliciting preferences but struggle with conflicting needs and trust.
problem How do LLM-advisors perform in complex financial domains where domain expertise is crucial?
method Lab-based user study with 64 participants, focusing on three challenges: preference elicitation, personalized guidance, and relationship building.
result LLM-advisors can match human performance in preference elicitation but struggle with conflicting needs and trust issues.