A new framework designs experiments for better decision-making.
problem Suboptimal experimental designs for downstream decision-making.
method Amortized decision-aware Bayesian Experimental Design (BED) with Transformer Neural Decision Process (TNDP).
result TNDP effectively designs experiments and facilitates accurate decision-making.
GoBOED optimizes experiments for specific decision-making objectives, improving downstream outcomes.
problem Reducing parameter uncertainty does not always improve decision-making in critical settings.
method Combines variational posterior surrogate and differentiable convex decision layer for gradient-based design optimization.
result GoBOED identifies designs that better align with specific decision objectives and reveals wider optimal design windows.
Linguistic calibration improves long-form text confidence.
problem LMs hallucinate, leading to suboptimal decisions.
method Defining linguistic calibration, training framework, reinforcement learning.
result Llama 2 7B is significantly more calibrated than baselines.
New approach to fairness in machine learning models using conformal prediction.
problem Fairness in machine learning models' downstream decision-making.
method Theoretical derivation and empirical evaluation of label-clustered conformal prediction.
result Label-clustered conformal prediction often provides a favorable balance between utility and substantive fairness.
Algorithm improves decision-making with partially observed contexts using pretrained models.
problem Improving decision-making with partially observed contexts in online linear contextual bandits.
method PULSE-UCB algorithm that uses pretrained models trained on auxiliary data to impute missing features.
result Achieves near-optimal performance in i.i.d. context case with Hölder-smooth missing features.
Bayesian models quantify uncertainty and facilitate optimal decision-making in downstream applications. For most models, however, practitioners are forced to use approximate inference techniques that lead to sub-optimal decisions due to incorrect posterior predictive distributions. We present a novel approach that corr…
Develops optimal uncertainty quantification for risk-averse decision makers.
problem Quantifying prediction uncertainty for risk-sensitive domains.
method Decision-theoretic foundations connecting uncertainty quantification with risk-averse decision-making.
result Risk-Averse Calibration (RAC) algorithm provides optimal prediction sets for risk-averse decision makers.
This paper generalizes BO uncertainty measures using decision-theoretic entropies.
problem Efficiently inferring optima of expensive black-box functions.
method Introduces a generalized entropy measure from statistical decision theory to optimize Bayesian optimization.
result Demonstrates strong empirical performance across various sequential decision-making tasks.
A new method for decision-focused learning reduces computational cost.
problem Efficiently solving combinatorial problems with uncertain parameters.
method Reframed as cost-sensitive multi-output regression, with novel loss components.
result Comparable downstream task quality with reduced computational cost.
Survey on calibration in machine learning, viewing it as indistinguishability.
problem Evaluating continuous probability predictions in discrete outcome settings.
method Defining and measuring calibration error through indistinguishability.
result Calibration measures quantify distinguishability between hypothesized and real-world outcomes.
Optimal decision-making using prediction sets to minimize risk.
problem Using prediction sets optimally for decision-making in uncertain scenarios.
method Decision-theoretic framework that seeks to minimize expected loss against a worst-case distribution.
result ROCP algorithm reduces critical mistakes compared to baselines, especially in costly out-of-set errors.
New approach calibrates predictions for better decision-making.
problem Achieving reliable predictions for multi-class problems is hard.
method Introduces decision calibration, a new approach to calibrate predictions.
result Designs a recalibration algorithm that makes predictions reliable for decision-making.
Proposes a new machine learning problem for automated temporal decision-making.
problem Automating human involvement in temporal decision-making processes.
method Develops a deep Bayesian neural network, ForeClassNet, with Boltzmann convolutions.
result Achieves superior performance in real-world Foreclassing datasets.
Develops methods for AI self-assessment to improve trustworthiness.
problem Uncertainty in AI predictions and lack of trust in AI systems.
method Uncertainty estimation techniques considering practical impacts and costs.
result Guidelines for selecting and designing effective AI self-assessment methods.
FWC creates fair synthetic samples for machine learning tasks.
problem Addressing biases in machine learning models for fair decision-making.
method FWC uses an efficient majority minimization algorithm to minimize Wasserstein distance while enforcing demographic parity.
result FWC achieves a competitive fairness-utility tradeoff and reduces biases in predictions from large language models.
The paper proposes a method to learn and leverage contextual preference distributions for better decision-making.
problem Heterogeneous and context-dependent human preferences in decision-making problems.
method A sequential learning-and-optimization pipeline using a bounded-variance score function gradient estimator to train a predictive model mapping contextual features to preference distributions.
result The approach reduces average post-decision surprise by up to 25 times compared to risk-averse baselines in a ridesharing environment.
Decision-alignment evaluates uncertainty quantification for decision-relevant UQ
problem Evaluation of uncertainty quantification metrics
method Introduce decision-alignment
result Proper scoring rules align with decision utility
Optimizes decision-making with uncertain variables using auxiliary observations.
problem Contextual stochastic optimization problems with uncertain variables and rich auxiliary observations.
method Trains forest decision policies by growing trees that optimize downstream decision quality, using optimization perturbation analysis for efficient approximations.
result Proves asymptotic optimality and empirical validation of the method's performance and efficiency.
In many machine learning applications, there are multiple decision-makers involved, both automated and human. The interaction between these agents often goes unaddressed in algorithmic development. In this work, we explore a simple version of this interaction with a two-stage framework containing an automated model and…
New decision-theoretic calibration error metric improves prediction reliability.
problem Improving the reliability of predictions for decision-making.
method Proposed Calibration Decision Loss (CDL) and an efficient algorithm to achieve near-optimal CDL.
result Near-optimal CDL guarantees vanishing payoff loss from miscalibration.
LLMs compress financial texts, but distort decision-making.
problem LLMs compress financial texts, altering decision-making.
method Analyzed two diagnostic patterns: decontextualization and model dependency. Proposed Agentic Context Compression.
result LLM-compressed financial texts alter decision-making.
Proposes FairRR to improve fairness in machine learning models through randomized response.
problem Achieving group fairness in machine learning models.
method Formulates group fairness as optimizing a design matrix in Randomized Response, proposing FairRR.
result Demonstrates FairRR yields excellent model utility and fairness.
LLMs produce volatile sentence-level sentiment classifications that affect financial decision-making.
problem Volatile outputs from LLMs impact financial text understanding tasks.
method Case study on US equity market investing via news sentiment analysis.
result Volatile LLM outputs lead to significant variations in portfolio construction and returns.
To make decisions based on a model fit with auto-encoding variational Bayes (AEVB), practitioners often let the variational distribution serve as a surrogate for the posterior distribution. This approach yields biased estimates of the expected risk, and therefore leads to poor decisions for two reasons. First, the mode…
Contextual linear optimization shows naive plug-in methods can outperform direct optimization.
problem Optimizing decisions with side observations to reduce uncertainty.
method Using off-the-shelf machine learning methods to learn a predictive model and plug it in for optimization.
result The naive plug-in approach achieves faster regret convergence rates than direct optimization methods.
Cost-effective method detects language model hallucinations.
problem Detecting unreliable outputs from LLMs.
method Pipeline including confidence score, input attributes calibration, and thresholding.
result Multi-scoring framework outperforms individual methods and reduces computational cost.
Paper proposes consistent estimators for learning to defer decisions to experts.
problem Learning algorithms often ignore expert decision-making in practical scenarios.
method Reduction to cost sensitive learning, novel surrogate loss for consistent estimation.
result Effective approach demonstrated on various tasks, showing consistency.
A smart method predicts and optimizes decisions online with resource constraints.
problem Online decision-making with resource constraints.
method Combines prediction and optimization with dual update using mirror descent.
result Regret bounds and convergence rates for general convex feasible regions.
The paper explores fair clustering, a niche area in machine learning.
problem Fairness in clustering remains underexplored despite its importance.
method Assesses existing work and proposes new directions for fair clustering.
result Widening normative principles and knowledge of downstream processes can enhance fair clustering research.
New online algorithms tackle dynamic decision-focused learning.
problem Dynamic decision-focused learning in evolving environments.
method Regularization and perturbation techniques for non-convex optimization.
result First provable guarantees for online decision-focused learning.
New method optimizes Gaussian process allocation for BO.
problem Existing methods for inducing point allocation in BO hinder performance.
method Proposes a new allocation strategy using quality-diversity decomposition.
result Demonstrates improved BO performance through local high-fidelity modeling.
New scaling laws optimize model size, training, and inference for better performance.
problem Trade-off between model size and inference cost in modern LLMs.
method Train-to-Test (T2) scaling laws that jointly optimize model size, training tokens, and inference samples. result Optimal pretraining decisions shift into overtraining regime, leading to stronger performance.
CIT and CIF improve feature selection for downstream prediction.
problem Feature selection bias in machine learning models.
method Conditional inference trees and forests with Bonferroni correction.
result CIF ranks top 3 among 18 regression methods and top 4 among 17 classification methods.
Proposes a method to learn adaptive ambiguity sets for robust optimization.
problem Misspecification in distributionally robust optimization (DRO).
method Learned predictive ambiguity sets (LPAS) using deep contextual models.
result Significantly improves portfolio optimization performance compared to baselines.
OTSS learns personalized decision weights from logged decisions and outputs.
problem Learning context-specific decision weights from logged decisions and outputs.
method Output-targeted soft-segmentation model that deploys personalized decision-ready weight vectors.
result OTSS achieves the lowest mean regret in benchmark settings.
Universal algorithm learns unknown distribution for various decision-making problems.
problem Various statistical measures in contextual sequential decision-making.
method Infinite-dimensional functional regression oracle for cumulative distribution functions.
result Utility regret rate bounded by polynomial decay of eigenvalue sequence.
New method optimizes treatment policies to avoid winner's curse.
problem Winner's curse in treatment policy optimization.
method Inference-aware policy optimization.
result Optimizes for both estimated performance and downstream evaluation.
Study optimal and equitable encouragement policies for treatment adherence.
problem Optimal treatment adherence policies in the presence of human non-adherence.
method Covariate-conditional no-direct-effect model of encouragement; tractable policy characterizations under constraints.
result Induced treatment take-up is the fairness target, not recommendation rates.
A new method for fair classification using characteristic function distance.
problem Fairness in high-stakes decision-making with sensitive groups.
method Proposes a novel approach based on characteristic function distance to ensure minimal sensitive information in learned representations.
result Consistently matches or achieves better fairness and predictive accuracy than existing methods.
End-to-end model predicts multiagent trajectories using game theory and neural nets.
problem Predicting trajectories of interacting agents in complex scenarios.
method Hybrid neural net with game-theoretic reasoning, using implicit layers to map preferences to Nash equilibria.
result Trains an interpretable model that predicts future trajectories and transfers to decision making.
Study shows uncertainty of deep learning models can be measured from their embeddings.
problem Uncertainty in contrastive learning models for critical applications.
method Estimating the distribution of training data in embedding space and accounting for local consistency.
result Uncertainty of an embedding vector correlates strongly with downstream accuracy.
DiffLOB models future market conditions for better decision-making.
problem Passive generative models cannot explore hypothetical market scenarios.
method Regime-conditioned diffusion model for counterfactual LOB generation.
result DiffLOB enables realistic and controllable generation of LOB trajectories.
A new robust scaling approach improves downstream metabolomics analysis.
problem Challenges in choosing scaling techniques for metabolomics data.
method Introduces a weighted scaling approach robust to outliers.
result The proposed method outperforms traditional scaling techniques in both outlier-free and outlier-present datasets.
TaskMet learns a metric to improve model performance on unseen tasks.
problem Deep models trained on one task may struggle on another task due to conflicting objectives.
method TaskMet learns a metric in the prediction space to balance task and prediction losses.
result TaskMet achieves better performance on downstream tasks without altering the prediction model.
Multiverse analysis helps prevent fairness hacking and evaluate model design decisions.
problem Downstream effects of ADM systems depend on implicit design and evaluation decisions.
method Turn implicit decisions into explicit ones, create a grid of decision combinations, compute fairness and performance metrics.
result Decisions regarding evaluation can lead to vastly different fairness metrics for the same model.
People are rated and ranked, towards algorithmic decision making in an increasing number of applications, typically based on machine learning. Research on how to incorporate fairness into such tasks has prevalently pursued the paradigm of group fairness: giving adequate success rates to specifically protected groups. I…
Decision-calibrated prediction sets improve power system operations by reducing unnecessary costs.
problem Balancing operating costs and reliability in power systems with renewable uncertainty.
method Learn conditional prediction sets as sub-level sets of norm-based score functions, calibrate uncertainty sets based on reliability of downstream decisions.
result Decision-calibrated sets lead to more efficient operations with smaller uncertainty sets and lower costs compared to standard coverage-based calibration.
Paper introduces novel model selection for CRO to balance robustness and decision risk.
problem Balancing robustness and decision risk in CRO.
method Conformalized Robust Optimization with Model Selection (CROMS) framework.
result Significant improvements in decision efficiency across various applications.