Unified Bayesian Optimization framework for model selection balancing effectiveness and training efficiency.
problem Balancing model effectiveness and training efficiency in machine learning model selection.
method Proposes a unified Bayesian Optimization framework to jointly optimize model effectiveness and training efficiency.
result Models selected using the proposed framework significantly improve training efficiency while maintaining strong effectiveness.
Optimizes user marketing campaigns to balance cost and effectiveness.
problem Lack of methods to optimize marketing campaigns considering cost and effectiveness.
method Proposes a treatment effect optimization algorithm using deep learning to balance cost and effectiveness.
result Demonstrates superior performance in cost-efficiency and real-world business value.
In causal inference, a variety of causal effect estimands have been studied, including the sample, uncensored, target, conditional, optimal subpopulation, and optimal weighted average treatment effects. Ad-hoc methods have been developed for each estimand based on inverse probability weighting (IPW) and on outcome regr…
Paper proves optimality of doubly robust estimators for treatment effects.
problem Estimating treatment effects in causal inference.
method Structure-agnostic framework of statistical lower bounds, using non-parametric regression and classification oracles.
result Doubly robust estimators are statistically optimal for ATE and ATT.
Investment herding can reduce household consumption, a phenomenon called crowding-out effect.
problem Investment herding's impact on household consumption.
method Optimal control theory to model and solve for household investment and consumption decisions.
result Existence of crowding-out effect due to investment herding.
Optimizes causal effects on unknown graphs using Causal Entropy Optimization.
problem Optimizing causal effects in unknown causal graphs.
method Causal Entropy Optimization (CEO) framework that generalizes Causal Bayesian Optimization (CBO). Incorporates causal structure uncertainty in surrogate models and intervention selection.
result CEO achieves faster convergence to global optimum compared to CBO and improves upon sequential structure learning.
In this paper, we show the spooky effect at a distance that arises in optimal estimation of multiple targets with the optimal sub-pattern assignment (OSPA) metric. This effect refers to the fact that if we have several independent potential targets at distant locations, a change in the probability of existence of one o…
New framework for choosing optimal proxy metrics from past experiments.
problem Difficult to measure long-term treatment effects in experiments.
method Statistical framework to define and construct optimal proxy metrics.
result Optimal proxy metric depends on experiment's sample size.
KOOW method provides optimal covariate balance for continuous treatments.
problem Estimating effects of continuous treatments with robustness to model misspecification and extreme weights.
method Kernel Optimal Orthogonality Weighting (KOOW) using convex optimization.
result KOOW provides optimal covariate balance and controls for extreme weights.
GP-UCB performs suboptimally under certain conditions, as shown by a new regret lower bound.
problem The suboptimality of GP-UCB under polynomial effective optimism.
method Analysis of effective optimism level and new regret lower bound.
result GP-UCB is not minimax optimal under polynomial growth of effective optimism.
Recent work on Bayesian optimization has shown its effectiveness in global optimization of difficult black-box objective functions. Many real-world optimization problems of interest also have constraints which are unknown a priori. In this paper, we study Bayesian optimization for constrained problems in the general ca…
Proposes ESCFR to estimate treatment effects from biased data.
problem Treatment selection bias in observational data.
method Stochastic optimal transport with relaxed mass-preserving and proximal factual outcome regularizers.
result Significantly better performance in estimating treatment effects.
Optimizes treatment allocation in networks considering indirect effects.
problem Finding optimal treatment allocation in network settings with interference.
method OTAPI: Optimizing Treatment Allocation in the Presence of Interference, integrating causal estimators into IM algorithms.
result OTAPI outperforms classic IM and UM approaches on synthetic and semi-synthetic datasets.
Weight decay is one of the standard tricks in the neural network toolbox, but the reasons for its regularization effect are poorly understood, and recent results have cast doubt on the traditional interpretation in terms of L2 regularization. Literal weight decay has been shown to outperform L2 regularization for…
AdamP optimizes momentum-based optimizers for scale-invariant weights, improving model performance.
problem Premature decay of effective step sizes in momentum-based optimizers for scale-invariant weights.
method Proposes SGDP and AdamP to eliminate the radial component at each optimizer step, preserving convergence properties.
result Uniform gains across multiple benchmarks, improving model performance.
SignSGD analysis quantifies its effects in high dimensions.
problem Understanding signSGD's effects in high-dimensional settings.
method High-dimensional analysis of signSGD, deriving SDE and ODE for risk.
result Quantification of signSGD's effects: effective learning rate, noise compression, diagonal preconditioning, gradient noise reshaping.
Optimizes molecular generation for chemist preferences.
problem Models lack inherent preferences for chemist-desired structures.
method Fine-tuning with Direct Preference Optimization.
result Approach is simple, efficient, and highly effective.
Bayesian model averaging improves causal effect estimation by averaging over multiple models.
problem Estimating causal effects under linear Structural Causal Models (SCMs).
method Bayesian model averaging using Gaussian scale mixture distributions for computational efficiency.
result Bayesian model averaging is optimal for causal effect estimation.
Unintended effects from scaling neural network outputs with adaptive learning rates.
problem Adaptive learning rate optimization's behavior is altered by output scaling, leading to misinterpretation.
method Presented a modified optimization algorithm to mitigate unintended effects.
result Adaptive learning rate's effectiveness is significantly impacted by output scaling, especially for small scaling factors.
DEBIAS learns causal effects from psychiatric longitudinal data by optimizing outcome weights.
problem Causal inference challenges in psychiatric longitudinal data due to symptom heterogeneity and latent confounding.
method DEBIAS algorithm that optimizes outcome weights to maximize durable treatment effects and minimize confounding.
result DEBIAS consistently outperforms state-of-the-art methods in recovering causal effects for clinically interpretable composite outcomes.
Machine learning algorithms typically rely on optimization subroutines and are well-known to provide very effective outcomes for many types of problems. Here, we flip the reliance and ask the reverse question: can machine learning algorithms lead to more effective outcomes for optimization problems? Our goal is to trai…
Paper optimizes experimental design for estimating treatment effect.
problem Estimating treatment effect with heterogeneous subjects and treatments.
method Adaptive experimental design incorporating bandit learning.
result Demonstrates optimality of proposed adaptive experiment framework.
A new method improves efficiency in finding optimal personalized treatment rules.
problem Heteroscedasticity and misspecified treatment-free effect models affect optimal ITR estimation.
method E-Learning framework that accounts for covariate-treatment dependent variance of residuals.
result E-Learning framework improves efficiency of optimal ITR estimation.
The Rashomon effect shows many models can perform similarly, explored in this paper.
problem Why do many models perform similarly in machine learning?
method Categorized causes into statistical, structural, and procedural sources.
result Structural multiplicity persists and cannot be resolved without additional assumptions.
A2D2E designs optimal evaluations for estimating main effects in black-box models.
problem Sensitivity to OOD evaluations and instability under feature correlation in estimating black-box model effects.
method A2D2E uses D-optimal hypercube designs to minimize main effect estimation variance.
result A2D2E outperforms existing methods, especially under high feature correlation.
Proposes a framework to reconcile policy learning and profit maximization in CATE estimation.
problem Aligning CATE estimation with profit maximization for optimal customer treatment decisions.
method Optimizes a novel objective function that concentrates learning capacity near the decision boundary, ensuring consistency with the original profit function.
result Consistent CATE estimates can be recovered from existing profit-maximization pipelines, allowing firms to navigate the trade-off between accuracy and profit.
We consider first order gradient methods for effectively optimizing a composite objective in the form of a sum of smooth and, potentially, non-smooth functions. We present accelerated and adaptive gradient methods, called FLAG and FLARE, which can offer the best of both worlds. They can achieve the optimal convergence …
Hybrid quantum algorithm tackles binary optimization problems with multiple constraints.
problem Efficiently solving binary optimization problems with multiple constraints using quantum algorithms.
method Combines QAOA with penalty dephasing and Zeno effect for non-Ising constraints.
result Significant improvement in solving practical aircraft loading problems.
RoME optimizes mobile health interventions by modeling user and time-specific effects.
problem Challenges in optimizing mobile health interventions due to participant heterogeneity, nonstationarity, and nonlinear relationships.
method RoME uses a Robust Mixed-Effects contextual bandit algorithm with random effects, network cohesion penalties, and debiased machine learning.
result RoME achieves robust regret bounds even with complex baseline rewards, demonstrating superior performance in simulations and studies.
The paper develops a method to optimize individualized treatment rules for cost-effectiveness.
problem Developing cost-effective individualized treatment rules for healthcare policy.
method Using conditional random forest and net-monetary-benefit (NMB) to estimate optimal CE-ITR.
result The approach optimizes healthcare resource allocation by maximizing health gains and minimizing costs.
The paper analyzes how wealth affects investment strategies in incomplete markets.
problem Investment strategies in markets with incomplete information.
method Developed a five-component decomposition for optimal portfolio choice, solved explicitly for HARA utility and nonrandom interest rate, and used a stochastic volatility model for US equity data.
result Demonstrated the impacts of wealth-dependent utilities on optimal portfolio allocation, including cycle-dependence and hysteresis effect.
The paper optimizes spatial experimental designs to improve causal effect estimation.
problem Optimizing spatial experimental designs to enhance causal effect estimation accuracy.
method Proposes a surrogate function for MSE and uses graph cut algorithms to learn optimal designs.
result The method accommodates spatial interference and covariance, is computationally efficient, and validated by theoretical and numerical experiments.
Study optimizes trading in multiple assets with cross-effects.
problem Optimizing trade execution in multiple assets with cross-impact effects.
method Formulated as a stochastic control problem, extended to progressively measurable controls, solved using linear-quadratic control theory.
result Cross-hedging effects can be optimal, e.g., trading in an asset without an initial position.
This paper investigates the nonparametric regression problem using SVMs with anisotropic Gaussian RBF kernels. Under the assumption that the target functions are resided in certain anisotropic Besov spaces, we establish the almost optimal learning rates, more precisely, optimal up to some logarithmic factor, presented …
The optimal approach is to theorize after examining data, not before.
problem Optimal sequencing of theory and empirical analysis for economic questions.
method Formalized a Bayesian model to trade off Darwinian and Statistical Learning.
result Post hoc theorizing is typically optimal in modern economics.
Sensitivity analysis for individualized effects in OTRs with binary risk factors.
problem Addressing omitted confounding in individualized effects of OTRs.
method Simulation-based sensitivity analysis to simulate unmeasured confounders.
result Benchmarking the strength of omitted confounding for binary risk factors.
Addressing the issue of SVMs parameters optimization, this study proposes an efficient memetic algorithm based on Particle Swarm Optimization algorithm (PSO) and Pattern Search (PS). In the proposed memetic algorithm, PSO is responsible for exploration of the search space and the detection of the potential regions with…
MOMENT selects and estimates mixed-effects models using moment identities.
problem Selecting and estimating random-effects covariance matrix and fixed-effects coefficients in multiresponse linear mixed-effects models.
method MOMENT is a stage-wise moment-based framework that reduces the random-effects selection problem to a smooth constrained convex optimization problem.
result MOMENT performs competitively and can outperform separate univariate analyses for correlated responses.
Gradient methods work well on overparameterized diagonal linear networks.
problem Understanding why gradient-based methods work well in overparameterized models.
method Study of Deep Diagonal Linear Networks with gradient flow analysis.
result Gradient flow on layer parameters induces a mirror-flow dynamic in the effective parameter space, leading to explicit convergence guarantees.
Bayesian optimization for long-term outcomes using fast and slow experiments.
problem Optimizing long-term system effects with short-term misleading results.
method Combining fast and slow experiments for Bayesian optimization.
result Sequential optimization over large action spaces in a short time.
Bayesian optimization has proven to be a highly effective methodology for the global optimization of unknown, expensive and multimodal functions. The ability to accurately model distributions over functions is critical to the effectiveness of Bayesian optimization. Although Gaussian processes provide a flexible prior o…
This study optimizes covariate density and propensity score for efficient ATE estimation.
problem Efficiently estimating average treatment effects (ATEs) with minimal variance.
method Adaptive experiment optimizing both covariate density and propensity score.
result Proposed method minimizes the semiparametric efficiency bound for ATE estimation.
New method uses SLL to create masks for PX in noisy optimization problems.
problem Effective optimization in noisy problems with hidden variable dependencies.
method Statistical Linkage Learning (SLL) for decomposition and mask construction.
result Proposed method maintains effectiveness in noisy conditions and outperforms state-of-the-art.
Muon optimizer simplifies matrix optimization with spectral orthogonalization.
problem Matrix optimization challenges, especially with large condition numbers.
method Simplified Muon optimizer using spectral orthogonalization of gradients.
result Simplified Muon converges linearly with independent scalar sequences, outperforming gradient descent and Adam.
New measure captures differences across entire distributions of counterfactual outcomes.
problem Capturing differences across entire distributions of counterfactual outcomes.
method Entropic optimal transport measure, statistical functional, smooth transformation of embeddings.
result Established first-order and second-order pathwise differentiability.
A novel framework synthesizes treatment data across sites using optimal transport.
problem Estimating treatment effects across different sites with varying conditions.
method Distributional causal inference, Optimal Transport for alignment of control group distributions.
result Synthetic treatment group data aligns with true target distribution under general conditions.
New algorithm improves causal effect estimation for continuous treatments.
problem Observational causal inference with continuous treatments.
method End-to-end entropy balancing for maximizing causal inference accuracy.
result Our algorithm estimates causal effect more accurately than baseline.
Analyzes robust portfolio optimization with multi-factor stochastic volatility.
problem Optimizing portfolios under uncertainty and volatility risks.
method Analytical derivation of optimal strategy under worst-case scenarios, comparison with strategies ignoring uncertainty, and numerical experiments.
result Effects of ambiguity and derivative trading on optimal portfolio selection.