Paper tackles inverse reinforcement learning with non-optimal demonstrations in zero-sum games.
problem Inverse reinforcement learning with sub-optimal expert demonstrations in zero-sum games.
method Introduces a new objective function and algorithm to find reward function and strategies without decoupling agents.
result Demonstrates recovery of reward functions and strategies with good quality from sub-optimal expert demonstrations.
Certainty equivalent controllers perform nearly optimally in LQ control problems with unknown dynamics.
problem Optimizing control in systems with unknown transition dynamics.
method Analysis of certainty equivalent controllers and comparison to optimal LQ controllers, using perturbation bounds for discrete Riccati equations.
result Sub-optimality gap scales as the square of the parameter error, improving upon previous results.
The study examines how full information and rationality affect portfolio decisions in uncertain markets.
problem Analyzing welfare effects of sub-optimal investment strategies in uncertain financial markets.
method Quantitative analysis of Constant Relative Risk Aversion investor behavior under parameter uncertainty.
result Full information and predictability significantly impact utility effects, with learning effects being marginal.
New pricing algorithm learns demand curves and optimizes prices in dynamic markets.
problem Dynamic pricing in markets with incomplete demand information and shifting conditions.
method Actor-Critic Information-Directed Pricing (ACIDP) using IDS algorithms and auditing procedures.
result ACIDP outperforms UCB and TS in market environment shifts.
The study explains how humans make sub-optimal decisions using learned approximations.
problem Human decision-making often deviates from optimal strategies.
method Combining meta-learning models with resource-bounded objectives.
result Empirical evidence shows patterns in human decision-making that align with the model.
New sampling method for graph signals using DPPs for perfect recovery on small graphs, and sub-optimal but faster approach for large graphs.
problem Sampling k-bandlimited signals on graphs efficiently.
method Determinantal Point Processes (DPP) for both small and large graphs.
result Preliminary experiments show efficient sampling especially for graphs with strong community structure.
New algorithm improves asset ranking for better cross-sectional portfolios.
problem Sub-optimal ranking of assets in cross-sectional systematic strategies.
method Learning-to-rank algorithms to enhance portfolio construction.
result Modern machine learning ranking algorithms boost Sharpe Ratios by approximately threefold.
New active learning strategy improves decision-making accuracy.
problem Maximizing decision-making accuracy in sequential data acquisition.
method Introduces a novel active learning criterion that maximizes expected information gain on the posterior decision distribution.
result Improved performance in decision-making accuracy compared to existing alternatives.
Dynamic sentiment analysis improves stock trading strategies.
problem Improving stock trading performance through dynamic agent switching.
method Combining sentiment analysis with deep reinforcement learning and dynamic agent switching.
result Dynamic sentiment-based trading strategy outperforms traditional methods.
This paper revisits optimal investment strategies for defined contribution pension schemes using forward preferences.
problem Optimal investment strategies derived from backward models are not time-consistent and sub-optimal in real scenarios.
method Introduces forward preferences and solves optimal investment strategies for defined contribution pension schemes.
result Constructs optimal investment strategies for defined contribution pension schemes using forward preferences.
Review of diversification models finds simple rules still best.
problem Testing new diversification models for out-of-sample performance.
method Tested sixteen strategies across six datasets.
result No new models consistently outperform simple diversification rules.
We solve S-shaped utility portfolio selection with SD constraints using algorithms and neural networks.
problem Optimizing portfolios with S-shaped utility functions under SD constraints.
method First-order SD constraint solution, numerical algorithm for SSD, neural network approach.
result Effective numerical and neural network solutions for SSD constrained problems.
We study an original problem of pure exploration in a strategic bandit model motivated by Monte Carlo Tree Search. It consists in identifying the best action in a game, when the player may sample random outcomes of sequentially chosen pairs of actions. We propose two strategies for the fixed-confidence setting: Maximin…
Regularizes deep networks for k-shot learning with limited data.
problem Sub-optimal and overfitting issues in fine-tuning pre-trained deep networks for k-shot learning.
method Cluster model parameters, propagate gradients within clusters, use reinforcement learning for optimal group assignments.
result Improves k-shot learning performance by more than 10% compared to state-of-the-art methods.
We consider the problem of \textit{best arm identification} with a \textit{fixed budget T T T }, in the K K K -armed stochastic bandit setting, with arms distribution defined on [ 0 , 1 ] [0,1] [ 0 , 1 ] . We prove that any bandit strategy, for at least one bandit problem characterized by a complexity H H H , will misidentify the best arm with pr…
Study various numerical methods for expectation propagation in machine learning.
problem Improving the convergence and stability of the expectation propagation algorithm for large-scale learning tasks.
method Numerical approximation strategies including Laplace method, Gaussian quadrature, and variational sampling.
result Variational sampling yields the best convergence for the expectation propagation algorithm in training linear binary classifiers.
Under the optimal withdrawal strategy of a policyholder, the pricing of variable annuities with Guaranteed Minimum Withdrawal Benefit (GMWB) is an optimal stochastic control problem. The surrender feature available in marketed products allows termination of the contract before maturity, making it also an optimal stoppi…
Unified hybrid RL algorithm improves online RL performance with offline data.
problem Improving reinforcement learning performance with limited online data.
method A unified hybrid RL algorithm combining offline and online data.
result Unified algorithm achieves state-of-the-art results in sub-optimality gap and online learning regret.
ODS improves adversarial attacks by maximizing output diversity.
problem Efficiency and effectiveness of adversarial attacks, especially black-box attacks.
method Output Diversified Sampling (ODS) that maximizes diversity in model outputs.
result ODS reduces the number of queries needed for black-box attacks on ImageNet by a factor of two.
A method for training autonomous vehicles using continuous human feedback to avoid sub-optimal decisions.
problem Training autonomous vehicles with limited and potentially sub-optimal human demonstrations.
method Continuous scalar feedback for each action to learn from sub-optimal demonstrations and evaluative feedback.
result The proposed method outperforms supervised learning on positive examples alone and learns from sub-optimal demonstrations.
TuneUp improves GNN training by focusing on hard-to-learn nodes.
problem Sub-optimal training of GNNs on all nodes equally.
method Two-stage training: base GNN + tail node improvement.
result Significant improvement in tail node prediction performance.
SAIL learns from sub-optimal demonstrations to improve sample efficiency in sparse reward tasks.
problem Reducing sample complexity in sparse-rewarded tasks.
method Self-Adaptive Imitation Learning (SAIL) that exploits sub-optimal demonstrations and efficient exploration.
result Significantly improved sample efficiency and better final performance across various tasks.
This paper analyzes the Dawid-Skene model in the dense limit and identifies regions where message passing algorithms fail.
problem Optimality of message passing algorithms in the Dawid-Skene model.
method Analysis of the dense limit of the Dawid-Skene model and identification of regions of sub-optimality.
result Characterization of regions where message passing algorithms do not match Bayes-optimal performance.
The paper examines variable annuities pricing and risk management using the Black-Scholes model and identifies key risk drivers.
problem Model risk in pricing and managing variable annuities using the Black-Scholes model.
method Derives a model-free decomposition of variable annuity prices and investigates hedging strategies.
result The spot price risk can always be eliminated by the BS-based hedging strategy, but there is gradual slippage and instantaneous leakage.
Exponentially vanishing sub-optimal local minima in neural networks with high probability.
problem Proving low error guarantees for multilayer neural networks with standard normal inputs.
method Analyzing a specific MNN model with piecewise linear units and quadratic loss.
result The volume of differentiable regions containing sub-optimal local minima is exponentially vanishing compared to global minima.
Improving Bayesian Optimization via Training-Aware Conditional Diffusion Models
problem Bayesian Optimization
method Bayesian Optimization with Conditional Diffusion Models
result DMS outperforms standard BO baselines
AE-DQN learns to eliminate sub-optimal actions in complex RL environments.
problem Challenges in RL with many actions, especially redundancy.
method Combines DQN with AEN that predicts invalid actions.
result Significant speedup and robustness in games with many actions.
New measure quantifies function similarity for optimization.
problem Measuring similarity between functions for optimization.
method Quantifies sub-optimality gaps and operation rules.
result Unified measure for various functional similarities.
New proof shows deep neural nets can have sub-optimal local minima.
problem Can over-parameterization eliminate sub-optimal local minima in deep neural networks?
method Counter-example with generic input data and non-linear activation functions.
result Sub-optimal local minima exist in deep neural networks regardless of width.
Corrects approximate Bayesian inference for better decision-making.
problem Sub-optimal decisions due to inaccurate posterior predictive distributions.
method Trains a separate model to correct decision-making under approximate posterior, combining Bayesian modeling with optimization.
result Empirically demonstrates improved predictive accuracy in various problems.
Enhances currency strategy Sharpe ratio by 30% using context-aware Learning to Rank.
problem Sub-optimal ranking of assets during critical market periods.
method Context-aware Learning to Rank model based on Transformer architecture.
result Significantly improves Sharpe ratio and various performance metrics.
New policy optimizes product assortment in the presence of unpredictable customers.
problem Optimizing product assortment in the presence of outlier customers.
method Developed a robust online assortment optimization policy using an active elimination strategy.
result Established upper and lower bounds on regret, showing optimality up to logarithmic factor in T T T . Unified approach translates classic bandit algorithms to structured settings.
problem Finite-armed structured bandit problem with unknown reward functions.
method Gradual estimation of hidden parameter θ* and use in mean reward functions.
result Structured bandit versions of UCB achieve bounded regret in practical scenarios.
This paper tackles robust policy learning under concept drifts, improving upon existing methods.
problem Tackles robust policy learning under concept drifts, improving upon existing methods.
method Develops a doubly-robust estimator and a learning algorithm to maximize policy value within a given policy class.
result The proposed algorithm achieves sub-optimality gap of the order κ ( Π ) n − 1 / 2 κ(Π)n^{-1/2} κ ( Π ) n − 1/2 , demonstrating substantial improvement over existing benchmarks. This work analyzes aggregation strategies for Bayesian deep learning models in federated learning.
problem Improper aggregation of Bayesian deep learning models in federated learning leads to sub-optimal performance.
method Six aggregation strategies for Bayesian deep learning models are analyzed using CIFAR-10 dataset and a fully variational ResNet-20 architecture.
result Aggregation strategy is a key hyperparameter affecting accuracy, calibration, uncertainty quantification, training stability, and client compute requirements.
Algorithm reduces regret in misspecified linear contextual bandits.
problem Misspecified linear contextual bandits with bounded misspecification.
method Data selection scheme for online regression, leveraging uncertainty.
result Regret bound of O ~ ( d 2 / Δ ) \tilde O(d^2/Δ) O ~ ( d 2 /Δ ) when ζ ≤ O ~ ( Δ / d ) ζ \leq \tilde O(Δ/\sqrt{d}) ζ ≤ O ~ ( Δ/ d ) . Wide networks avoid bad local minima, proving phase transition.
problem Understanding the optimization landscape of neural networks.
method Proving phase transitions in loss surfaces of wide and narrow networks.
result Phase transition from narrow to wide networks: no bad local minima.
Paper introduces Market-adaptive Ratio for better portfolio management.
problem Traditional risk-adjusted ratios fail to account for bull and bear markets.
method Integrates ρ ρ ρ parameter and uses reinforcement learning to adjust portfolio allocations dynamically. result Market-adaptive Ratio outperforms traditional ratios in bull and bear markets.
Logarithmic regret achieved in Q-learning with positive gap.
problem Achieving logarithmic cumulative regret in Q-learning with positive sub-optimality gap.
method Optimistic Q-learning with logarithmic regret bound.
result Logarithmic cumulative regret bound proven for optimistic Q-learning.
Improved causal discovery methods for large graphs without strict assumptions.
problem Sub-optimal solutions due to faithfulness assumption violations.
method Super-structure estimation and local search strategies.
result The proposed method scales to hundreds of nodes with high accuracy.
Paper tackles sample-efficient RL for linearly realizable MDPs with limited revisiting.
problem Sample-efficient reinforcement learning for linearly realizable MDPs with limited revisiting.
method Develops a new sampling protocol that allows for backtracking and revisiting states in a controlled manner.
result Achieves polynomial sample complexity scaling with feature dimension, horizon, and inverse sub-optimality gap.
Deep fictitious play converges to Nash equilibrium in stochastic differential games.
problem Finding Nash equilibrium in large stochastic differential games.
method Decouples the game into sub-optimization problems and solves each player's optimal strategy with deep BSDE method.
result Deep fictitious play converges to the true Nash equilibrium.
This paper explores how imperfect reward models can improve online RLHF.
problem Sample efficiency in online RLHF from imperfect reward models.
method Identifies policy coverability and proposes TPO algorithm for transfer learning.
result TPO algorithm improves sample efficiency compared to standard online learning.
Study robustness of early-stopping GD for linear regression attacks.
problem Robustness of gradient-descent methods to adversarial attacks.
method Early-stopping strategies, gradient-descent, Mahalanobis attacks, feature-dependent learning rates, data transformations.
result Early-stopped GD is optimally robust to Euclidean-norm attacks but sub-optimal for Mahalanobis attacks.
LEARN-SAM improves RL from sub-optimal demonstrations by localizing expert policies and selectively using demonstrations.
problem Improving RL from sub-optimal or sparse demonstrations.
method Local Ensemble and Reparameterization with Split and Merge of expert policies (LEARN-SAM).
result LEARN-SAM boosts learning speed and accuracy by selectively using demonstrations.
Improves RL from historical data by stitching trajectories.
problem Lack of high-quality data for offline RL.
method Trajectory Stitching (TS) to augment historical data with synthetic actions.
result Improves RL policy performance over baseline.
Meta-learning variance reduced via Laplace approximation for regression tasks.
problem High variance in meta-learning due to limited support data for each task.
method Laplace approximation to estimate posterior variance and reduce gradient estimate variance.
result Effective variance reduction in meta-learning, improving generalization performance.
A bilevel optimization method for reweighting data leads to suboptimal results.
problem Reweighting data for better model performance on a different distribution.
method Formalizes data reweighting as a bilevel optimization problem and analyzes the limitations of classical solvers.
result The final data weights are often very sparse, indicating suboptimal solutions.