SAIL learns from sub-optimal demonstrations to improve sample efficiency in sparse reward tasks.
problem Reducing sample complexity in sparse-rewarded tasks.
method Self-Adaptive Imitation Learning (SAIL) that exploits sub-optimal demonstrations and efficient exploration.
result Significantly improved sample efficiency and better final performance across various tasks.
LEARN-SAM improves RL from sub-optimal demonstrations by localizing expert policies and selectively using demonstrations.
problem Improving RL from sub-optimal or sparse demonstrations.
method Local Ensemble and Reparameterization with Split and Merge of expert policies (LEARN-SAM).
result LEARN-SAM boosts learning speed and accuracy by selectively using demonstrations.
A method for training autonomous vehicles using continuous human feedback to avoid sub-optimal decisions.
problem Training autonomous vehicles with limited and potentially sub-optimal human demonstrations.
method Continuous scalar feedback for each action to learn from sub-optimal demonstrations and evaluative feedback.
result The proposed method outperforms supervised learning on positive examples alone and learns from sub-optimal demonstrations.
Unified approach translates classic bandit algorithms to structured settings.
problem Finite-armed structured bandit problem with unknown reward functions.
method Gradual estimation of hidden parameter θ* and use in mean reward functions.
result Structured bandit versions of UCB achieve bounded regret in practical scenarios.
This paper considers the problem of inverse reinforcement learning in zero-sum stochastic games when expert demonstrations are known to be not optimal. Compared to previous works that decouple agents in the game by assuming optimality in expert strategies, we introduce a new objective function that directly pits expert…
In this paper, we propose to combine imitation and reinforcement learning via the idea of reward shaping using an oracle. We study the effectiveness of the near-optimal cost-to-go oracle on the planning horizon and demonstrate that the cost-to-go oracle shortens the learner's planning horizon as function of its accurac…
New method improves IL from imperfect demos using confidence scores.
problem Learning optimal policies from imperfect demonstrations is challenging.
method Proposes two confidence-based IL methods: 2IWIL and IC-GAIL.
result Confidence scores from sub-optimal demos significantly improve IL performance.
Unified hybrid RL algorithm improves online RL performance with offline data.
problem Improving reinforcement learning performance with limited online data.
method A unified hybrid RL algorithm combining offline and online data.
result Unified algorithm achieves state-of-the-art results in sub-optimality gap and online learning regret.
Certainty equivalent controllers perform nearly optimally in LQ control problems with unknown dynamics.
problem Optimizing control in systems with unknown transition dynamics.
method Analysis of certainty equivalent controllers and comparison to optimal LQ controllers, using perturbation bounds for discrete Riccati equations.
result Sub-optimality gap scales as the square of the parameter error, improving upon previous results.
Study finds average 2.02 bps loss in automated market maker routing.
problem Measuring sub-optimality in automated market maker routing.
method Three reproducible optimal benchmarks: SCO, FVO, G-FVO; bisection-based algorithm for optimal routing.
result Average 2.02 bps loss per trade, \$24 million total loss.
New measure quantifies function similarity for optimization.
problem Measuring similarity between functions for optimization.
method Quantifies sub-optimality gaps and operation rules.
result Unified measure for various functional similarities.
New proof shows deep neural nets can have sub-optimal local minima.
problem Can over-parameterization eliminate sub-optimal local minima in deep neural networks?
method Counter-example with generic input data and non-linear activation functions.
result Sub-optimal local minima exist in deep neural networks regardless of width.
We solve S-shaped utility portfolio selection with SD constraints using algorithms and neural networks.
problem Optimizing portfolios with S-shaped utility functions under SD constraints.
method First-order SD constraint solution, numerical algorithm for SSD, neural network approach.
result Effective numerical and neural network solutions for SSD constrained problems.
EBQL reduces bias in Q-learning for improved performance.
problem Over- and under-estimation biases in Q-learning degrade performance.
method Ensemble Bootstrapping to reduce both over- and under-estimation biases.
result EBQL outperforms other Q-learning methods in Atari games.
Algorithm safely learns from sub-optimal baseline policies while satisfying constraints.
problem Safe reinforcement learning with constraints when baseline policy is sub-optimal.
method Iterative policy optimization alternating between return maximization, baseline distance minimization, and constraint projection.
result Consistently outperforms baselines, achieving 10x fewer constraint violations and 40% higher reward.
AdMRL improves meta-reinforcement learning by minimizing worst-case sub-optimality gap.
problem Meta-reinforcement learning's sensitivity to task distribution shift.
method Model-based adversarial approach with minimax objective and alternating optimization.
result Efficacy in worst-case performance, generalization to out-of-distribution tasks, and sample efficiency.
A new method learns policies from sub-optimal trajectories.
problem Reinforcement learning's brittleness and difficulty in tuning.
method Conditioning policies on the reward of sub-optimal trajectories.
result Effective policies can be learned without expert demonstrations.
UAIL uses uncertainty estimation to improve control systems in safety-critical tasks.
problem Improving control systems in safety-critical domains like autonomous driving.
method UAIL applies Monte Carlo Dropout to estimate uncertainty in control output and selectively acquire new training data.
result UAIL can reliably predict infractions and outperforms existing algorithms.
Algorithm reduces regret in misspecified linear contextual bandits.
problem Misspecified linear contextual bandits with bounded misspecification.
method Data selection scheme for online regression, leveraging uncertainty.
result Regret bound of O ~ ( d 2 / Δ ) \tilde O(d^2/Δ) O ~ ( d 2 /Δ ) when ζ ≤ O ~ ( Δ / d ) ζ \leq \tilde O(Δ/\sqrt{d}) ζ ≤ O ~ ( Δ/ d ) . Background: Statistical mechanics results (Dauphin et al. (2014); Choromanska et al. (2015)) suggest that local minima with high error are exponentially rare in high dimensions. However, to prove low error guarantees for Multilayer Neural Networks (MNNs), previous works so far required either a heavily modified MNN mod…
Paper analyzes convergence of entropy-regularized reinforcement learning methods.
problem Ensuring sub-optimality control in entropy-regularized RL.
method Unified analysis of regularized MPI and VI schemes, providing convergence rates.
result Established sufficient conditions for convergence and provided explicit rates.
Wide networks avoid bad local minima, proving phase transition.
problem Understanding the optimization landscape of neural networks.
method Proving phase transitions in loss surfaces of wide and narrow networks.
result Phase transition from narrow to wide networks: no bad local minima.
Recently, network lasso has drawn many attentions due to its remarkable performance on simultaneous clustering and optimization. However, it usually suffers from the imperfect data (noise, missing values etc), and yields sub-optimal solutions. The reason is that it finds the similar instances according to their feature…
Logarithmic regret achieved in Q-learning with positive gap.
problem Achieving logarithmic cumulative regret in Q-learning with positive sub-optimality gap.
method Optimistic Q-learning with logarithmic regret bound.
result Logarithmic cumulative regret bound proven for optimistic Q-learning.
Paper tackles sample-efficient RL for linearly realizable MDPs with limited revisiting.
problem Sample-efficient reinforcement learning for linearly realizable MDPs with limited revisiting.
method Develops a new sampling protocol that allows for backtracking and revisiting states in a controlled manner.
result Achieves polynomial sample complexity scaling with feature dimension, horizon, and inverse sub-optimality gap.
Improves BC policies by generating new plausible trajectories.
problem Sub-optimal data quality in BC leads to poor policy performance.
method Trajectory Stitching (TS) generates new plausible transitions.
result TS significantly improves behavioural policies over original data.
AWAC combines offline and online data to accelerate RL learning.
problem Challenges in applying RL to real-world robotic control due to exploration and sample complexity.
method Combines sample-efficient dynamic programming with maximum likelihood policy updates.
result AWAC enables rapid learning of robotic skills with prior data and online experience.
CoCoRL learns safe constraints from demonstrations with unknown rewards.
problem Learning safe constraints from demonstrations with different unknown rewards.
method Convex Constraint Learning for Reinforcement Learning (CoCoRL) constructs a convex safe set based on demonstrations.
result CoCoRL learns constraints that lead to safe driving behavior and can safely transfer to different tasks and environments.
Softmax Bellman operator improves Q-function performance in RL despite sub-optimality.
problem Softmax Bellman operator's impact on value functions in RL is problematic.
method Revisited theoretical properties of softmax Bellman operator, proving convergence and overestimation reduction.
result Softmax Bellman operator leads to superior policies in practice, even outperforming double Q-learning.
The paper provides guarantees for feedback control with sensor errors.
problem Certifying performance and safety in feedback control systems with sensor errors.
method Solving a supervised learning problem to characterize sensor errors and providing uniform error bounds.
result Finite-time convergence rate on sub-optimality of using a regressor in closed-loop for waypoint tracking.
New algorithm reduces sample complexity for planning in MDPs.
problem Planning in MDPs with unknown transitions.
method MDP-GapE, a trajectory-based MCTS algorithm.
result Proves upper bound on sample complexity in terms of sub-optimality gaps.
Previous work on recommender systems mainly focus on fitting the ratings provided by users. However, the response patterns, i.e., some items are rated while others not, are generally ignored. We argue that failing to observe such response patterns can lead to biased parameter estimation and sub-optimal model performanc…
Characterizes kernel interpolation in large dimensions, revealing optimal and sub-optimal regions.
problem Understanding the phase diagram of kernel interpolation in large dimensions.
method Characterization of variance and bias under various source conditions.
result Determined the ( s , γ ) (s,γ) ( s , γ ) -phase diagram of large-dimensional kernel interpolation. New pricing algorithm learns demand curves and optimizes prices in dynamic markets.
problem Dynamic pricing in markets with incomplete demand information and shifting conditions.
method Actor-Critic Information-Directed Pricing (ACIDP) using IDS algorithms and auditing procedures.
result ACIDP outperforms UCB and TS in market environment shifts.
Learning how to act when there are many available actions in each state is a challenging task for Reinforcement Learning (RL) agents, especially when many of the actions are redundant or irrelevant. In such cases, it is sometimes easier to learn which actions not to take. In this work, we propose the Action-Elimination…
Enhances demonstrations with safety specifications using LTL.
problem Learning from demonstrations without safety specifications.
method Converts LTL specifications into a differentiable loss for training.
result System learns dynamic movement primitives that satisfy complex specifications.
Improved reinforcement learning with emergency stops.
problem Reducing exploration in reinforcement learning.
method Emergency stop mechanisms to reduce sample complexity.
result Significant improvement in sample complexity and speed.
This paper tackles robust policy learning under concept drifts, improving upon existing methods.
problem Tackles robust policy learning under concept drifts, improving upon existing methods.
method Develops a doubly-robust estimator and a learning algorithm to maximize policy value within a given policy class.
result The proposed algorithm achieves sub-optimality gap of the order κ ( Π ) n − 1 / 2 κ(Π)n^{-1/2} κ ( Π ) n − 1/2 , demonstrating substantial improvement over existing benchmarks. We state the problem of inverse reinforcement learning in terms of preference elicitation, resulting in a principled (Bayesian) statistical formulation. This generalises previous work on Bayesian inverse reinforcement learning and allows us to obtain a posterior distribution on the agent's preferences, policy and optio…
The paper develops Kalman filters for unknown systems with sample complexity bounds.
problem Designing Kalman filters for systems with unknown parameters and noise.
method Combines system identification with Kalman filter design, ensuring robustness and sub-optimality guarantees.
result Proves sub-optimality guarantees for both Certainty Equivalent and robust Kalman filters with sample complexity bounds.
New algorithm improves asset ranking for better cross-sectional portfolios.
problem Sub-optimal ranking of assets in cross-sectional systematic strategies.
method Learning-to-rank algorithms to enhance portfolio construction.
result Modern machine learning ranking algorithms boost Sharpe Ratios by approximately threefold.
The study explains how humans make sub-optimal decisions using learned approximations.
problem Human decision-making often deviates from optimal strategies.
method Combining meta-learning models with resource-bounded objectives.
result Empirical evidence shows patterns in human decision-making that align with the model.
Thompson Sampling fails to perform well in high dimensions.
problem Thompson Sampling's suboptimality in high-dimensional combinatorial semi-bandits.
method Analysis of TS for combinatorial semi-bandits, including non-linear and linear reward functions, with Bernoulli rewards and uniform priors.
result TS's regret scales exponentially in the ambient dimension and minimax regret scales almost linearly in high dimensions.
We introduce a globally-convergent algorithm for optimizing the tree-reweighted (TRW) variational objective over the marginal polytope. The algorithm is based on the conditional gradient method (Frank-Wolfe) and moves pseudomarginals within the marginal polytope through repeated maximum a posteriori (MAP) calls. This m…
Reviews recent findings on neural network landscapes.
problem Non-convexity of loss functions causing bad landscapes.
method Rigorous geometric analysis and empirical exploration.
result Wide neural nets may have sub-optimal local minima.
The MAP estimate's log-likelihood sub-optimality is hard to bound in general.
problem Bounding the expected log-likelihood sub-optimality of MAP for exponential families.
method Interpreting MAP as stochastic mirror descent and analyzing convergence rates.
result Current convergence results do not apply to standard examples of exponential families.
New method optimizes portfolio weights as functions, outperforming traditional approaches.
problem Optimizing portfolio weights in mean-variance models.
method Functional optimization approach, treating weights as functions of past values.
result Gradient-ascent algorithms can solve functional optimization problems for mean-variance portfolio management.
Corrects approximate Bayesian inference for better decision-making.
problem Sub-optimal decisions due to inaccurate posterior predictive distributions.
method Trains a separate model to correct decision-making under approximate posterior, combining Bayesian modeling with optimization.
result Empirically demonstrates improved predictive accuracy in various problems.