New algorithm for MIPS without preprocessing, controlling suboptimality.
problem Fast query time and suboptimality control for MIPS.
method Casting MIPS as a Best Arm Identification problem, introducing a new bandit setting.
result Outperforms state-of-the-art methods on synthetic and real-world datasets.
Guiding reinforcement learning with suboptimal controllers speeds up training.
problem Sparse rewards in reinforcement learning make exploration inefficient.
method Use a suboptimal controller to guide exploration, applying a Q-filter loss conditionally.
result The approach leads to faster policy refinement and better performance.
New algorithm avoids re-planning in tree-search algorithms, reducing suboptimal actions.
problem Avoiding re-planning in tree-search algorithms to reduce suboptimal actions.
method A new algorithm that decides at each step whether to re-plan or use a sub-tree, based on sub-tree statistics.
result The probability of selecting a suboptimal action converges to zero and decays logarithmically.
New method handles robust and adaptive control of linear systems with non-convex costs.
problem Robust and adaptive control of linear systems with unknown parameters.
method Combining non-asymptotic linear regression, interval prediction, and tree-based planning.
result First end-to-end suboptimality analysis for robust and adaptive MPC with non-convex costs.
Deep RL improves space heating control with faster computation and robustness.
problem Suboptimal performance and inability to adapt to dynamic conditions in classical heating control methods.
method Deep reinforcement learning algorithm for optimal control of space heating systems.
result Outperforms rule-based control by 5-10% in simulated environments.
A new measure helps compute suboptimality in entropy-regularized methods.
problem Computing suboptimality in entropy-regularized variational objectives when unnormalised densities are unavailable.
method Introduced 'kernel gradient discrepancy' (KGD) to compute suboptimality explicitly.
result KGD characterizes kernel Stein discrepancy (KSD) in the standard Bayesian context and measures variational gradient size.
MAPS and MAPS-SE learn from multiple suboptimal experts to improve policies efficiently.
problem Learning from multiple suboptimal experts in reinforcement learning.
method Active policy improvement from multiple black-box oracles.
result MAPS and MAPS-SE achieve sample efficiency and state-wise imitation learning.
Study online data poisoning attacks in sequential training data.
problem Manipulate sequential training data to influence online learning outcomes.
method Formulated as a stochastic optimal control problem, solved with model predictive control and deep reinforcement learning.
result Upper bounds the suboptimality of attacks due to unknown data generating distribution.
Proposes a curriculum learning algorithm to maximize cumulative return in reinforcement learning.
problem Maximizing cumulative return in reinforcement learning tasks.
method Task sequencing algorithm maximizing cumulative return, using curriculum learning to minimize suboptimal actions.
result Significantly better performance on cumulative return maximization compared to metaheuristic algorithms.
FinFlowRL learns from experts to optimize financial control in changing markets.
problem Traditional finance control methods fail in real-world, non-stationary markets.
method Imitation-Reinforcement Learning framework that pretrains on expert strategies and finetunes in noise space.
result Consistently outperforms individually optimized experts across diverse market conditions.
Improves model-based control and exploration by estimating model uncertainty.
problem Inaccuracies in model predictions lead to frequent re-planning, inefficiency, and unreliability.
method Estimates model uncertainty using reconstruction error and uses it for better control and active exploration.
result Improves control performance and exploration efficiency by choosing confident model predictions and planning for high uncertainty.
In this paper, we study the portfolio optimization problem with general utility functions and when the return and volatility of underlying asset are slowly varying. An asymptotic optimal strategy is provided within a specific class of admissible controls under this problem setup. Specifically, we first establish a rigo…
New method identifies flawed internal models of the world in animals.
problem How animals make decisions with partial sensory information.
method Generalizes Inverse Rational Control to continuous nonlinear dynamics and noise.
result Identifies the best internal model explaining an agent's actions.
Revisits VIC method to correct intrinsic reward bias in stochastic environments.
problem Intrinsic reward bias in VIC leading to suboptimal solutions.
method Proposes two methods based on transitional probability model and Gaussian mixture model to correct bias.
result Achieves maximal empowerment through corrected intrinsic reward.
This work ensures policy gradient methods converge to global optima for certain control problems.
problem Non-convex optimization challenges in policy gradient methods for complex control problems.
method Identifies structural properties ensuring non-convex objective functions have no suboptimal stationary points.
result Policy gradient methods converge to global optima under certain conditions, satisfying a Polyak-Lojasiewicz condition.
AutoRL automates reward tuning for RL, improving performance on complex tasks.
problem Suboptimal policies when using direct rewards in RL.
method Evolutionary search over RL agents to find optimal rewards.
result Improves RL performance on complex tasks.
A new RL method uses flows to mix policies for better exploration.
problem Learning in high-dimensional state spaces with deceptive rewards.
method Uses normalizing flows to attract and repel policies in a population.
result Empirically outperforms Soft-Actor Critic (SAC) on MuJoCo tasks.
Two entropy measures quantify suboptimal portfolio performance.
problem Measuring suboptimality in investment portfolios.
method Relative entropy (KL divergence) calculations.
result Suboptimal portfolios appear better than Kelly portfolios under certain measures.
PFPN uses particle filtering to improve character control in physics-based simulations.
problem Premature commitment to suboptimal actions in high-dimensional continuous control problems for articulated characters.
method Proposes a particle-based action policy using particle filtering to dynamically explore and discretize the action space.
result Demonstrates better imitation performance and robustness to external perturbations compared to Gaussian policies.
FineMorphs models smooth transformations for multivariate regression.
problem Efficiently modeling complex transformations for multivariate regression.
method Optimal control of affine and diffeomorphic transformations using smooth vector fields.
result FineMorphs can reduce dimensionality and adapt to large datasets.
Develops a learning model predictive controller for competitive racing.
problem Lack of exploration in state space and complexity in obstacle avoidance.
method Explores state space through multiple initializations and develops a new method for convex terminal set selection.
result Yields a richer terminal safe set and maintains convexity.
Safe linear bandits over unknown polytopes avoid safety violations and suboptimal actions.
problem Online approach to linear programming with unknown constraints and risks.
method Doubly-optimistic strategy (DOSS) for safe linear bandits over polytopes.
result DOSS achieves tight bounds on efficacy regret and safety violations.
Novel IRL method identifies suboptimal medical decisions in ICU data.
problem Identifying suboptimal medical decisions in clinical settings.
method Incorporates Inverse Reinforcement Learning with a pruning step to identify and remove suboptimal actions.
result Pruning step effectively identifies clinical priorities and values from suboptimal data.
We show LLMs can be locally linear, enabling better control of activations.
problem Suboptimal control of LLM activations during generation.
method Model LLM inference as a linear dynamical system, compute feedback controllers using Jacobians, and adapt classical control theory.
result Robust, fine-grained control of LLM activations across models and tasks.
Improved MPC with neural networks and active sets for large-scale problems.
problem Large-scale model predictive control with guarantees on feasibility and stability.
method Combines offline-trained neural network and online primal active set solver.
result Achieves 2x reduction in online inference time for large problems.
Optimal control strategy uses random noise to adaptively control systems with unknown parameters.
problem Online adaptive control of linear quadratic regulator with unknown system parameters.
method Certainty equivalent control with exploratory random noise, refined estimates of system matrices.
result Achieves optimal regret scaling as Θ(√(d_u^2 d_x T)) with self-bounding ODE method.
Novel framework finds globally optimal energy-efficient power control in wireless networks.
problem Energy-efficient power control in wireless networks.
method Branch-and-bound procedure with problem-specific bounds for faster convergence.
result Global solution for common energy-efficient power control problems with reduced complexity.
Improved text generation with constraints using discrete auto-regressive biasing.
problem Balancing fluency and constraint satisfaction in LLM outputs.
method Discrete Auto-regressive Biasing, leveraging gradients in discrete text space.
result Significantly improved constraint satisfaction with comparable fluency.
Novel framework learns policies for MPC with near-optimal guarantees.
problem Safe and near-optimal policy learning for MPC.
method Primal-dual neural networks for supervised learning of policies and certificate policies.
result Guaranteed feasibility and optimality of learned policies.
Algorithm improves continuous control with context, achieving provable efficiency.
problem Building an agent that generalizes to unseen environments in continuous control.
method Uses upper confidence bound (UCB) to refine decoder estimates and balance exploration-exploitation.
result Proves $\widetilde{O}\left(\sqrt{T}
ight)$ regret bound and ε ε ε -suboptimal policy transfer. Unified framework for risk-aware policy learning in contextual bandits.
problem Optimizing decision rules in high-stakes domains with adverse outcomes.
method Distributional framework for Lipschitz-continuous risk functionals, with novel empirical concentration inequalities.
result Data-dependent suboptimality bounds with an i l d e O ( 1 / n ) ilde{\mathcal{O}}(1/\sqrt{n}) i l d e O ( 1/ n ) rate, matching risk-neutral offline policy optimization. Robo-advisors use MPC to create dynamic investment strategies.
problem Static allocation methods limit robo-advisors' effectiveness.
method Combines MPC with Hidden Markov Model and Black-Litterman for dynamic asset allocation.
result MPC-based strategies outperform static approaches in dynamic and risk-budgeting criteria.
New suboptimal algorithm for best subset selection in high-dimensional data.
problem Nonconvex and computationally challenging best subset selection in linear regression.
method Introducing a new suboptimal algorithm and comparing it with other popular methods.
result The new procedure is a competitive suboptimal algorithm for high-dimensional data.
TabNAS improves neural architecture search for tabular datasets by rejecting suboptimal architectures.
problem Finding optimal neural architectures for tabular datasets with resource constraints.
method Develops a reinforcement learning controller motivated by rejection sampling to handle resource constraints.
result TabNAS finds better models that obey resource constraints compared to previous methods.
Optimizing dividend payments for an insurance company with bounded rates.
problem Maximizing expected exponential utility of discounted dividends under bounded dividend rates.
method Suboptimal strategies are evaluated using a new method to estimate the distance to the value function.
result The optimal strategy is of barrier type with a non-linear barrier.
MAVEN improves multi-agent exploration by hybridizing value and policy-based methods.
problem Exploration and suboptimality in complex multi-agent environments.
method MAVEN combines value and policy-based approaches with a latent space for hierarchical control.
result MAVEN achieves significant performance improvements on challenging multi-agent tasks.
We consider continuous-time stochastic optimal control problems featuring Conditional Value-at-Risk (CVaR) in the objective. The major difficulty in these problems arises from time-inconsistency, which prevents us from directly using dynamic programming. To resolve this challenge, we convert to an equivalent bilevel op…
The maximum a posteriori (MAP) configuration of binary variable models with submodular graph-structured energy functions can be found efficiently and exactly by graph cuts. Max-product belief propagation (MP) has been shown to be suboptimal on this class of energy functions by a canonical counterexample where MP conver…
New method optimizes memory usage in neural networks, improving sequential learning.
problem Current memory models in neural networks waste memory and computation.
method Formulated an optimization problem to maximize information storage, introduced Cached Uniform Writing.
result Proved Cached Uniform Writing optimizes memory usage and outperforms other methods.
Paper develops a framework for learning interpretable representations of sequential decision behavior.
problem Obtaining a transparent description of existing behavior.
method Inverse decision modeling framework, formalizing both forward and inverse problems.
result Learning interpretable representations of behavior, including suboptimal actions, biased beliefs, and imperfect knowledge.
Study shows how to learn optimal policies quickly in stochastic control problems.
problem Learning optimal policies in large, continuous state and action spaces with limited data.
method Analyzes three geometric exponents to quantify fast policy regret convergence.
result Shows that fast policy regret convergence is induced by specific geometric structures.
This paper explains CART random forests using stochastic control theory.
problem Understanding the inner workings of CART random forests.
method Developed a stochastic-control perspective on CART random forests, interpreting feature subsampling as a random feasible action set and the split rule as a policy.
result Established that the CART policy is locally stabilizing but globally suboptimal for the forest objective.
The Lasso is suboptimal in sparse linear regression due to design matrix constraints.
problem The suboptimality of the Lasso estimator in sparse linear regression.
method Characterization of optimal estimation rates and analysis of the Lasso estimator.
result The Lasso is provably minimax rate-suboptimal when the minimum singular value is small.
Paper explores limits of imitation learning in MDPs, setting new suboptimality bounds.
problem Understanding the statistical limits of imitation learning in MDPs.
method Analyzes minimax statistical limits in two settings: pre-interaction and interaction.
result Establishes suboptimality bounds for imitation learning in MDPs, showing improvements with knowledge of transition.
New bounds on adaptivity cost in stochastic optimization.
problem Understanding the cost of changing strategies in stochastic optimization.
method Proving impossibility results for adaptivity in non-smooth stochastic convex optimization.
result Lower bounds on the price of adaptivity for different levels of uncertainty.
New method infers intent from suboptimal behavior by modeling incorrect internal beliefs about dynamics.
problem Inferring intent from suboptimal human behavior using traditional methods assumes near-optimality, which is not always valid.
method Model suboptimal behavior as internal model misspecification, estimating incorrect beliefs about dynamics.
result Accurately models human intent by accounting for internal model inaccuracies.
PEGR improves deep learning models' robustness against noisy data.
problem Learning signals from noisy data in deep learning models.
method Per-example gradient regularization (PEGR) to suppress noise.
result PEGR enhances test error and robustness against noise perturbations.
PE-SVI reduces SVI inference complexity by finding a suitable start point.
problem Complex posterior inference in graphical models leads to suboptimal learning.
method PE-SVI uses a pseudo-encoded start point to reduce gradient steps and step sizes.
result PE-SVI achieves the same ELBo objective as SVI with less than 1% of the required steps.