Optimal policies for MAB with cost constraint developed.
problem Maximizing outcomes in multi-armed bandit with cost constraint.
method Developed asymptotically optimal policies for MAB under cost constraint.
result Constructed feasible uniformly fast (f-UF) convergent policies achieving asymptotic lower bound on regret.
Optimizes resource allocation in MAB models with resource constraints.
problem Optimizing resource allocation under limited and replenishing resources.
method Derives asymptotic lower bounds and constructs optimal policies.
result Achieves optimal regret for feasible uniformly fast policies.
New algorithm exploits curvature of feasible sets for fast online convex optimization.
problem Online convex optimization with fast rates.
method Adapting FTL algorithm to curvature of feasible sets.
result Achieves logarithmic regret bound of O ( ρ log T ) O(ρ\log T) O ( ρ log T ) in stochastic environments. PS framework selects best policy from library for CSO problems.
problem Policy selection in CSO with heterogeneous performance across covariate space.
method PS framework constructs library of candidate policies and learns a meta-policy to select the best one.
result PS consistently outperforms best single policy in heterogeneous CSO problems.
The paper analyzes reinforcement learning methods for estimating weights and quality functions with fast convergence rates.
problem Estimating weights and quality functions in reinforcement learning with function approximation.
method The paper uses minimax methods for estimating marginal importance weights and q-functions.
result The minimax approach enables fast rates of convergence for weights and quality functions, achieving first-order efficiency.
Reward-poisoning attacks can force RL agents to learn bad policies, and we categorize and quantify their feasibility.
problem Reward-poisoning attacks can manipulate RL agents to learn undesirable policies.
method Categorize attacks by infinity-norm constraint, provide thresholds for feasibility, and develop adaptive attack strategies.
result Adaptive reward-poisoning attacks can achieve the nefarious policy in polynomial steps, while non-adaptive attacks require exponential steps.
Estimates policy performance in small-data settings without sacrificing data.
problem Poor performance of cross-validation in small-data optimization.
method Uses sensitivity analysis to estimate gradient of optimal objective value.
result Explicit high-probability bounds on error of estimator for small-data, large-scale problems.
Efficient online control algorithms for noisy systems with quadratic losses.
problem Controlling linear systems with known dynamics and adversarial losses.
method Novel SDP relaxation for steady-state distribution, ensuring strongly stable policies.
result Guaranteed O ( T ) O(\sqrt{T}) O ( T ) regret for online learning algorithms. Efficiently identifies good policies by choosing contexts for human feedback.
problem Efficiently identifying good policies in applications with high feedback costs.
method Introduces offline contextual dueling bandit setting and an upper-confidence-bound style algorithm.
result Proves a regret bound and shows superior performance over uniformly sampled contexts.
New Q-learning algorithm reduces sample complexity for large discount factors.
problem Large discount factors make Q-learning algorithms inefficient.
method Introduces a new Q-learning algorithm with uniformly bounded sample complexity.
result The new algorithm achieves asymptotic covariance that is a quadratic in 1 / ( 1 − ρ ∗ γ ) 1/(1- ρ^* γ) 1/ ( 1 − ρ ∗ γ ) . Safe RL for autonomous vehicles using PCPO with trust regions and parallel learners.
problem Unexplainable behaviours and lack of safety guarantees in RL for real vehicles.
method PCPO framework with trust regions and parallel learners.
result Safe learning confirmed for autonomous vehicles with fast convergence.
New method uses entropy to improve policy gradient exploration.
problem Limited exploration in policy gradient methods.
method Entropy regularization with discounted future state distribution.
result Proves convergence to locally optimal policy.
A controller optimizes sampling from unknown distributions to maximize a score function.
problem Optimizing sampling from unknown distributions to maximize a score function.
method Uniformly Fast (UF) sampling policies and UCB policy.
result UCB policy is asymptotically optimal and achieves the lower bound for sub-optimal activations.
Adaptive vehicle trajectory prediction for safer autonomous driving.
problem Inability of current methods to guarantee physical feasibility and adapt to human driving policies.
method Bayesian recurrent neural network combining policy and physical models, with gradient-based training and parameter adaptation.
result The proposed method ensures physical feasibility and adaptability to human driving policies.
New method improves solving combinatorial optimization problems with smoothed policies.
problem Solving combinatorial optimization problems repeatedly with varying instances.
method Smoothed policies with controlled random perturbations to linear oracle, leading to differentiable surrogate risk.
result Generalization bound decomposes excess risk into bias, estimation, and optimization components.
New bounds for learning near-optimal policies in CMDPs with constraints.
problem Optimizing policies in CMDPs with constraints.
method Model-based algorithm addressing relaxed and strict feasibility.
result Near-optimal sample complexity bounds for CMDPs.
Efficiently audits model fairness with continuous monitoring and flexible data collection.
problem Continuous monitoring and flexible data collection for fairness auditing.
method Sequential, anytime-valid inference and game-theoretic statistics.
result Demonstrated efficacy on three fairness datasets.
New algorithm reduces sample complexity for constrained MDPs.
problem Learning policies in constrained average-reward MDPs.
method Model-based algorithm for relaxed and strict feasibility settings.
result Achieves minimax-optimal bounds for constrained MDPs.
DNN policies improve stochastic AC OPF for power grid optimization.
problem Optimizing power grid operations under uncertainty.
method Deep neural network (DNN) policies for real-time generator dispatch decisions.
result DNN policies enforce feasibility constraints and produce near optimal solutions.
New method for rigorous confidence intervals in off-policy evaluation.
problem Evaluate new policies from off-policy data without executing them.
method Variational framework using kernel Bellman loss.
result Efficient method for tight confidence intervals in various settings.
We consider fast kernel summations in high dimensions: given a large set of points in d d d dimensions (with d ≫ 3 d \gg 3 d ≫ 3 ) and a pair-potential function (the {\em kernel} function), we compute a weighted sum of all pairwise kernel interactions for each point in the set. Direct summation is equivalent to a (dense) matrix-vec…
New algorithm reduces sample complexity for safe reinforcement learning.
problem Safe reinforcement learning in constrained MDPs with performance and safety constraints.
method Model-based primal-dual algorithm balancing regret and bounded constraint violations.
result Proves near-optimal policies with bounded violations or zero violations in CMDPs.
We characterize value functions in partially observable MDPs as semi-algebraic sets.
problem Understanding feasible value functions in partially observable Markov decision processes.
method Characterization of feasible value functions as semi-algebraic sets defined by polynomial inequalities.
result The feasible set of value functions in POMDPs is a semi-algebraic set, not a polytope as in MDPs.
New methods for efficient exploration under unknown linear constraints in bandits.
problem Optimizing decisions under unknown linear constraints in bandit problems.
method Lagrangian relaxation, computationally efficient extensions of existing methods, constraint-adaptive stopping rule.
result LAGEX achieves asymptotically optimal sample complexity, LATS shows asymptotic optimality up to novel constants.
Paper introduces CageBO for optimizing complex public policy problems.
problem Complex decision-making and implicit constraints in public policy.
method CageBO framework using conditional variational autoencoder.
result CageBO outperforms baselines in optimizing large-scale police redistricting.
A scalable framework optimizes multi-asset portfolios with constraints.
problem Optimizing multi-asset portfolios with inequality constraints.
method Integrates neural policies with Pontryagin's Maximum Principle, enforcing feasibility via log-barrier regularization.
result Recover KKT-optimal policies in high-dimensional problems without violating constraints.
Paper proposes a new approach for agents to explore environments efficiently.
problem Learning policies that explore uniformly and mix quickly in environments without external rewards.
method Introduces a surrogate objective to maximize entropy and develops a model-based reinforcement learning algorithm, IDE 3 ^{3} 3 AL. result Demonstrates improved exploration and mixing in hard-exploration tasks.
AAPI improves regret bound for undiscounted continuing learning in uniformly ergodic MDPs.
problem Improving regret bounds for undiscounted continuing learning in uniformly ergodic MDPs.
method Adaptive approximate policy iteration (AAPI) with online learning techniques and data-dependent adaptive learning rate.
result AAPI achieves a i l d e O ( T 2 / 3 ) ilde{O}(T^{2/3}) i l d e O ( T 2/3 ) regret bound, improving over the best existing bound of i l d e O ( T 3 / 4 ) ilde{O}(T^{3/4}) i l d e O ( T 3/4 ) . Paper tackles efficient off-policy evaluation in long-horizon settings.
problem Efficient off-policy evaluation in long-horizon settings with diminishing overlap.
method Derives efficiency bounds for OPE under Markovian and time-invariant structures, develops a new DRL estimator.
result DRL estimator provides efficient OPE even with just one dependent trajectory in time-invariant Markov decision processes.
Let ( X , P ) (X, P) ( X , P ) be a toric variety. In this note, we show that the C 0 C^0 C 0 -norm of the Calabi flow φ ( t ) \varphi(t) φ ( t ) on X X X is uniformly bounded in [ 0 , T ) [0, T) [ 0 , T ) if the Sobolev constant of φ ( t ) \varphi(t) φ ( t ) is uniformly bounded in [ 0 , T ) [0, T) [ 0 , T ) . We also show that if ( X , P ) (X, P) ( X , P ) is uniform K K K -stable, then the modified Calabi flow converges expone…
New algorithm reduces best-in-class regret in contextual bandits.
problem Compete with the best policy in a class without model restrictions.
method Proposes an algorithm that updates policies by minimizing a pessimistic objective, including a clipped inverse-propensity estimate and variance penalty.
result Achieves fast best-in-class regret rates, including polylogarithmic rates in the parametric case.
New policy minimizes cost in dynamic inventory management with censored data.
problem Minimizing cumulative cost in inventory management with censored demand data.
method Developed a randomized policy, Exponentially Weighted Forecaster, with a cost estimator.
result Achieved optimal regret scaling with respect to key primitives.
SFPO optimizes LLM reasoning by repositioning before updating, improving stability and efficiency.
problem Noisy gradients from low-quality rollouts cause instability and inefficient exploration in on-policy RL algorithms.
method Decomposes each step into three stages: a short fast trajectory, repositioning, and slow correction, preserving the objective and rollout process unchanged.
result SFPO consistently improves stability, reduces rollouts, and accelerates convergence, outperforming GRPO on math reasoning benchmarks.
Paper develops a fast method to find near-optimal power solutions.
problem Solving AC OPF on fast timescales for large networks.
method Leverages machine learning to map system loading to optimal generation values.
result Near-optimal and feasible solutions found on milliseconds timescales.
SnareNet adds repair layers to neural networks to ensure outputs meet physical constraints.
problem Unconstrained neural network predictions violate physical or safety requirements.
method SnareNet appends a differentiable repair layer that navigates constraints to produce feasible outputs.
result SnareNet consistently improves objective quality while satisfying constraints more reliably.
RPO uses past and future state-action info for better policy optimization.
problem Sample inefficiency in on-policy reinforcement learning methods.
method Reflective Policy Optimization (RPO) integrates past and future state-action info for policy improvement.
result RPO improves policy performance and contracts the solution space, leading to faster convergence.
Watermarks DRL policies with minimal performance impact.
problem Detecting and managing unauthorized use of proprietary DRL policies.
method Integrates a unique identifier into DRL policies' responses to a sequence of state transitions.
result Demonstrates feasibility with DQN policy in Cartpole environment.
Solves online 3D bin packing with deep reinforcement learning under constraints.
problem Challenges of packing items immediately without information and constraints.
method Constrained deep reinforcement learning (DRL) with feasibility predictor.
result Significantly outperforms state-of-the-art methods in online 3D bin packing.
New algorithm solves constrained MDPs with generative model, reducing sample complexity.
problem Maximizing rewards while adhering to constraints in MDPs.
method Primal-dual framework using mirror descent value iteration.
result Near-optimal sample complexity bounds for CMDPs.
Calabi flow works well with bounded curvature on compact manifolds.
problem Stability of extremal Kähler metrics under the Calabi flow.
method Extending the Calabi flow with L p L^p L p scalar curvature bounds. result Calabi flow converges exponentially to extremal Kähler metrics.
Designs efficient factorial experiments for product design under budget constraints.
problem Designing effective experiments for product design with limited traffic and overlapping experiments.
method Two-stage design: first stage samples and infers performance, second stage selects a final policy.
result The method outperforms one-shot tensor completion and unstructured best-arm benchmarks.
Study shows how to learn optimal policies quickly in stochastic control problems.
problem Learning optimal policies in large, continuous state and action spaces with limited data.
method Analyzes three geometric exponents to quantify fast policy regret convergence.
result Shows that fast policy regret convergence is induced by specific geometric structures.
Community detection, which aims to cluster N N N nodes in a given graph into r r r distinct groups based on the observed undirected edges, is an important problem in network data analysis. In this paper, the popular stochastic block model (SBM) is extended to the generalized stochastic block model (GSBM) that allows for ad…
The paper tackles identifying Pareto Set with constraints using bandit feedback.
problem Identifying the Pareto Set under feasibility constraints in a multivariate bandit setting.
method Fixed-confidence identification algorithm that outperforms existing methods.
result The sample complexity of the proposed algorithm is near-optimal.
Statistical model checking for PCTL on MDPs using reinforcement learning.
problem Model checking PCTL specifications on MDPs with statistical methods.
method Reinforcement learning for policy search, statistical model checking with UCB-based Q-learning.
result Provably guaranteed statistical model checking method for PCTL specifications on MDPs.
Paper tackles efficient policy gradient estimation from off-policy data.
problem Estimating policy gradients from off-policy data is challenging and inefficient.
method Derives asymptotic lower bounds, proposes a meta-algorithm with 3-way robustness, and establishes convergence guarantees.
result Meta-algorithm achieves the lower bound on mean-squared error without parametric assumptions.
Dual Policy Iteration combines fast and slow policies for better reinforcement learning performance.
problem Improving reinforcement learning algorithms for practical applications.
method Alternates between a fast, reactive policy and a slow, non-reactive policy, optimizing both under each other's supervision.
result Demonstrates improved performance on various continuous control Markov Decision Processes.
In each dimension N ≥ 3 N\geq 3 N ≥ 3 and for each real number λ ≥ 1 λ\geq 1 λ ≥ 1 , we construct a family of complete rotationally symmetric solutions to Ricci flow on R N \mathbb{R}^{N} R N which encounter a global singularity at a finite time T T T . The singularity forms arbitrarily slowly with the curvature blowing up arbitrarily fast at the r…