New insights into cascade feedback linearization of control systems.
problem Obtaining a cascade feedback linearization for invariant control systems.
method Introducing truncated versions of operators from the calculus of variations to prove new theorems.
result Established new geometry and foundational theorems for future work.
Test for linearizing 2-input systems with 2D feedback.
problem Linearizability of two-input systems by feedback.
method Algorithmic test for 2D endogenous feedback.
result Systematic derivation of flat outputs.
New algorithm reduces regret in delayed feedback generalised linear bandits.
problem Regret in delayed feedback generalised linear bandits.
method Adaptation of optimistic algorithm to delayed feedback.
result Achieves a regret bound independent of the horizon's delay penalty.
This paper explores a new form of the linear bandit problem in which the algorithm receives the usual stochastic rewards as well as stochastic feedback about which features are relevant to the rewards, the latter feedback being the novel aspect. The focus of this paper is the development of new theory and algorithms fo…
Study shows how to control jump-diffusion processes with stable feedback controls in reinforcement learning.
problem Control jump-diffusion processes with unknown coefficients in reinforcement learning.
method Lipschitz continuous optimal feedback controls, stability analysis of forward-backward SDEs, least-squares algorithm.
result Achieves O ( N ln N ) O(\sqrt{N\ln N}) O ( N ln N ) regret for linear-convex learning problems with jumps. New algorithm controls linear systems with bandit feedback, achieving optimal regret.
problem Controlling linear systems with bandit feedback under adversarial costs.
method Developed a new algorithm for linear control with memory optimization technique.
result Achieved optimal regret growth proportional to square root of time horizon.
Neural algorithms optimize arm selection with human preference feedback for complex reward functions.
problem Optimizing arm selection with noisy human preference feedback for complex, non-linear reward functions.
method Neural network to estimate reward function using preference feedback, upper confidence bound and Thompson sampling algorithms.
result Sub-linear regret guarantees for efficient arm selection in contextual dueling bandits.
Advocates a local feedback approach for RL in unknown systems.
problem Finding optimal feedback laws in unknown nonlinear dynamical systems.
method Searches over a local feedback representation consisting of an open-loop sequence and an optimal linear feedback law.
result Results in highly efficient training and superior performance compared to global methods.
A new dual test for forward-flatness simplifies computations.
problem Checking forward-flatness in discrete-time systems.
method A unique sequence of integrable codistributions.
result Computational efficiency and comparison with dynamic feedback linearization.
Paper studies CLO with partial feedback, improving decision-making in uncertain contexts.
problem Improving decision-making in contexts with uncertain cost coefficients using partial feedback.
method Unified class of offline learning algorithms for CLO with different types of feedback, using IERM framework.
result Fast-rate regret bound for IERM with partial feedback and misspecified model classes.
Easy conditions found for simplifying complex systems.
problem Linearizing complex two-input systems.
method Endogenous dynamic feedback with a dimension of at most two.
result Necessary and sufficient conditions for linearizability.
The paper studies symmetry reduction of control systems and its implications for feedback linearization.
problem Understanding symmetry reduction and its effects on feedback linearizability of control systems.
method Generalizing the notion of transversality of Lie group actions, analyzing the geometry of invariant distributions, and extending the S-G-S test.
result Classification of SFL quotients based on geometric properties of the control system and symmetry group.
A new algorithm for conversational recommendation systems using dueling bandits in GLMs.
problem Limited user feedback in existing conversational bandit methods.
method Integrates dueling bandits with relative feedback in generalized linear models.
result Theoretical and empirical validation of ConDuel's efficacy.
Unified LP framework for offline reward learning from human demonstrations and feedback.
problem Reward learning from human demonstrations and feedback with robustness and sample efficiency.
method A novel linear programming framework for offline reward learning.
result Unified LP framework achieves better performance compared to MLE.
Proposes a new theoretical framework for PbRL that requires less human feedback.
problem Lack of theoretical work capturing practical PbRL frameworks.
method Introduces a reward-agnostic PbRL framework that acquires exploratory trajectories before human feedback.
result Demonstrates improved sample complexity for learning optimal policies in linear and low-rank MDPs.
FA algorithm provides convergence guarantees for deep linear networks.
problem Training efficiency and convergence of deep neural networks.
method Theoretical analysis of Feedback Alignment (FA) algorithm for deep linear networks.
result Certain initializations lead to implicit anti-regularization, affecting learning effectiveness.
Study absolute equivalence for Pfaffian systems, applying to control systems.
problem Absolute equivalence of Pfaffian systems with specific independence conditions.
method Structural results for Pfaffian systems of corank 3, applied to control systems.
result Dynamic feedback linearization of control systems with 2 inputs.
The paper tackles exact linearization and control of flat discrete-time systems.
problem Exact linearization and control of flat nonlinear discrete-time systems.
method Investigates conditions for choosing new inputs and feedbacks that may depend on forward-shifts of the new input.
result Easily verifiable conditions for choosing a feasible input and a new input that minimizes forward-shifts of the flat output.
Banker-OMD improves online learning with delayed feedback.
problem Handling delayed feedback in online learning.
method Generalized Online Mirror Descent (OMD) framework.
result Achieves nearly-optimal performance in three bandit scenarios.
Neural ODEs control graph dynamics with low energy feedback.
problem Controlling complex dynamical systems on graphs.
method Neural Ordinary Differential Equation Control (NODEC) framework.
result NODEC learns low-energy control signals for graph dynamical systems.
The paper tackles combinatorial pure exploration with various feedback structures and proposes efficient algorithms.
problem Identifying the optimal action in a combinatorial space with limited feedback and nonlinear rewards.
method Designs polynomial-time adaptive algorithms for CPE-BL and CPE-PL, providing sample complexity analyses.
result The proposed algorithms achieve sample complexity close to lower bounds and outperform existing methods.
Geometric framework for dynamic feedback linearization of control systems with symmetry.
problem Dynamic feedback linearization of control systems with symmetry.
method Geometric framework based on Lie symmetry, systematic procedure for all smooth, generic system trajectories.
result Sufficient condition for dynamic feedback linearizability obtained.
Paper tackles combinatorial reinforcement learning with preference feedback.
problem Modeling long-term user engagement in scenarios like recommender systems and online advertising.
method Assumes a contextual MNL preference model with linear mean utilities and approximates item values. Proposes MNL-VQL algorithm.
result Achieves nearly minimax-optimal regret for linear MDPs with preference feedback.
A new algorithm for identifying the best arm in linear feedback with safety constraints.
problem Identifying the best arm in linear feedback with safety constraints.
method A gap-based algorithm that ensures safety while minimizing sample complexity.
result The algorithm achieves meaningful sample complexity while ensuring safety.
DART optimizes subset selection in non-linear bandit problems.
problem Optimizing subset selection in non-linear bandit problems with correlated rewards.
method DART algorithm for combinatorial bandits without individual arm feedback or linearity assumption.
result DART achieves a regret bound of i l d e O ( K K N T ) ilde{\mathcal{O}}(K\sqrt{KNT}) i l d e O ( K K N T ) . Two algorithms improve online reinforcement learning in adversarial linear MDPs with bandit feedback.
problem Online reinforcement learning in linear MDPs with adversarial losses and bandit feedback.
method Two algorithms: one computationally inefficient with $\widetilde{\mathcal{O}}\left(\sqrt{K}
ight)$ regret, and one computationally efficient with $\widetilde{\mathcal{O}}\left(K^{\frac{3}{4}}
ight)$ regret.
result Achieved improved regret performance compared to existing approaches.
Paper solves tracking control for ( x , u ) (x,u) ( x , u ) -flat systems using classical states.
problem Tracking control for ( x , u ) (x,u) ( x , u ) -flat systems. method Quasi-static feedback of classical states.
result Achieves linear, decoupled and asymptotically stable tracking error dynamics.
Study tests feasibility of linear programs with bandit feedback.
problem Testing feasibility of unknown linear programs with bandit feedback.
method Developed a novel test based on low-regret algorithms and a nonasymptotic law of iterated logarithms.
result Proved that the test is reliable and adapts to the signal level, with mean sample costs scaling as \( \widetilde{O}(d^2/Γ^2) \).
Active learning framework for optimizing human preferences in reinforcement learning.
problem Selecting most informative feedback for training models of human preferences.
method Proposes an active learning framework to collect preferential feedback online or offline.
result Errors in DPO logit estimates diminish with more feedback.
We apply the potential force estimation method to artificial time series of market price produced by a deterministic dealer model. We find that dealers' feedback of linear prediction of market price based on the latest mean price changes plays the central role in the market's potential force. When markets are dominated…
A predictor that is deployed in a live production system may perturb the features it uses to make predictions. Such a feedback loop can occur, for example, when a model that predicts a certain type of behavior ends up causing the behavior it predicts, thus creating a self-fulfilling prophecy. In this paper we analyze p…
Feedback improves ESN performance by 30-60% across various tasks.
problem Reducing computational complexity in ESNs for complex sequential data processing.
method State feedback to modify the internal reservoir state of ESNs.
result Feedback significantly improves ESN performance, reducing error by 30-60%.
Improved algorithms solve multi-period multi-class packing problems with bandit feedback.
problem Optimizing item packing under budget constraints with class-dependent rewards and bandit feedback.
method Developed a new estimator and a closed-form bandit policy for linear contextual multi-class multi-period packing problems.
result The proposed policy achieves sublinear regret in non-degenerate contexts, significantly outperforming benchmarks.
Study uses geometric algebra to analyze credit cycles, revealing dangerous feedback loops.
problem Understanding and predicting dangerous feedback loops in credit cycles.
method Represent economic states as multi-vectors in Clifford algebra, focusing on bivector elements for rotational coupling.
result Geometric relationship between unemployment and credit contraction shifts from simple correlation to dangerous rotational dynamics during crises.
We study an online decision making problem where on each round a learner chooses a list of items based on some side information, receives a scalar feedback value for each individual item, and a reward that is linearly related to this feedback. These problems, known as contextual semibandits, arise in crowdsourcing, rec…
Paper introduces a new framework for optimizing non-convex functions.
problem Optimizing non-convex functions, especially DR-submodular and concave functions.
method Developed a general meta-algorithm to convert linear/quadratic optimization to optimization of upper-linearizable/quadratizable functions.
result Unified approach to concave and DR-submodular optimization problems.
Stochastic linear bandits are a natural and well-studied model for structured exploration/exploitation problems and are widely used in applications such as online marketing and recommendation. One of the main challenges faced by practitioners hoping to apply existing algorithms is that usually the feedback is randomly …
New algorithm reduces reinforcement learning regret for linear MDPs with unknown transitions.
problem Adversarial linear mixture MDPs with bandit feedback and unknown transition.
method Proposes a new algorithm with a least square estimator and self-normalized concentration.
result Achieves improved regret bound with high probability.
New algorithm reduces risk in online games with limited feedback.
problem Risk-averse learning in repeated unknown games with bandit feedback.
method Proposes a momentum-based algorithm to estimate CVaR using historical cost values.
result Achieves sub-linear regret and outperforms existing methods in numerical experiments.
New method tackles online DR-submodular maximization with improved regret guarantees.
problem Online maximization of non-monotone DR-submodular functions over down-closed convex sets.
method 1/e-linearization through exponential reparametrization, surrogate potential, and reduction to online linear optimization.
result Achieves O ( T 1 / 2 ) O(T^{1/2}) O ( T 1/2 ) static regret with single gradient query per round, improving state of the art. We study online linear regression problems in a distributed setting, where the data is spread over a network. In each round, each network node proposes a linear predictor, with the objective of fitting the \emph{network-wide} data. It then updates its predictor for the next round according to the received local feedbac…
Empirical data reveals that the liquidity flow into the order book (depositions, cancellations andmarket orders) is influenced by past price changes. In particular, we show that liquidity tends todecrease with the amplitude of past volatility and price trends. Such a feedback mechanism inturn increases the volatility, …
New RL algorithm handles delayed feedback with posterior sampling.
problem Challenges of delayed feedback in reinforcement learning with linear function approximation.
method Posterior sampling with delayed feedback for value-based RL.
result Achieves optimal regret guarantee with improved computational efficiency.
Algorithm learns fair division from noisy feedback in uncertain markets.
problem Learning fair division in uncertain markets with noisy feedback.
method Wrapper algorithms using dual averaging to learn item and agent values from bandit feedback.
result Asymptotically achieves optimal Nash social welfare in linear Fisher markets.
New RL algorithm optimizes policies with bandit feedback, matching previous bounds.
problem Optimizing policies with unknown transitions and bandit feedback.
method Optimistic Trust Region Policy Optimization (TRPO) algorithm.
result Sub-linear regret bounds for both stochastic and adversarial rewards.
Master-slave architecture tackles combinatorial multi-armed bandits with diversity constraints.
problem Solving top- K K K combinatorial multi-armed bandits with non-linear feedback and diversity constraints. method Master-slave architecture with six slave models, teacher learning, and policy co-training.
result Significantly outperforms existing algorithms in synthetic and real datasets.
Algometrics analyzes how predictive models affect their own forecasts in algorithmic markets.
problem How predictive models affect their own forecasts in algorithmic markets.
method Introduces algometrics, a framework for time series with feedback, proving three results on deployment risk.
result Deployment risk cannot be identified from passive historical data alone, and historical rankings can invert under crowding.
The paper tackles fair sequential decision making with biased linear bandit feedback.
problem Fair sequential decision making with biased linear bandit feedback.
method Phased elimination algorithm to correct unfair evaluations, establishing upper bounds on regret.
result The worst-case regret is smaller than O ( κ ∗ 1 / 3 log ( T ) 1 / 3 T 2 / 3 ) \mathcal{O}(κ_*^{1/3}\log(T)^{1/3}T^{2/3}) O ( κ ∗ 1/3 log ( T ) 1/3 T 2/3 ) .