arXiv research

Paper tackles stochastic $k$-submodular bandits with full feedback, achieving sublinear regret.

Study optimizes online learning for vector-valued data regression.

In this paper, we study fundamental problems of maximizing DR-submodular continuous functions that have real-world applications in the domain of machine learning, economics, operations research and communication systems. It captures a subclass of non-convex optimization that provides both theoretical and practical guar…

Paper introduces efficient online sub-sampling for RL with function approximation, reducing policy updates.

New method achieves small-loss regret bounds in random-order model.

A new method for online VI in SSMs using asymptotic contrast.

In this paper we introduce a novel online time series forecasting model we refer to as the pM-GP filter. We show that our model is equivalent to Gaussian process regression, with the advantage that both online forecasting and online learning of the hyper-parameters have a constant (rather than cubic) time complexity an…

Polynomial-time RL algorithm for constant actions under linear Bellman completeness.

New algorithm achieves logarithmic regret for adversarial online control.

Transforms offline algorithms to online with low regret in random order model.

Online random forests improve Q-learning performance in specific gym environments.

Algorithm finds Nash equilibria in complex games with function approximation.

In this paper, we consider an online optimization process, where the objective functions are not convex (nor concave) but instead belong to a broad class of continuous submodular functions. We first propose a variant of the Frank-Wolfe algorithm that has access to the full gradient of the objective functions. We show t…

Maximize to Explore integrates RL components for efficient policy discovery.

New insights on eluder dimension for function approximation in machine learning.

New coverage conditions improve sample efficiency in online reinforcement learning.

Paper optimizes GAIL for online and offline learning with linear approximations.

Algorithm for online learning in MDPs with linear function approximation and bandit feedback.

In this paper, we provide two new stable online algorithms for the problem of prediction in reinforcement learning, \emph{i.e.}, estimating the value function of a model-free Markov reward process using the linear function approximation architecture and with memory and computation costs scaling quadratically in the siz…

We present a reduction from reinforcement learning (RL) to no-regret online learning based on the saddle-point formulation of RL, by which "any" online algorithm with sublinear regret can generate policies with provable performance guarantees. This new perspective decouples the RL problem into two parts: regret minimiz…

Paper learns Koopman operator from sparse data, escaping function space constraints.

New framework for resilient bi-criteria optimization under noisy feedback.

Many machine learning frameworks, such as resource-allocating networks, kernel-based methods, Gaussian processes, and radial-basis-function networks, require a sparsification scheme in order to address the online learning paradigm. For this purpose, several online sparsification criteria have been proposed to restrict …

The paper provides tight bounds for improving multi-armed bandits problem.

Paper analyzes online reinforcement learning with outcome-based feedback, providing efficient algorithms and fundamental limits.

Improved reinforcement learning algorithm with linear approximation for unknown dynamics.

We propose a plan online and learn offline (POLO) framework for the setting where an agent, with an internal model, needs to continually act and learn in the world. Our work builds on the synergistic relationship between local model-based control, global value function learning, and exploration. We study how local traj…

Hierarchical clustering is a widely used approach for clustering datasets at multiple levels of granularity. Despite its popularity, existing algorithms such as hierarchical agglomerative clustering (HAC) are limited to the offline setting, and thus require the entire dataset to be available. This prohibits their use o…

Study on maximizing submodular functions with limited updates, achieving tight bounds and poly-time algorithms.

Transforms offline greedy algorithms to online algorithms for combinatorial problems.

We present a new online boosting algorithm for adapting the weights of a boosted classifier, which yields a closer approximation to Freund and Schapire's AdaBoost algorithm than previous online boosting algorithms. We also contribute a new way of deriving the online algorithm that ties together previous online boosting…

Stochastic approximation (SA) is a key method used in statistical learning. Recently, its non-asymptotic convergence analysis has been considered in many papers. However, most of the prior analyses are made under restrictive assumptions such as unbiased gradient estimates and convex objective function, which significan…

Reinforcement learning (RL) has had many successes in both "deep" and "shallow" settings. In both cases, significant hyperparameter tuning is often required to achieve good performance. Furthermore, when nonlinear function approximation is used, non-stationarity in the state representation can lead to learning instabil…

AAPI improves regret bound for undiscounted continuing learning in uniformly ergodic MDPs.

New RL approach uses resets to learn complex MDPs efficiently.

New method uses online learning to improve AIPW estimators for adaptively collected data.

New online learning algorithm combines PA and TER for binary classification.

Kernel-based methods enjoy powerful generalization capabilities in handling a variety of learning tasks. When such methods are provided with sufficient training data, broadly-applicable classes of nonlinear functions can be approximated with desired accuracy. Nevertheless, inherent to the nonparametric nature of kernel…

In this paper, we consider the problem of online learning of Markov decision processes (MDPs) with very large state spaces. Under the assumptions of realizable function approximation and low Bellman ranks, we develop an online learning algorithm that learns the optimal value function while at the same time achieving ve…

We uncover a fairly general principle in online learning: If regret can be (approximately) expressed as a function of certain "sufficient statistics" for the data sequence, then there exists a special Burkholder function that 1) can be used algorithmically to achieve the regret bound and 2) only depends on these suffic…

Paper proposes OPF policy for fair resource allocation with sublinear regret.

Study online learning of neural networks with margin condition.

Uniform TD(0) bound derived for function approximation with Markov noise.

New algorithm fills gaps in offline data for hybrid RL, achieving similar gains without coverage assumptions.

Framework optimizes battery storage for markets by separating long-term degradation from short-term market dynamics.

Regularized online learning is widely used in machine learning applications. In online learning, performing exact minimization ($i.e.,$ implicit update) is known to be beneficial to the numerical stability and structure of solution. In this paper we study a class of regularized online algorithms without linearizing the…

SnAp approximates RTRL for online training of sparse recurrent networks.

Online Matrix Factorization (OMF) is a fundamental tool for dictionary learning problems, giving an approximate representation of complex data sets in terms of a reduced number of extracted features. Convergence guarantees for most of the OMF algorithms in the literature assume independence between data matrices, and t…