arXiv research

New method improves deep RL by combining emphatic weightings with replay data.

Policy gradient methods are widely used for control in reinforcement learning, particularly for the continuous action setting. There have been a host of theoretically sound algorithms proposed for the on-policy setting, due to the existence of the policy gradient theorem which provides a simplified form for the gradien…

New algorithms improve reinforcement learning stability and performance.

Recently, \citet{SuttonMW15} introduced the emphatic temporal differences (ETD) algorithm for off-policy evaluation in Markov decision processes. In this short note, we show that the projected fixed-point equation that underlies ETD involves a contraction operator, with a $\sqrtγ$-contraction modulus (where $γ$ is the …

We propose a new objective, the counterfactual objective, unifying existing objectives for off-policy policy gradient algorithms in the continuing reinforcement learning (RL) setting. Compared to the commonly used excursion objective, which can be misleading about the performance of the target policy when deployed, our…

We consider the off-policy evaluation problem in Markov decision processes with function approximation. We propose a generalization of the recently introduced \emph{emphatic temporal differences} (ETD) algorithm \citep{SuttonMW15}, which encompasses the original ETD($λ$), as well as several other off-policy evaluation …

PER-ETD improves ETD by reducing variance to polynomial complexity.

We present the first provably convergent two-timescale off-policy actor-critic algorithm (COF-PAC) with function approximation. Key to COF-PAC is the introduction of a new critic, the emphasis critic, which is trained via Gradient Emphasis Learning (GEM), a novel combination of the key ideas of Gradient Temporal Differ…

Neural networks and linear systems linked, revealing training loss and kernel limitations.

Temporal difference learning and Residual Gradient methods are the most widely used temporal difference based learning algorithms; however, it has been shown that none of their objective functions is optimal w.r.t approximating the true value function $V$. Two novel algorithms are proposed to approximate the true value…

This paper extends off-policy reinforcement learning to the multi-agent case in which a set of networked agents communicating with their neighbors according to a time-varying graph collaboratively evaluates and improves a target policy while following a distinct behavior policy. To this end, the paper develops a multi-…

This paper studies users' perception regarding a controversial product, namely self-driving (autonomous) cars. To find people's opinion regarding this new technology, we used an annotated Twitter dataset, and extracted the topics in positive and negative tweets using an unsupervised, probabilistic model known as topic …

The evaluation of machine learning algorithms in biomedical fields for applications involving sequential data lacks standardization. Common quantitative scalar evaluation metrics such as sensitivity and specificity can often be misleading depending on the requirements of the application. Evaluation metrics must ultimat…

For a 3-manifold M, McMullen derived from the Alexander polynomial of M a norm on H^1(M, R) called the Alexander norm. He showed that the Thurston norm on H^1(M, R), which measures the complexity of a dual surface, is an upper bound for the Alexander norm. He asked if these two norms were equal on all of H^1(M,R) when …

New algorithms improve policy evaluation in reinforcement learning.

The problem of on-line off-policy evaluation (OPE) has been actively studied in the last decade due to its importance both as a stand-alone problem and as a module in a policy improvement scheme. However, most Temporal Difference (TD) based solutions ignore the discrepancy between the stationary distribution of the beh…

A new weighted MCC measure improves classifier performance evaluation.

Stability of weighted extremal manifolds proven through blowups.

Develops theory of weightings for Lie groupoids and algebroids.

The paper extends spin geometry to weighted manifolds and defines a new mass for Ricci flow.

Paper generalizes CR Obata theorem to weighted Sasakian manifolds.

The study explores weightings on submanifolds and their geometric properties.

New mass and staticity concepts derived from weighted curvature maps.

Proves existence and uniqueness of weighted metrics for smooth spaces.

Defines and proves properties of weighted renormalized volume coefficients.

A new weighted FDA method improves face recognition accuracy.

Proves positive mass theorem for non-spin weighted manifolds.

New invariants help solve existence of weighted cscK metrics.

Derives integral formulae on weighted manifolds.

Method measures weight similarity in neural networks using normalization and statistical inference.

The paper studies weighted Ricci curvatures and characterizes Randers metrics.

Study on stable minimal hypersurfaces under Ricci curvature constraints.

Explains weightings along submanifolds, focusing on Lie groupoids.

The study establishes comparison theorems for weighted Finsler manifolds and spacetimes.

The study analyzes weighted manifolds with curvature bounds, proving eigenvalue estimates and inequalities.

The paper generalizes K-stability results to singular and weighted settings.

Study on deformation of weighted scalar curvature, proving geometric results and stability.

Reverse-weighted portfolios outperform in commodity futures markets.

Paper extends trigonometric summation formula with weights.

The Penrose theorem and Hawking's topology theorem are extended to weighted spacetimes.

A new method trains deep networks by separating weight locations from values.

A new method to improve deep neural networks using weight rescaling.

The paper predicts edge weights in weighted directed networks using metric geometry.

Optimal weight windows are found by projecting the origin onto a convex polytope.

Optimizes weights for better model performance in shifting data.

Study of weighted nonlinear flags in symplectic geometry.

Due to a resource-constrained environment, network compression has become an important part of deep neural networks research. In this paper, we propose a new compression method, \textit{Inter-Layer Weight Prediction} (ILWP) and quantization method which quantize the predicted residuals between the weights in all convol…

Study on convergence rate of weighted Yamabe flow.