Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

3416811,0221,362 · Jun 202019922001200920172026
48 results for general advantage estimator

Deep neural networks outperform other methods in estimating non-smooth functions with singularities.

problem Estimating functions with singularities on hypersurfaces.
method Developed a minimax rate analysis for DNNs, proving their almost optimal convergence rate.
result DNNs outperform other estimators in estimating non-smooth functions with singularities.

Paper introduces DTAE to optimize RL algorithms, balancing exploration and exploitation.

problem Balancing exploration and exploitation in reinforcement learning.
method Soft policy optimization with entropy and dual-track advantage estimator (DTAE).
result DTAE accelerates RL algorithm convergence and improves performance.

Bayesian estimators for causal inference using hierarchical Gaussian Processes.

problem Estimating causal effects in sharp and fuzzy RD/RK designs.
method Hierarchical Gaussian Process models for regression and classification.
result Hierarchical GP models improve precision and coverage of RD/RK estimations.

Proposes a new simulator for complex arrival processes.

problem Modeling and simulating complex arrival processes with non-stationary and multi-dimensional rates.
method Integrates Monte Carlo and GANs to model a broad class of arrival processes.
result Consistent and efficient estimation of the simulator using Wasserstein distance.

The estimation of advantage is crucial for a number of reinforcement learning algorithms, as it directly influences the choices of future paths. In this work, we propose a family of estimates based on the order statistics over the path ensemble, which allows one to flexibly drive the learning process, towards or agains…

2019-09-15abs ↗pdf ↗

This paper improves Q-learning bounds using reference-advantage decomposition.

problem Improving Q-learning bounds in MDPs with positive suboptimality gaps.
method Develops a novel error decomposition framework to prove gap-dependent regret bounds.
result Establishes logarithmic gap-dependent regret bounds for Q-learning.

Infinite horizon off-policy policy evaluation is a highly challenging task due to the excessively large variance of typical importance sampling (IS) estimators. Recently, Liu et al. (2018a) proposed an approach that significantly reduces the variance of infinite-horizon off-policy evaluation by estimating the stationar…

2019-10-16abs ↗pdf ↗

TROLL improves RL for LLMs by replacing clipping with a trust region projection.

problem Clipping in RL for LLMs causes instability and suboptimal performance.
method TROLL uses a discrete differentiable trust region projection to replace clipping, balancing computational cost and effectiveness.
result TROLL consistently outperforms PPO-like clipping in training speed, stability, and final success rates.

New proof for stability estimates in complex equations without pluripotential theory.

problem Stability estimates for complex Monge-Ampère and Hessian equations.
method New proof using general degenerations of background metrics.
result Uniform stability estimates for both equations under various degenerations.

In traditional reinforcement learning, an agent maximizes the reward collected during its interaction with the environment by approximating the optimal policy through the estimation of value functions. Typically, given a state s and action a, the corresponding value is the expected discounted sum of rewards. The optima…

2018-06-10abs ↗pdf ↗

Entrocraft addresses RL performance saturation in LLMs by customizing entropy curves.

problem Performance saturation in RL algorithms for LLMs.
method Entrocraft uses rejection sampling to bias advantage distributions for customized entropy schedules.
result Entrocraft significantly improves generalization, output diversity, and long-term training in 4B models.

The authors propose a parametric model called the arena model for prediction in paired competitions, i.e. paired comparisons with eliminations and bifurcations. The arena model has a number of appealing advantages. First, it predicts the results of competitions without rating many individuals. Second, it takes full adv…

2018-11-25abs ↗pdf ↗

GANICE improves GAN-based causal inference by minimizing averaged Wasserstein risk.

problem Estimating interventional outcome distributions and quantiles in causal inference.
method GANICE uses extended Wasserstein distance and a cellwise critic to minimize averaged Wasserstein risk.
result GANICE achieves minimax optimality and consistently outperforms existing methods.

We present a method to derive local estimates for some classes of fully nonlinear elliptic equations. The advantage of our method is that we derive Hessian estimates directly from C0C^0 estimates. Also, the method is flexible and can be applied to a large class of equations.

2005-10-29abs ↗pdf ↗

We develop and implement a novel fast bootstrap for dependent data. Our scheme is based on the i.i.d. resampling of the smoothed moment indicators. We characterize the class of parametric and semi-parametric estimation problems for which the method is valid. We show the asymptotic refinements of the proposed procedure,…

2020-01-14abs ↗pdf ↗

DRL agents perform poorly at high decision frequencies, but a new algorithm improves performance.

problem DRL agents struggle at high decision frequencies, leading to poor performance.
method Proved that DRL agents' action-conditioned return distributions collapse to their policy's return distribution as decision frequency increases. Defined superiority as a probabilistic generalization of advantage for high-frequency value-based RL.
result Proper modeling of superiority distribution improves performance of controllers at high decision frequencies.

After presenting Actor Critic Methods (ACM), we show ACM are control variate estimators. Using the projection theorem, we prove that the Q and Advantage Actor Critic (A2C) methods are optimal in the sense of the L2L^2 norm for the control variate estimators spanned by functions conditioned by the current state and acti…

2019-07-23abs ↗pdf ↗

Novel method learns memory kernels in Langevin equations.

problem Estimating memory kernels in Langevin equations.
method Regularized Prony method for correlation functions, followed by regression over Sobolev norm-based loss function with RKHS regularization.
result Method outperforms other regression estimators in exponentially weighted L^2 space.

Recent policy optimization approaches have achieved substantial empirical success by constructing surrogate optimization objectives. The Approximate Policy Iteration objective (Schulman et al., 2015a; Kakade and Langford, 2002) has become a standard optimization target for reinforcement learning problems. Using this ob…

2019-10-09abs ↗pdf ↗

Improved GANs estimate convergence rate for density estimation.

problem Improving the accuracy of density estimation with GANs.
method Proved an oracle inequality for JS divergence between GAN estimate and true density.
result JS-divergence rate of convergence is (logn/n)2β/(2β+d)(\log{n}/n)^{2β/(2β+ d)}.

Policy optimization on high-dimensional continuous control tasks exhibits its difficulty caused by the large variance of the policy gradient estimators. We present the action subspace dependent gradient (ASDG) estimator which incorporates the Rao-Blackwell theorem (RB) and Control Variates (CV) into a unified framework…

2018-05-09abs ↗pdf ↗

Quantum machine learning offers advantages for broader learning tasks.

problem Demonstrate QML advantage over classical methods for general learning tasks.
method Construct a new family of supervised learning tasks and prove their hardness.
result Prove provable advantage of QML based on general quantum computational advantages.

High-dimensional sparse modeling via regularization provides a powerful tool for analyzing large-scale data sets and obtaining meaningful, interpretable models. The use of nonconvex penalty functions shows advantage in selecting important features in high dimensions, but the global optimality of such methods still dema…

2016-05-11abs ↗pdf ↗

A new method for optimizing functions without gradients, improving efficiency and convergence.

problem Optimizing functions without gradient information in machine learning.
method Hybrid Gradient Descent (HGE) using random and coordinate-wise gradient estimates.
result The proposed method achieves optimal convergence rates in convex cases and generalizes to non-convex cases.

Blind Source Separation is a widely used technique to analyze multichannel data. In many real-world applications, its results can be significantly hampered by the presence of unknown outliers. In this paper, a novel algorithm coined rGMCA (robust Generalized Morphological Component Analysis) is introduced to retrieve s…

2015-07-08abs ↗pdf ↗

The multivariate version of the Mixed Tempered Stable is proposed. It is a generalization of the Normal Variance Mean Mixtures. Characteristics of this new distribution and its capacity in fitting tails and capturing dependence structure between components are investigated. We discuss a random number generating procedu…

2016-09-04abs ↗pdf ↗

In this paper, we investigate a divide and conquer approach to Kernel Ridge Regression (KRR). Given n samples, the division step involves separating the points based on some underlying disjoint partition of the input space (possibly via clustering), and then computing a KRR estimate for each partition. The conquering s…

2016-08-05abs ↗pdf ↗

We present a Dirichlet process mixture model over discrete incomplete rankings and study two Gibbs sampling inference techniques for estimating posterior clusterings. The first approach uses a slice sampling subcomponent for estimating cluster parameters. The second approach marginalizes out several cluster parameters …

2012-03-15abs ↗pdf ↗

New algorithm estimates treatment effects for more efficient contextual bandits.

problem Contextual bandits struggle with action-independent reward redundancies.
method Reduces contextual bandits to heterogeneous treatment effect estimation.
result Heterogeneous treatment effect estimation leads to more efficient model estimation.

New VAE model improves data fitting without sacrificing computational efficiency.

problem Limitation of Gaussian assumption in VAE for continuous variable fitting.
method Infinite mixture of asymmetric Laplace distribution in decoder, nonparametric M-estimator for quantile estimation.
result Model demonstrates superior data privacy adjustment and better distribution fitting.