arXiv research

Rate GENERIC extends thermodynamics principles to non-equilibrium systems.

Entropy rate of sequential data-streams naturally quantifies the complexity of the generative process. Thus entropy rate fluctuations could be used as a tool to recognize dynamical perturbations in signal sources, and could potentially be carried out without explicit background noise characterization. However, state of…

Researchers calculate Shannon entropy rates of hidden Markov processes efficiently.

New methods for inferring, predicting, and estimating continuous-time, discrete-event processes.

Sharp bounds on neural network approximation rates and widths.

The aim of this paper is to state and prove polynomial analogues of the classical Manning inequality relating the topological entropy of a geodesic flow with the growth rate of the volume of balls in the universal covering. To this aim we use two numerical conjugacy invariants, the {\em strong polynomial entropy $h_{po…

The paper examines how randomness in forex returns increases during financial crises.

We show that the entropy of a hyperbolic group acting on its ideal boundary is closely related to the exponential rate of its growth.

Softmax policy gradient methods converge at $O(1/t)$ rate with constants depending on problem and initialization.

New risk measure considers horizon risk and interest rate uncertainty.

This thesis applies entropy as a model independent measure to address three research questions concerning financial time series. In the first study we apply transfer entropy to drawdowns and drawups in foreign exchange rates, to study their correlation and cross correlation. When applied to daily and hourly EUR/USD and…

Entropy measures geodesic flow complexity.

We establish optimal rates for online regression for arbitrary classes of regression functions in terms of the sequential entropy introduced in (Rakhlin, Sridharan, Tewari, 2010). The optimal rates are shown to exhibit a phase transition analogous to the i.i.d./statistical learning case, studied in (Rakhlin, Sridharan,…

Estimating the entropy based on data is one of the prototypical problems in distribution property testing and estimation. For estimating the Shannon entropy of a distribution on $S$ elements with independent samples, [Paninski2004] showed that the sample complexity is sublinear in $S$, and [Valiant--Valiant2011] showed…

The paper analyzes the convergence rates of Q-learning with entropy regularization and linear function approximation.

The author proposes a finance trading strategy named Entropy Oriented Trading and apply thermodynamics on the strategy. The state variables are chosen so that the strategy satisfies the second law of thermodynamics. Using the law, the author proves that the rate of investment (ROI) of the strategy is equal to or more t…

In this paper, we propose a general framework to learn a robust large-margin binary classifier when corrupt measurements, called anomalies, caused by sensor failure might be present in the training set. The goal is to minimize the generalization error of the classifier on non-corrupted measurements while controlling th…

We consider the entropy of the solution to the heat equation on a Riemannian manifold. When the manifold is compact, we provide two estimates on the rate of change of the entropy in terms of the lower bound on the Ricci curvature and the spectral gap respectively. Our explicit computation for the three dimensional hype…

In this paper we study the consistency of an empirical minimum error entropy (MEE) algorithm in a regression setting. We introduce two types of consistency. The error entropy consistency, which requires the error entropy of the learned function to approximate the minimum error entropy, is shown to be always true if the…

The paper characterizes probability and entropy of exponentially growing sample spaces.

In this article we prove two formulas for the topological entropy of an F-optical Hamiltonian flow induced by a C^{\infty} Hamiltonian, where F is a Lagrangian distribution. In these formulas, we calculate the topological entropy as the exponential growth rate of the average of the determinant of the differential of th…

Improved first-order algorithm for entropy regularized OT with faster convergence.

Sharp lower bounds on shallow neural networks' approximation rates are derived.

Unique continuation result for expanding Ricci solitons.

Gibbs sampler contracts entropy under strong log-concavity, improving mixing time.

The paper tightens the regret rate for linear bandit problems.

In this paper, we propose a general framework to learn a robust large-margin binary classifier when corrupt measurements, called anomalies, caused by sensor failure might be present in the training set. The goal is to minimize the generalization error of the classifier on non-corrupted measurements while controlling th…

Paper proves volume growth estimate for steady gradient Ricci solitons.

Investigates the relationship between US money supply and asset indices over 2001-2019.

Entropy-regularized NPG converges linearly with linear function approximation.

Let $M$ be a compact Riemannian manifold without boundary and $V:M\to \mathbb R$ a smooth function. Denote by $P_t$ and ${\rm d}μ=e^V\,{\rm d} x$ the semigroup and symmetric measure of the second order differential operator $L=Δ+\nabla V\cdot\nabla$. For some suitable convex function $Φ:{\mathcal I}\to\mathbb R$ define…

The paper proves a quantitative rigidity result for spaces with specific curvature bounds.

Mathematical study of excess growth rate connects info theory with finance.

Gradient descent converges with arbitrary stepsize for separable data under Fenchel-Young losses.

Paper proposes deep neural networks for nonparametric regression from dependent data.

New method uses entropy dissipation to prove isoperimetric inequalities.

Fine-tuning improves information conveyance in language models by reorganizing uncertainty into more informative sequences.

There are many industrial situations where rods are used to stir a fluid, or where rods repeatedly stretch a material such as bread dough or taffy. The goal in these applications is to stretch either material lines (in a fluid) or the material itself (for dough or taffy) as rapidly as possible. The growth rate of mater…

Maps on infinite-type surfaces linked to 3-manifold flows.

Maximizes coding rate difference for robust, discriminative features.

Paper develops a new probabilistic method for American options using entropy regularization.

The profile of a sample is the multiset of its symbol frequencies. We show that for samples of discrete distributions, profile entropy is a fundamental measure unifying the concepts of estimation, inference, and compression. Specifically, profile entropy a) determines the speed of estimating the distribution relative t…

AECF improves multimodal inference robustness and calibration.

Holomorphic automorphisms on hyperkähler manifolds with high entropy are Kummer examples.

Building accurate language models that capture meaningful long-term dependencies is a core challenge in natural language processing. Towards this end, we present a calibration-based approach to measure long-term discrepancies between a generative sequence model and the true distribution, and use these discrepancies to …

Improved exploration methods for reinforcement learning with reduced sample complexity.

New method accelerates convergence for entropy-regularized reinforcement learning problems.

Study exact minimax rates for density estimation over convex classes, extending previous work.