This paper develops sparse alternatives to continuous distributions, including new types of Gaussians and attention mechanisms.
problem Creating flexible continuous distributions with varying support for machine learning applications.
method Defining Ω-regularized prediction maps and Fenchel-Young losses for arbitrary domains, and deriving new types of Gaussians and attention mechanisms. result Sparse alternatives to continuous distributions, including deformed exponential families and β-Gaussians, are introduced. New loss functions based on f-divergences improve language model performance.
problem Improving multiclass classification and language modeling performance.
method Constructing new convex loss functions using f-divergences and deriving an operator for computation.
result The α-divergence loss function with α=1.5 performs well across various tasks. The paper studies convergence rates of Tsallis entropic regularization in optimal transport.
problem Optimal transport with regularization.
method Γ-convergence and quantization/shadow arguments.
result Derives convergence rate of Tsallis entropic regularization.
We establish linear regret bounds for convex smooth losses using Fenchel-Young losses.
problem Establishing linear regret bounds for convex smooth losses.
method Constructing a convex smooth surrogate loss using Fenchel-Young losses generated by the convolutional negentropy.
result We derive a smooth loss with a linear surrogate regret bound.
In this paper, we present a new class of Markov decision processes (MDPs), called Tsallis MDPs, with Tsallis entropy maximization, which generalizes existing maximum entropy reinforcement learning (RL). A Tsallis MDP provides a unified framework for the original RL problem and RL with various types of entropy, includin…
Simplified proof for Tsallis-INF algorithm without conjugate functions.
problem Deriving a best-of-both-worlds guarantee for Tsallis-INF.
method Modern tools from online convex optimization, avoiding conjugate functions.
result A slimmer proof with simplified constants.
A novel PP algorithm using GMMs and GAs for detecting informative structures.
problem Detecting informative structures in multivariate datasets.
method Gaussian mixture models (GMMs) and Genetic Algorithms (GAs) for optimal projection.
result The approach effectively detects informative structures in multivariate datasets.
Modified Bakry-Émery criterion inequality for Tsallis entropy monotonicity.
problem Establishing improved logarithmic Sobolev inequalities and monotonicity of Tsallis entropy.
method Proving a one-parameter family of weighted Bakry-Émery Γ2 criterion inequalities and a modified inequality. result Yields a family of sharp Sobolev inequalities and monotonicity of Tsallis entropy.
Paper finds a new principle for optimizing consumption and wealth using Tsallis entropy.
problem Optimal consumption-investment problem with recursive utility.
method Established connection to quadratic BSDE, derived stochastic maximum principle.
result Proved existence of optimal strategy and analyzed coupled system.
The construction of efficient and effective decision trees remains a key topic in machine learning because of their simplicity and flexibility. A lot of heuristic algorithms have been proposed to construct near-optimal decision trees. ID3, C4.5 and CART are classical decision tree algorithms and the split criteria they…
This is full length article (draft version) where problem number of topics in Topic Modeling is discussed. We proposed idea that Renyi and Tsallis entropy can be used for identification of optimal number in large textual collections. We also report results of numerical experiments of Semantic stability for 4 topic mode…
OT-ICA uses optimal transport to find independent components, outperforming traditional methods.
problem Finding independent components from linear mixtures of signals.
method OT-ICA uses the squared Wasserstein distance to maximize non-Gaussianity, optimizing projections via gradient descent.
result OT-ICA outperforms traditional proxy-based methods in various applications.
In this study, we analyze the aerospace stocks prices in order to characterize the sector behavior. The data analyzed cover the period from January 1987 to April 1999. We present a new index for the aerospace sector and we investigate the statistical characteristics of this index. Our results show that this index is we…
MGD combines maximum entropy and diffusion methods for efficient sampling.
problem Generating samples from limited information in high dimensions.
method Moment Guided Diffusion (MGD) using stochastic differential equations.
result MGD efficiently samples maximum entropy distributions in finite time.
Develops a Best-of-Both-Worlds algorithm for linear contextual bandits with Tsallis entropy.
problem Linear contextual bandits with i.i.d. contexts.
method Follow-The-Regularized-Leader (FTRL) with Tsallis entropy.
result Achieves $O\left(\log(T)^{\frac{1+β}{2+β}}T^{\frac{1}{2+β}}
ight)$ regret under margin condition.
Estimate relaxation times in nonextensive systems using gradient flow for Tsallis entropy maximization.
problem Estimating relaxation times in financial market dynamics.
method Developing a method using EGF for maximizing Tsallis entropy.
result Longer relaxation times for nonextensive systems compared to Shannon entropy.
Optimal control in latent factor models uses Tsallis entropy for exploration.
problem Optimal control in models with latent factors.
method Reward exploration with Tsallis entropy and derive q-Gaussian distribution over states. result Optimal policy derived in a model-agnostic setting.
A hybrid impurity measure balances theoretical soundness and computational efficiency.
problem Developing a robust impurity measure for decision trees.
method Integrates Tsallis entropy with an exponential polarization component.
result Simple parametric measures outperform ITC, but ITC variants are competitive with strong theoretical guarantees.
This work broadens calibeating to various proper losses using Bregman divergence.
problem Calibration for a wide range of proper losses.
method Regret minimization and Bregman divergence approach.
result U-calibration results for a family of Tsallis losses with logarithmic regret and dimension independence.
This work generalizes calibeating for a broader range of proper losses using Bregman divergence.
problem Calibration for a wide range of proper losses beyond Brier and log loss.
method Regret minimization based on Bregman divergence for a family of proper losses.
result U-calibration results for a family of Tsallis losses with logarithmic regret and dimension independence.
Improved regret bounds for Tsallis-INF in adversarial bandits and corruptions.
problem Adversarial bandits and corruptions in multiarmed bandit problems.
method Improved regret bounds for Tsallis-INF algorithm.
result Achieves $\mathcal{O}\left(\left(\sum_{i
eq i^*} \frac{1}{Δ_i}
ight)\log_+\left(\frac{(K-1)T}{\left(\sum_{i
eq i^*} \frac{1}{Δ_i}
ight)^2}
ight)+\sqrt{C\left(\sum_{i
eq i^*}\frac{1}{Δ_i}
ight)\log_+\left(\frac{(K-1)T}{C\sum_{i
eq i^*}\frac{1}{Δ_i}}
ight)}
ight)$ regret bound.
This paper introduces a new potential function using Tsallis entropy for neural network optimization.
problem The challenge of obtaining exponential convergence in neural network optimization.
method Utilizes a linearized potential function based on Csiszár type of Tsallis entropy.
result Derives an exponential convergence result in neural network optimization.
Coupled entropy corrects flaws in Tsallis entropy for complex systems.
problem Misinterpretation of generalized temperature and entropy.
method Derived from generalized Pareto and Student's t distributions.
result Provides balanced measure of uncertainty for complex systems.
This study uses Tsallis entropy to analyze diversification and integration in Italian stock market companies.
problem Examining the industrial structure and market reactions of cross-shareholding networks.
method Developed Tsallis entropy approach to model diversification and integration using copulas.
result Entropy analysis reveals insights into market polarisation and fairness.
Study analyzes stock market dynamics using Tsallis statistics and GHE, revealing pre-bubble and post-bubble market characteristics.
problem Understanding stock market dynamics and predicting market bubbles.
method Non-linear analysis using time-dependent Tsallis statistics and Generalized Hurst Exponents.
result Temporal trends of q-triplet values differ before and after market bubbles, indicating significant market dynamics changes.
Paper presents characteristic function of Tsallis q-Gaussian and its applications.
problem Modeling input quantities in measurement models using Tsallis q-Gaussians.
method Developed a characteristic function and proposed a numerical method for its inversion.
result Exact probability distribution of output quantities can be determined.
In this paper, we propose a novel maximum causal Tsallis entropy (MCTE) framework for imitation learning which can efficiently learn a sparse multi-modal policy distribution from demonstrations. We provide the full mathematical analysis of the proposed framework. First, the optimal solution of an MCTE problem is shown …
We find the wealth distribution for an economic agent in the financial market, in analogy with standard derivation of generaliz Boltzman (Tsallis) factor in statistical mechanics. In this respect, Tsallis entropic index separates two different regimes, the large and small size market. The Pareto like wealth distributio…
Study on utility maximization with Tsallis entropy in reinforcement learning.
problem Exploring utility maximization with Tsallis entropy in reinforcement learning.
method Introducing Tsallis entropy regularizer to induce exploration, investigating specific examples, characterizing well-posedness, designing reinforcement learning algorithm.
result Characterized well-posedness and provided semi-closed-form solutions for specific examples, found distinct optimal strategies.
Earlier studies have shown that stock market distributions can be well described by distributions derived from Tsallis entropy, which is a generalization of Shannon entropy to non-extensive systems. In this paper, Tsallis relative entropy (TRE), which is the generalization of Kullback-Leibler relative entropy (KLRE) to…
The paper calculates bounds for risk metrics and entropies under partial information constraints.
problem Analyzing risk metrics and entropies for unimodal, symmetric distributions with limited information.
method Develops lower and upper bounds for worst-case distortion riskmetrics and weighted entropy for unimodal, symmetric distributions with known mean and variance.
result Sharp upper bounds for distortion riskmetrics and weighted entropy for symmetric distributions.
New risk measure considers horizon risk and interest rate uncertainty.
problem Dynamic risk evaluation considering horizon risk and interest rate uncertainty.
method Introduced a risk measure based on generalized Tsallis entropy.
result New q-entropic risk measure quantifies capital requirement.
This work models financial market returns with asymmetric Tsallis distributions, improving fit over symmetric q-Gaussians.
problem Non-symmetric behavior of stock market returns over time scales.
method Linear combination of two independent normalized half q-Gaussians with different parameters.
result Asymmetric distributions provide better fits to stock market returns than symmetric q-Gaussians, especially over longer time scales.
We developed a strategic of optimal portfolio based on information theory and Tsallis statistics. The growth rate of a stock market is defined by using q-deformed functions and we find that the wealth after n days with the optimal portfolio is given by a q-exponential function. In this context, the asymptotic optim…
A pricing principle is introduced for non-attainable claims in incomplete markets.
problem Pricing non-attainable contingent claims in incomplete markets.
method Distorted Radon-Nikodym derivative and Tsallis relative entropy over a family of equivalent martingale measures.
result The pricing principle is closely related to backward stochastic differential equations and is arbitrage-free and time-consistent.
New method uses asymmetric Tsallis relative entropy for better risk assessment in financial portfolios.
problem Improving risk assessment for financial portfolios using asymmetric data.
method Generalized Tsallis relative entropy (ATRE) for asymmetric distributions of returns.
result ATRE shows better risk-return profiles, especially during market crashes.
We recently showed that the S&P500 stock market index is well described by Tsallis non-extensive statistics and nonlinear Fokker-Planck time evolution. We argued that these results should be applicable to a broad range of markets and exchanges where anomalous diffusion and `heavy' tails of the distribution are present.…
The theoretical basis for a candidate variational principle for the information bottleneck (IB) method is formulated within the ambit of the generalized nonadditive statistics of Tsallis. Given a nonadditivity parameter q, the role of the \textit{additive duality} of nonadditive statistics (q∗=2−q) in relating…
Recently deep reinforcement learning (DRL) has achieved outstanding success on solving many difficult and large-scale RL problems. However the high sample cost required for effective learning often makes DRL unaffordable in resource-limited applications. With the aim of improving sample efficiency and learning performa…
In this paper we study the possible microscopic origin of heavy-tailed probability density distributions for the price variation of financial instruments. We extend the standard log-normal process to include another random component in the so-called stochastic volatility models. We study these models under an assumptio…
Sparse RSP routing improves graph exploration and classification.
problem Optimal randomized routing and distance measures on weighted graphs.
method Tsallis divergence regularization for sparse RSP.
result Sparse random walk converges to least-cost graph as temperature decreases.
One of the major issues studied in finance that has always intrigued, both scholars and practitioners, and to which no unified theory has yet been discovered, is the reason why prices move over time. Since there are several well-known traditional techniques in the literature to measure stock market volatility, a centra…
Distributions derived from non-extensive Tsallis statistics are closely connected with dynamics described by a nonlinear Fokker-Planck equation. The combination shows promise in describing stochastic processes with power-law distributions and superdiffusive dynamics. We investigate intra-day price changes in the S&P500…
We derive an algorithm that achieves the optimal (within constants) pseudo-regret in both adversarial and stochastic multi-armed bandits without prior knowledge of the regime and time horizon. The algorithm is based on online mirror descent (OMD) with Tsallis entropy regularization with power α=1/2 and reduced-varian…
q-VAE extracts disentangled latent spaces for robot control and dynamic systems.
problem Disentangled representation learning for latent spaces in robotics.
method Proposes q-VAE based on Tsallis statistics, improving disentanglement and latent dynamics.
result Improves disentangled representation while maintaining data reconstruction accuracy.
We establish an analogy between the motion of spring whose mass increases linearly with time and volatile stock markets dynamics within an economic model based on simple temporal demand and supply functions [J. Phys. A: Math. Gen. 33, 3637 (2000)]. The total system energy E_t is shown to be proportional to a decreasing…
Researchers explore gauge freedom in entropies of q-Gaussian measures.
problem Exploring the gauge freedom of entropies in q-Gaussian measures. method Introducing a refined q-logarithmic function to demonstrate gauge freedom. result Different escort expectations can lead to the same entropy but different relative entropies.
The paper analyzes worst-case distortion risk metrics and weighted entropy under partial information.
problem Analyzing worst-case distortion risk metrics and weighted entropy with limited information.
method General distributions, partial information (mean and variance), various entropies and risk measures.
result Provides worst-case results for distortion risk metrics and weighted entropy.