The well known maximum-entropy principle due to Jaynes, which states that given mean parameters, the maximum entropy distribution matching them is in an exponential family, has been very popular in machine learning due to its "Occam's razor" interpretation. Unfortunately, calculating the potentials in the maximum-entro…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
169,051 papers · 148 categories
Trend · papers per month
48 results for “maximum entropy on the mean”
Paper develops MRCs for supervised classification using generalized maximum entropy.
problem Developing robust classifiers for decision problems.
method Generalized maximum entropy principle applied to minimax risk classifiers.
result Learning techniques for determining MRCs with performance guarantees.
Extended wMEM approach for MEG inverse problem using wavelet and spatial filters.
problem Infer brain activity from full space-time data in MEG.
method Wavelet decomposition, spatial filters, Kronecker product modeling, numerical optimization.
result Smooth numerical optimization problem solved with reasonable dimensionality.
The paper extends entropy maximization to multiscale settings and applies it to neural networks.
problem Achieving optimal risk bounds in neural networks using multiscale entropy.
method Generalizing maximum entropy to multiscale settings and applying it to neural networks.
result The multiscale Gibbs posterior can achieve a smaller excess risk than the single-scale Gibbs posterior in a teacher-student scenario.
New method calibrates reference distributions for bounded support.
problem Lack of principled method for bounded-support statistical reference distributions.
method Formulated maximum entropy on projective space of nonnegative measures.
result Prescribed acceptance region uniquely determines deformation parameter.
A new nonparametric approach for system identification has been recently proposed where the impulse response is modeled as the realization of a zero-mean Gaussian process whose covariance (kernel) has to be estimated from data. In this scheme, quality of the estimates crucially depends on the parametrization of the cov…
Entropic Determinantsstat.ML
The ability of many powerful machine learning algorithms to deal with large data sets without compromise is often hampered by computationally expensive linear algebra tasks, of which calculating the log determinant is a canonical example. In this paper we demonstrate the optimality of Maximum Entropy methods in approxi…
A new algorithm learns diverse policies in reinforcement learning.
problem Learning diverse behaviors in reinforcement learning.
method Proposes Maximum Entropy Diverse Exploration (MEDE) algorithm.
result The set of policies learned by MEDE capture the same modalities as the optimal maximum entropy policy.
A neural network derived from first principles using MaxEnt.
problem Developing a neural network from first principles.
method Derived a neural network using the principle of Maximum Entropy, with linear dimension-reducing transformations and conditional mean estimators.
result Unified theoretical justification for activation functions like sigmoid, softplus, and relu.
Maximum entropy modeling is a flexible and popular framework for formulating statistical models given partial knowledge. In this paper, rather than the traditional method of optimizing over the continuous density directly, we learn a smooth and invertible transformation that maps a simple distribution to the desired ma…
The MEM method uses data-driven priors for linear inverse problems, proving convergence and estimating differences.
problem Linear inverse problems with approximate priors.
method Maximum Entropy on the Mean (MEM) method with data-driven priors.
result Empirical mean convergence and estimates for prior differences based on epigraphical distance.
New analysis shows entropy term cancels out in likelihood-based OOD detection.
problem Curious likelihood values for out-of-distribution data.
method Decomposed average likelihood into KL divergence and entropy terms.
result Entropy term explains OOD behaviour and cancels out in expectation.
Method determines asset prices in incomplete markets to optimize portfolios.
problem Optimizing portfolios in incomplete markets with price constraints.
method Maximum entropy in the mean to adjust distortion function from bid-ask data.
result Prices of assets comply with portfolio optimization constraints.
MGD combines maximum entropy and diffusion methods for efficient sampling.
problem Generating samples from limited information in high dimensions.
method Moment Guided Diffusion (MGD) using stochastic differential equations.
result MGD efficiently samples maximum entropy distributions in finite time.
Researchers use Gaussian processes to approximate Lagrange multipliers for Maximum-Entropy distributions.
problem Finding Lagrange multipliers for Maximum-Entropy distributions is computationally challenging.
method Employed Gaussian processes to approximate the Lagrange multipliers as a map of moments. Optimized hyperparameters by maximizing log-likelihood.
result Data-driven Maximum-Entropy closure performs well in approximating non-equilibrium distributions.
Enhances RL by controlling policy stochasticity through trajectory entropy constraints.
problem Non-stationary Q-value estimation and short-sighted entropy tuning in maximum entropy RL.
method Proposes TECRL framework with separate Q-functions for reward and entropy, enforcing a trajectory entropy constraint.
result DSAC-E algorithm achieves higher returns and better stability on OpenAI Gym benchmarks.
The paper extends flow theory with free boundaries, proving key bounds and theorems.
problem Mean convex mean curvature flow with free boundary conditions.
method Triple-approximation scheme combining maximum principle and various theorems.
result A priori bound on the ratio of second fundamental form to mean curvature.
New distances measure mixtures of Gaussians, useful in machine learning.
problem Comparing distributions with disjoint supports.
method Schoenberg-Rao distances based on concave Rao's entropy.
result Closed-form distances for mixtures of Gaussians.
The paper presents a method to estimate joint interventional distributions from marginal interventional data.
problem Estimating joint interventional distributions from marginal interventional data.
method The paper extends the Causal Maximum Entropy method to use interventional data and employs Lagrange duality to prove the solution lies in the exponential family.
result The method allows for causal feature selection and inference of joint interventional distributions.
We present a new statistical learning paradigm for Boltzmann machines based on a new inference principle we have proposed: the latent maximum entropy principle (LME). LME is different both from Jaynes maximum entropy principle and from standard maximum likelihood estimation.We demonstrate the LME principle BY deriving …
New MC-Tree method combines Monte Carlo and binomial tree for option pricing and CVA.
problem Combining Monte Carlo and binomial tree methods for accurate and efficient option pricing and CVA calculations.
method MC-Tree method that mixes Monte Carlo and binomial tree parameters, using maximum entropy distributions for compound densities.
result MC-Tree method provides accurate and efficient option pricing and CVA calculations.
Unification of field theory and maximum entropy methods for learning probability densitiesphysics.data-an
The need to estimate smooth probability distributions (a.k.a. probability densities) from finite sampled data is ubiquitous in science. Many approaches to this problem have been described, but none is yet regarded as providing a definitive solution. Maximum entropy estimation and Bayesian field theory are two such appr…
We discuss the systemic risk implied by the interbank exposures reconstructed with the maximum entropy method. The maximum entropy method severely underestimates the risk of interbank contagion by assuming a fully connected network, while in reality the structure of the interbank network is sparsely connected. Here, we…
Robust diffusion adaptive estimation algorithms based on the maximum correntropy criterion (MCC), including adaptation to combination MCC and combination to adaptation MCC, are developed to deal with the distributed estimation over network in impulsive (long-tailed) noise environments. The cost functions used in distri…
A new IRL model recovers reward and state structure from expert demonstrations.
problem Limitation of classical maximum entropy model in capturing state structure.
method Generalized maximum causal entropy for IRL models.
result Empirically outperforms classical models in recovering reward and state structure.
Synthesizes sensor likelihoods to enforce accuracy constraints in uncertain systems.
problem Designing sensing architectures for systems with uncertain or unavailable sensor models and accuracy requirements.
method Inverts the design flow, synthesizing measurement likelihoods that minimize Kullback-Leibler divergence from the prior while enforcing an accuracy bound.
result The method synthesizes a maximum-entropy posterior and induced likelihood, accommodating various discrepancy metrics.
Unified framework for maximum entropy RL using Tsallis entropy.
problem Generalizing maximum entropy reinforcement learning with various entropies.
method Tsallis MDPs with Tsallis entropy maximization, controlling entropic index.
result Different entropic indices lead to different optimal policies and exploration tendencies.
Bayesian models use hyperparameters to indirectly assign priors, and this work shows how these priors can be derived from maximum entropy principles.
problem Understanding the assumptions and dependencies in Bayesian hierarchical models.
method Demonstrates how canonical distributions and maximum entropy principles can be used to derive marginal priors in hierarchical models.
result Marginal priors in hierarchical models derived from maximum entropy principles have different constraints compared to the original priors.
Upper bound on index of rotationally symmetric self-shrinking tori.
problem Stability of singularities in mean curvature flow.
method Entropy functional and eigenvalue analysis.
result Upper bound on the index of rotationally symmetric self-shrinking tori.
The maximum entropy principle can be used to assign utility values when only partial information is available about the decision maker's preferences. In order to obtain such utility values it is necessary to establish an analogy between probability and utility through the notion of a utility density function. According…
Maximum Entropy Framework for a Universal Rank Order distribution with Socio-economic Applicationsphysics.soc-ph
Paper derives the maximum entropy characteristics of a rank order distribution for socio-economic applications.
problem Deriving the maximum entropy characteristics of a rank order distribution for socio-economic applications.
method Maximum entropy framework, deriving the discrete generalized beta distribution under a bivariate utility constraint.
result The discrete generalized beta distribution is a natural maximum entropy distribution under an appropriate bivariate utility constraint.
Improved exploration methods for reinforcement learning with reduced sample complexity.
problem Challenges in reinforcement learning exploration in unknown environments.
method Proposed game-theoretic and trajectory entropy algorithms with improved sample complexity.
result Established statistical advantage of entropy-regularized MDPs for exploration and reduced sample complexity.
Paper proposes a policy-search algorithm to learn entropy-maximizing exploration policies in reward-free environments.
problem Reward-free learning in high-dimensional, continuous-control domains.
method Maximum Entropy POLicy optimization (MEPOL) algorithm that maximizes a non-parametric state entropy estimate.
result MEPOL learns a maximum-entropy exploration policy that facilitates learning various reward-based tasks.
The paper introduces a new intrinsic reward method for exploration in reinforcement learning.
problem Improving exploration in reinforcement learning agents.
method Intrinsic rewards proportional to the entropy of future state-action features.
result The new objective leads to improved visitation of features within individual trajectories.
The problem of determining the joint probability distributions for correlated random variables with pre-specified marginals is considered. When the joint distribution satisfying all the required conditions is not unique, the "most unbiased" choice corresponds to the distribution of maximum entropy. The calculation of t…
Paper finds a new principle for optimizing consumption and wealth using Tsallis entropy.
problem Optimal consumption-investment problem with recursive utility.
method Established connection to quadratic BSDE, derived stochastic maximum principle.
result Proved existence of optimal strategy and analyzed coupled system.
Here we present an application of two maxentropic procedures to determine the probability density distribution of compound sums of random variables, using only a finite number of empirically determined fractional moments. The two methods are the Standard method of Maximum Entropy (SME), and the method of Maximum Entrop…
MEP-Net uses MEP to generate solutions from limited data.
problem Generating solutions to scientific problems with incomplete information.
method Combines MEP with neural networks to learn complex distributions from moment constraints.
result Demonstrates MEP-Net's effectiveness in modeling biochemical reaction networks and generating complex distributions.
This work addresses two main issues of the standard Kernel Entropy Component Analysis (KECA) algorithm: the optimization of the kernel decomposition and the optimization of the Gaussian kernel parameter. KECA roughly reduces to a sorting of the importance of kernel eigenvectors by entropy instead of by variance as in K…
MEMe efficiently approximates large-scale ML problems with hundreds of moments.
problem Efficient approximation in large-scale machine learning.
method Maximum entropy algorithm with hundreds of moments for computationally efficient approximations.
result Superior to existing approaches in fast log determinant estimation and Bayesian optimisation.
Unified framework for network model assessment using maximum entropy.
problem Statistical inference for network models.
method Constrained entropy-maximization problem, Lagrange multipliers.
result Consistent goodness-of-fit and two-sample tests for network models.
A new nonparametric approach for system identification has been recently proposed where the impulse response is seen as the realization of a zero--mean Gaussian process whose covariance, the so--called stable spline kernel, guarantees that the impulse response is almost surely stable. Maximum entropy properties of the …
A new method for RL with continuous actions improves stability and scalability.
problem Stability and scalability issues in existing RL methods.
method Soft policy gradient with entropy regularization, combined with double sampling for soft Bellman equation.
result Outperforms off-policy prior methods in continuous action RL tasks.
Utility Function from Maximum Entropy Principlecond-mat.stat-mech
We apply the maximum entropy principle to economic systems in equilibrium and find the density function for the market's wealth. This is the same as price density which is used for insurance pricing. The risk aversion parameter of the agent then it's utility function with respect to this density is derived.
Improved PCA accuracy using maximum entropy method.
problem Accuracy issues in classical PCA.
method Model uncertainty with random variables, apply Maximum Entropy Method.
result Improved estimates of distances between data items.
Paper presents a bias mitigation method using maximum entropy.
problem Bias in datasets due to over/under-representation of groups.
method Maximum entropy principle applied to data preprocessing.
result Achieves target fairness metrics and maintains classifier accuracy.
New algorithm estimates semi-continuous data density using entropy maximization.
problem Estimating density functions for semi-continuous data.
method Maximum entropy principle, requiring only constraint function samples.
result Estimate has significantly less bias compared to existing methods.
RL agents optimize only specified features; this project infers unmentioned preferences from the state of the environment.
problem RL agents are indifferent to features not specified in a reward function, leading to unconsidered preferences.
method Developed an algorithm based on Maximum Causal Entropy IRL to infer preferences and side effects from the state of the environment.
result Information from the initial state can infer both side effects to avoid and preferences for environment organization.