A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
VES-Gamma adapts EI using information-theoretic principles.
A new acquisition function RMES improves Bayesian optimization performance.
Supervised topic models utilize document's side information for discovering predictive low dimensional representations of documents. Existing models apply the likelihood-based estimation. In this paper, we present a general framework of max-margin supervised topic models for both continuous and categorical response var…
We define On-Average KL-Privacy and present its properties and connections to differential privacy, generalization and information-theoretic quantities including max-information and mutual information. The new definition significantly weakens differential privacy, while preserving its minimalistic design features such …
The Ryu-Takayanagi (RT) formula relates the entanglement entropy of a region in a holographic theory to the area of a corresponding bulk minimal surface. Using the max flow-min cut principle, a theorem from network theory, we rewrite the RT formula in a way that does not make reference to the minimal surface. Instead, …
We propose a max-pooling based loss function for training Long Short-Term Memory (LSTM) networks for small-footprint keyword spotting (KWS), with low CPU, memory, and latency requirements. The max-pooling loss training can be further guided by initializing with a cross-entropy loss trained network. A posterior smoothin…
Study shows strong min-max principle for phase transitions.
Entropy Search (ES) and Predictive Entropy Search (PES) are popular and empirically successful Bayesian Optimization techniques. Both rely on a compelling information-theoretic motivation, and maximize the information gained about the of the unknown function; yet, both are plagued by the expensive computatio…
Maximum entropy distributions with discrete support in dimensions arise in machine learning, statistics, information theory, and theoretical computer science. While structural and computational properties of max-entropy distributions have been extensively studied, basic questions such as: Do max-entropy distributio…
Paper proposes a new uncertainty measure for active learning in neural networks.
GIBBON unifies Bayesian optimization for various problem types.
The paper analyzes RLHF with human feedback and provides convergence results for MLE and pessimistic MLE.
Bayesian optimization methods improved for min max optimization problems.
JES optimizes expensive functions by considering joint entropy over input and output spaces.
Entropy is a natural geometric quantity measuring the complexity of a surface embedded in . For dynamical reasons relating to mean curvature flow, Colding-Ilmanen-Minicozzi-White conjectured that the entropy of any closed surface is at least that of the self-shrinking two-sphere. We prove this conjecture …
Study shows fast rates for inverse reinforcement learning with linear rewards.
BMBO-DARN optimizes expensive functions with varying fidelities.
Attention to entropic communication improves message decoding and cooperation.
Inspired by the success of Convolutional Neural Networks (CNNs) for supervised prediction in images, we design the Deconvolutional Generative Model (DGM), a new probabilistic generative model whose inference calculations correspond to those in a given CNN architecture. The DGM uses a CNN to design the prior distributio…
A new active learning method considers both uncertainty and diversity to minimize labeling and decision costs.
Paper develops MRCs for supervised classification using generalized maximum entropy.
Unified framework connects EI and information-theoretic acquisition functions.
In this paper, we present a novel and general framework called {\it Maximum Entropy Discrimination Markov Networks} (MaxEnDNet), which integrates the max-margin structured learning and Bayesian-style estimation and combines and extends their merits. Major innovations of this model include: 1) It generalizes the extant …
We present a new statistical learning paradigm for Boltzmann machines based on a new inference principle we have proposed: the latent maximum entropy principle (LME). LME is different both from Jaynes maximum entropy principle and from standard maximum likelihood estimation.We demonstrate the LME principle BY deriving …
The well known maximum-entropy principle due to Jaynes, which states that given mean parameters, the maximum entropy distribution matching them is in an exponential family, has been very popular in machine learning due to its "Occam's razor" interpretation. Unfortunately, calculating the potentials in the maximum-entro…
MESMOC optimizes constrained multi-objective problems efficiently.
Paper finds a new principle for optimizing consumption and wealth using Tsallis entropy.
Quantum machine learning uses quantum cross entropy to minimize loss, but measurement loss affects this process.
Proves multiplicity one for boundary minimal hypersurfaces in compact manifolds.
Bayesian optimization (BO) is a model-based approach to sequentially optimize expensive black-box functions, such as the validation error of a deep neural network with respect to its hyperparameters. In many real-world scenarios, the optimization is further subject to a priori unknown constraints. For example, training…
MEP-Net uses MEP to generate solutions from limited data.
Study on spectral gaps of hyperbolic surfaces as genus increases.
The paper calculates bounds for risk metrics and entropies under partial information constraints.
Rate GENERIC extends thermodynamics principles to non-equilibrium systems.
Improved MESMOC+ optimizes constrained multi-objective problems efficiently.
Pointwise localization allows more precise localization and accurate interpretability, compared to bounding box, in applications where objects are highly unstructured such as in medical domain. In this work, we focus on weakly supervised localization (WSL) where a model is trained to classify an image and localize regi…
A pricing principle is introduced for non-attainable claims in incomplete markets.
The well-known Gumbel-Max trick for sampling from a categorical distribution can be extended to sample elements without replacement. We show how to implicitly apply this 'Gumbel-Top-' trick on a factorized distribution over sequences, allowing to draw exact samples without replacement using a Stochastic Beam Sea…
This paper presents a new approach, called perturb-max, for high-dimensional statistical inference that is based on applying random perturbations followed by optimization. This framework injects randomness to maximum a-posteriori (MAP) predictors by randomly perturbing the potential function for the input. A classic re…
New theorem finds new minimal hypersurfaces in hyperbolic space.
We apply the maximum entropy principle to economic systems in equilibrium and find the density function for the market's wealth. This is the same as price density which is used for insurance pricing. The risk aversion parameter of the agent then it's utility function with respect to this density is derived.
The maximum entropy principle can be used to assign utility values when only partial information is available about the decision maker's preferences. In order to obtain such utility values it is necessary to establish an analogy between probability and utility through the notion of a utility density function. According…
Bayesian models use hyperparameters to indirectly assign priors, and this work shows how these priors can be derived from maximum entropy principles.
Paper develops a new probabilistic method for American options using entropy regularization.
New algorithm estimates semi-continuous data density using entropy maximization.
New causal versions of MaxEnt and PIR avoid paradoxical probability updates.
We consider the problem of discriminative factor analysis for data that are in general non-Gaussian. A Bayesian model based on the ranks of the data is proposed. We first introduce a new {\em max-margin} version of the rank-likelihood. A discriminative factor model is then developed, integrating the max-margin rank-lik…
In a standard setting of Bayesian optimization (BO), the objective function evaluation is assumed to be highly expensive. Multi-fidelity Bayesian optimization (MFBO) accelerates BO by incorporating lower fidelity observations available with a lower sampling cost. In this paper, we focus on the information-based approac…