A new Tsallis entropy criterion unifies decision tree split criteria.
problem Improving decision tree performance using a unified split criterion.
method Proposes a Tsallis Entropy Criterion (TEC) algorithm to unify Shannon entropy, Gain Ratio, and Gini index.
result TEC algorithm achieves statistically significant improvement over classical algorithms.
New algorithms minimize risk in MNL bandits, achieving near-optimal performance.
problem Minimizing risk in multi-armed bandit problems.
method Designing algorithms for various risk criteria (e.g., CVaR, Sharpe ratio, entropy risk).
result Near-optimal regret for the designed algorithms.
Improved pre-trained embeddings through effective entropy maximization.
problem Developing high-quality pre-trained embeddings for future tasks.
method E2MC criterion defined in terms of low-dimensional constraints.
result Significant improvement in downstream performance.
Three LF training criteria improve neural network acoustic models without cross-entropy pre-training.
problem Improving purely sequence-trained neural network acoustic models.
method Comparison of three lattice-free discriminative training criteria (MMI, bMMI, sMBR) on LVCSR tasks.
result LF-bMMI models outperform plain LF-MMI models by 5% WER on Switchboard datasets.
Suggests stopping criteria for feature selection using mutual information.
problem Automatic determination of optimal feature subset size and stopping criterion.
method Monitoring conditional mutual information (CMI) among groups of variables using Renyi's α-entropy.
result Easy to implement stopping criteria for feature selection.
A new method reduces compounding errors in model-based reinforcement learning.
problem Compounding errors in long horizon predictions from model-based reinforcement learning.
method Maximum Entropy Model Rollouts (MEMR) with non-uniform sampling and prioritized experience replay.
result Significantly reduces computation requirements compared to other model-based methods.
This work compares and evaluates various sampling methods for neural language models.
problem Lack of systematic comparison and myths about sampling methods.
method Monte Carlo sampling, importance sampling, compensated partial summation, noise contrastive estimation.
result All sampling methods can perform equally well if posterior probabilities are corrected.
Proposes a new loss function for deep neural networks.
problem Deep neural networks lack a direct method to discriminate between correct and competing classes.
method Introduces a discriminative loss function based on negative log likelihood ratio.
result Significantly outperforms cross-entropy loss on image classification tasks.
Gathering the most information by picking the least amount of data is a common task in experimental design or when exploring an unknown environment in reinforcement learning and robotics. A widely used measure for quantifying the information contained in some distribution of interest is its entropy. Greedily minimizing…
The ultimate goal of optimization is to find the minimizer of a target function.However, typical criteria for active optimization often ignore the uncertainty about the minimizer. We propose a novel criterion for global optimization and an associated sequential active learning strategy using Gaussian processes.Our crit…
The study examines entropy and pressure at infinity in negatively curved manifolds, linking them to strong positive recurrence.
problem Investigating strong positive recurrence in negatively curved manifolds.
method Defining and comparing entropy and pressure at infinity through different measures.
result Strong positive recurrence potentials admit finite Gibbs measures.
New statistical test for change-point detection using relative entropy.
problem Offline change-point detection using divergence metrics.
method Study of empirical relative entropy distributions, derivation of approximations, introduction of new Berry-Esseen bounds.
result Theoretical and practical validation of relative entropy for change-point detection.
A new efficient learning criterion QMEE reduces computational complexity of MEE.
problem High computational complexity of MEE criterion in large datasets.
method Proposes quantization approach to reduce computational complexity from quadratic to linear.
result Demonstrates excellent performance of QMEE in various examples.
Density destructors simplify complex PDFs to maximize entropy, linking to information theory.
problem Complex multivariate PDFs are hard to analyze.
method Invertible transforms that progressively remove structure from PDFs.
result Density destructors can improve estimates of information theoretic quantities.
The paper proposes pruning techniques for improving ensemble GP models.
problem Improving the generalization ability and reducing computational burden of GP ensemble models.
method Combining syntax and semantics-based GP models, and using pruning criteria based on correlation and entropy.
result Pruning criteria based on correlation and entropy can improve the generalization ability of the ensemble model.
Detecting and recovering labels in binomial logistic mixtures is challenging due to an information gap.
problem Detecting and recovering labels in binomial logistic mixtures
method Propose two feasibility-aware inference procedures
result Avoid misleading component selections and improve label probability calibration
Bayesian active learning improves holistic educational assessments.
problem Gap between holistic CJ and criterion-based rubrics in education.
method Extends Bayesian CJ to handle multiple LO components, using entropy-based active learning.
result Enhanced predictive rankings with uncertainty estimates and quantified assessor agreement.
New method uncovers zero entropy in dependent observations after finite samples.
problem Understanding uncertainty reduction in dependent observations.
method Minimum list entropy coupling, greedy algorithm.
result Zero entropy achieved with O(log(1/P_min)) samples for dependent observations.
We investigate the position of the Buchen-Kelly density in a family of entropy maximising densities which all match European call option prices for a given maturity observed in the market. Using the Legendre transform which links the entropy function and the cumulant generating function, we show that it is both the uni…
This work compares lattice-free and lattice-based training criteria for LVCSR.
problem Improving acoustic model performance in speech recognition.
method Direct comparison of lattice-free and lattice-based sequence discriminative training criteria using GPU.
result Lattice-free MMI performance is comparable to lattice-based criteria, while lattice-based sMBR remains superior.
Entropy measure quantifies volatility correlation and risk diversity in asset portfolios.
problem Quantifying volatility correlation and risk diversity in asset portfolios.
method Kullback-Leibler cluster entropy DC[P∥Q] for empirical and model probability distributions of realized volatility. result Portfolio built on diversity indexes derived from Kullback-Leibler entropy measure of realized volatility exhibits better performance.
MIM learns joint distributions with mutual information and low divergence.
problem Learning joint distributions over observations and latent variables.
method Probabilistic auto-encoder with three design principles: low divergence, high mutual information, and low marginal entropy.
result MIM learns representations with high mutual information, consistent encoding and decoding distributions, effective latent clustering, and comparable data log likelihood to VAE.
Paper presents a neural network for estimating wavefronts in direction of arrival scenarios.
problem Estimating the number of wavefronts in direction of arrival scenarios.
method Cross-entropy trained multilayer neural network for online adaptation of antenna array imperfections.
result The method outperforms classical model order selection schemes in accuracy, especially at low signal-to-noise-ratios.
Unsupervised model predicts facial attractiveness with high accuracy.
problem Capturing the complexity of facial attractiveness through machine learning.
method Infer probabilistic models of facial preferences using Maximum Entropy and neural networks.
result High prediction accuracy in gender classification of sculpting subjects.
The paper develops a theory for speculative decoding acceptance criteria.
problem Speculative decoding's acceptance criteria and their rejection regions.
method Characterization of rejection regions as lower level sets of the target distribution, derivation of exact and margin-based certificates.
result Relaxed and tree-based acceptance criteria substantially enlarge the region of certified acceptance.
Optimizes neural computation by combining multiple constraints using maximum entropy method.
problem Defines physical limits of neuron-like engineering by optimizing multiple performance criteria.
method Uses Jaynes' maximum entropy method to combine and optimize multiple constraints.
result Identifies a Shannon bits/joule statement as a result of combining constraints.
Optimized KECA extracts more expressive features by optimizing kernel decomposition and Gaussian kernel parameter.
problem Improving feature extraction efficiency and robustness in kernel-based data analysis.
method Optimized KECA method using ICA framework with gradient ascent search for optimal feature extraction.
result OKECA produces more expressive features than KECA, and is more robust to kernel parameter selection.
FCA unmixes matrices from mixtures using free probability theory.
problem Unmixing mixtures of freely independent random variables.
method Develops Free Component Analysis (FCA) using free probability theory.
result FCA outperforms vanilla ICA in various applications.
We obtain a compactness result for Fano manifolds and Kähler Ricci flows. Comparing to the more general Riemannian versions by Anderson and Hamilton, in this Fano case, the curvature assumption is much weaker and is preserved by the Kähler Ricci flows. One assumption is the boundedness of the Ricci potential and the ot…
Develops a mean-field theory for multi-head self-attention under cross-entropy training.
problem Mean-field analysis of multi-head self-attention under cross-entropy training.
method Mean-field theory for a simplified single-layer causal multi-head self-attention model.
result Proves a static finite-head approximation bound for the optimal risk.
In the world of modern financial theory, portfolio construction has traditionally operated under at least one of two central assumptions: the constraints are derived from a utility function and/or the multivariate probability distribution of the underlying asset returns is fully known. In practice, both the performance…
This work improves deep learning from noisy crowdsourced labels.
problem Learning label correction and neural classifier from noisy crowdsourced data.
method Coupled Cross-Entropy Minimization (CCEM) with identifiability and regularization.
result The CCEM criterion correctly identifies annotators' confusion and neural classifier under realistic conditions.
Here we present an application of two maxentropic procedures to determine the probability density distribution of compound sums of random variables, using only a finite number of empirically determined fractional moments. The two methods are the Standard method of Maximum Entropy (SME), and the method of Maximum Entrop…
The abstract discusses geometric inequalities related to black hole formation.
problem Establishing geometric inequalities for black hole formation.
method Applying Bekenstein's entropy bounds and Penrose's inequality to axisymmetric bodies.
result New criteria for black hole formation involving angular momentum, charge, and matter energy.
A new AI framework reduces costs and improves performance.
problem High costs and over-confidence in AI models.
method Entropy-optimizing reformulation of Boltzmann machines.
result Cheaper, more performant AI models with validated reliability.
New Bayesian models optimize quantiles and expectiles for stochastic functions.
problem Optimizing for quantiles and expectiles in stochastic functions.
method Proposed variational models and BO strategies for quantile and expectile regression.
result Proposed models and strategies outperform existing methods in heteroscedastic, non-Gaussian settings.
The Sinkhorn flow is a gradient flow in a nonlocal Wasserstein geometry.
problem Entropy-regularized optimal transport and its applications.
method Thermodynamic interpretation of the Sinkhorn algorithm.
result The Sinkhorn flow is the gradient flow of entropy in a nonlocal Wasserstein geometry.
Paper optimizes approximating high-dimensional diffusions by independent coordinates.
problem Optimizing approximations of high-dimensional diffusions by independent coordinates.
method Introduces independent projection as optimal for two criteria.
result Independent projection is optimal for two criteria related to entropy and convergence.
Interactive system learns user's data insights and shows new patterns.
problem Lack of methods to elicit user knowledge and find new patterns in data.
method Information-theoretic approach using Maximum Entropy distribution and user feedback.
result System efficiently learns and shows new data insights from users.
This paper proposes an improved active learning method using classification trees.
problem Reducing the size of training sets while maintaining high accuracy in supervised learning.
method A wrapper active learning method using a classification tree to sub-sample from low-entropy regions.
result The proposed method constructs accurate classification models even with severely restricted labeled data.
Criteria ensure manifold triangulation without differentiable structure.
problem Ensuring a manifold can be triangulated without differentiable structure.
method Local coordinate chart criteria for homeomorphism verification.
result Criteria guarantee triangulation of manifolds without Delaunay property.
We introduce a geometric evolution equation of hyperbolic type, which governs the evolution of a hypersurface moving in the direction of its mean curvature vector. The flow stems from a geometrically natural action containing kinetic and internal energy terms. As the mean curvature of the hypersurface is the main drivi…
We give some general criteria of being a homeomorphism for continuous mappings of topological manifolds, as well as criteria of being a diffeomorphism for smooth mappings of smooth manifolds. As an illustration, we apply these criteria to the problems arising in two- and three-dimensional grid generation.
The study reveals flaws in pruning criteria and proposes a new assumption for better filter selection.
problem Flaws in existing pruning criteria for CNNs.
method Empirical experiments and Convolutional Weight Distribution Assumption.
result The Convolutional Weight Distribution Assumption improves filter selection in pruning.
The paper categorizes and analyzes various event-linked perpetual futures contracts.
problem Developing a risk-design framework for complex event-linked perpetual futures.
method Formal taxonomy of seven pure-form canonical variants, organized along four design axes.
result Detailed analysis of microstructure properties and limitations of various variants.
New method finds optimal hyperparameters for multiple tasks and criteria.
problem Finding optimal hyperparameters for multiple tasks and criteria.
method Multi-Task Multi Criteria (MTMC) method that provides Pareto-optimal solutions.
result The method selects optimal hyperparameters based on given criteria significance coefficients.
The study shows how discrete subgroups' critical exponents relate to their Zariski density in certain groups.
problem Understanding the density of discrete subgroups in semisimple Lie groups.
method Critical exponents and unitary representations.
result Discrete subgroups with critical exponents greater than a certain value are Zariski dense.
Improved recommendations using latent embeddings from user reviews.
problem Lack of consideration for latent embeddings in multi-criteria recommender systems.
method Utilized variational autoencoders to map user reviews into latent embeddings, which are then compressed into discrete vectors for multi-criteria recommendation.
result The proposed method significantly outperforms baselines across various datasets and evaluation measures.