Optimized KECA extracts more expressive features by optimizing kernel decomposition and Gaussian kernel parameter.
problem Improving feature extraction efficiency and robustness in kernel-based data analysis.
method Optimized KECA method using ICA framework with gradient ascent search for optimal feature extraction.
result OKECA produces more expressive features than KECA, and is more robust to kernel parameter selection.
Researchers calculate entropy of heat kernel on manifolds for very small times.
problem Estimating entropy of heat kernel on compact Riemannian manifolds for small times.
method Asymptotic expansion, polynomial expressions in curvature tensor components.
result First three coefficients of entropy expansion computed and expressed as polynomials.
New method adds user constraints to Markov chains for better data reduction.
problem No systematic framework to impose user-defined constraints on Markov chains.
method Path entropy maximization to derive transition probabilities with user constraints.
result Improved nonlinear dimensionality reduction with user-prescribed constraints.
New ICA algorithm improves source PDF estimation for better performance.
problem Inaccurate estimation of source PDFs leads to poor ICA performance.
method Entropy maximization with kernels, using global and local constraints.
result ICA-EMK outperforms competing algorithms in simulations and real-world data.
Paper characterizes embeddability of function spaces into Lp-type RKBS via metric entropy.
problem Characterizing embeddability of function spaces into Lp-type RKBS. method Establishes a connection between metric entropy growth and embeddability.
result A bound on metric entropy growth allows embedding into Lp-type RKBS. Entropy analysis via kernel methods for probabilistic inference.
problem Entropy analysis of probability distributions.
method Kernel methods and reproducing kernel Hilbert spaces for entropy estimation.
result New upper-bounds on log partition functions for probabilistic inference.
Paper proposes a new method to learn distribution kernels via entropy maximization.
problem Challenges in applying kernel methods to distribution regression tasks.
method Proposes a novel objective for unsupervised learning of data-dependent distribution kernels based on entropy maximization.
result Demonstrates the effectiveness of the learned kernel across different modalities.
In signal analysis and synthesis, linear approximation theory considers a linear decomposition of any given signal in a set of atoms, collected into a so-called dictionary. Relevant sparse representations are obtained by relaxing the orthogonality condition of the atoms, yielding overcomplete dictionaries with an exten…
Information theory provides principled ways to analyze different inference and learning problems such as hypothesis testing, clustering, dimensionality reduction, classification, among others. However, the use of information theoretic quantities as test statistics, that is, as quantities obtained from empirical data, p…
If pricing kernels are assumed non-negative then the inverse problem of finding the pricing kernel is well-posed. The constrained least squares method provides a consistent estimate of the pricing kernel. When the data are limited, a new method is suggested: relaxed maximization of the relative entropy. This estimator …
Entropy study on synthetic spaces with curvature bounds.
problem Entropy functional on synthetic spaces with curvature bounds.
method Rigorous justification of entropy formula, monotonicity, and rigidity properties; heat kernel bounds.
result Bounds for heat equation solutions on synthetic spaces.
A new nonparametric approach for system identification has been recently proposed where the impulse response is modeled as the realization of a zero-mean Gaussian process whose covariance (kernel) has to be estimated from data. In this scheme, quality of the estimates crucially depends on the parametrization of the cov…
Study bounds on kernel function entropy for finite measures.
problem Investigate bounds on the ε-entropy of kernel classes.
method Sharp upper and lower bounds for p in [1, +∞] derived from eigenvalue behavior and Mercer series convergence.
result Proves tighter bounds for general kernels compared to previous work.
The paper establishes bounds for Ricci flows using entropy and heat kernel methods.
problem Analyzing geometric and analytic properties of Ricci flows.
method Entropy and heat kernel bounds, monotonicity formula for variance of conjugate heat kernels.
result Optimal bounds for Ricci flows, including volume, heat kernel, and entropy estimates.
Researchers calculate the minimal entropy of 3-manifolds, proving it's additive.
problem Calculating the minimal entropy of 3-manifolds.
method Using the JSJ decomposition and properties of hyperbolic components.
result The cube of the minimal entropy is proportional to the simplicity volume.
GALILEO creates better mixture models for categorical data.
problem Generating high-quality mixture models for categorical data.
method Entropy-based density metric and annealing of components.
result GALILEO consistently finds high-quality clusters with the optimal number of components.
Novel kernelized Renyi's entropy improves deep learning generalization bounds.
problem Improving generalization bounds for deep learning algorithms.
method Kernelized Renyi's entropy, a new information theoretical measure.
result Theoretical bounds are tighter than current SOTA results.
The paper extends Ricci flow theory with Type-I scalar curvature bounds, proving entropy convergence and characterizing singular sets.
problem Extending Ricci flow theory with Type-I scalar curvature bounds.
method Type-I rescaling procedure and entropy analysis of conjugate heat kernels.
result Entropy of Ricci flow solutions converges to soliton entropy, characterizing singular sets.
Kernel models and DNNs perform similarly in acoustic modeling but DNNs outperform in speech recognition.
problem Comparing speech recognition performance of kernel models and DNNs.
method Comparison of kernel-based acoustic models and DNNs on speech recognition metrics.
result DNNs outperform kernel models in speech recognition but not in acoustic modeling.
Kernel ridge regression imputation with consistent variance estimation for handling missing data.
problem Handling missing data in statistical analysis.
method Kernel ridge regression imputation combined with entropy method for variance estimation.
result Root-n consistency of the imputation estimator in a Sobolev space setting.
New summary measures reveal geometric structure in weighted measures on manifolds.
problem Lack of geometric information in standard weight-only summaries.
method Heat-kernel entropy profiles, tracking nonuniformity across scales.
result Geometric effective sample size discounts nearby or duplicate particles.
Paper develops a new kernel expansion method using entropic optimal features for sparse and efficient kernel approximation.
problem Efficient kernel approximation with reduced computational cost and feature dissimilarity.
method Develops a novel optimal design maximizing entropy among kernel features, resulting in a sparse kernel expansion.
result Achieves optimal statistical accuracy with only $O(N^{ rac{1}{4}})$ features, significantly reducing time and space costs.
New entropy estimator outperforms state-of-the-art methods.
problem Estimation of entropy and mutual information in data science.
method Combines geometric and kernel approaches with local bandwidth choices.
result Asymptotic bias of the estimator is universal and pre-computable.
ED-VAE improves VAEs by explicitly including entropy components in ELBO.
problem Limitations of traditional VAEs with ELBO in generating high-quality samples and interpreting latent spaces.
method Introduces ED-VAE, a re-formulation of ELBO that includes entropy and cross-entropy components.
result Significantly enhances model flexibility and improves interpretability and generative performance.
Ancient flows by curvature powers in 2D have finite entropy.
problem Existence of non-homothetic ancient flows by powers of curvature in R2. method Determined Morse indices and kernels of the linearized operator of shrinkers. Constructed flows using unstable eigenfunctions.
result Existence of ancient flows with finite entropy.
New method quantifies multivariate redundancy using maximum entropy decompositions.
problem Elusive multivariate measures of redundancy that comply with nonnegativity and axioms.
method Maximum entropy framework, rooted tree-based decompositions of mutual information.
result Quantifies different multivariate redundancy contributions.
Kernel methods' derivatives make complex models more interpretable.
problem Interpreting complex kernel models.
method Deriving kernel functions' derivatives and applying them to various kernel methods.
result Derivatives of kernel functions can be computed and applied to improve model interpretation.
Combines BTEM and T-PLS for accurate spectral recovery and calibration.
problem Calibrating pure spectra of minority components in mixtures without prior knowledge.
method Band target entropy minimization (BTEM) and target partial least squares (T-PLS).
result Estimated amounts from BTEM-T-PLS similar to MCR-ALS on simple mixtures, superior on complex ones.
A new measure helps compute suboptimality in entropy-regularized methods.
problem Computing suboptimality in entropy-regularized variational objectives when unnormalised densities are unavailable.
method Introduced 'kernel gradient discrepancy' (KGD) to compute suboptimality explicitly.
result KGD characterizes kernel Stein discrepancy (KSD) in the standard Bayesian context and measures variational gradient size.
New IP analysis for deep neural networks using Rényi's entropy and tensor kernels.
problem Estimating mutual information in high-dimensional hidden layers of deep neural networks.
method Matrix-based Rényi's entropy coupled with tensor kernels for convolutional layers.
result First comprehensive IP analysis of large-scale DNNs and CNNs.
New distances measure mixtures of Gaussians, useful in machine learning.
problem Comparing distributions with disjoint supports.
method Schoenberg-Rao distances based on concave Rao's entropy.
result Closed-form distances for mixtures of Gaussians.
Compactifies a component by studying metric degeneration.
problem Compactify the SO(2,3)-Hitchin component.
method Study metric degeneration on a surface in pseudo-hyperbolic space.
result Establish closure in projectivized geodesic currents space.
Optimizes graph spectral density learning for large networks.
problem Ad-hoc kernel function and bandwidth selection in graph spectral techniques.
method Maximum Entropy approach to learn a smooth graph spectral density.
result Outperforms comparable iterative spectral approaches on synthetic and real graphs.
Many pattern recognition methods rely on statistical information from centered data, with the eigenanalysis of an empirical central moment, such as the covariance matrix in principal component analysis (PCA), as well as partial least squares regression, canonical-correlation analysis and Fisher discriminant analysis. R…
A new nonparametric approach for system identification has been recently proposed where the impulse response is seen as the realization of a zero--mean Gaussian process whose covariance, the so--called stable spline kernel, guarantees that the impulse response is almost surely stable. Maximum entropy properties of the …
Kernel methods estimate nonlinear principal components symmetrically.
problem Nonlinear constraints in data analysis.
method Regularized data-analytic procedure using kernel methods.
result Consistency of estimated APCs established.
Improved convergence rates for Stein Variational Gradient Descent in finite-particle settings.
problem Improving convergence rates for Stein Variational Gradient Descent in finite-particle settings.
method Analyzing the time derivative of relative entropy and splitting it into dominant and smaller parts.
result Finite-particle convergence rates of order 1/\sqrt{N} for Kernelized Stein Discrepancy and Wasserstein-2 metrics.
In this paper we introduce a new logarithmic entropy functional for the linear heat equation on complete Riemannian manifolds and prove that it is monotone decreasing on complete Riemannian manifolds with nonnegative Ricci curvature. Our results are simpler version, without Ricci flow, of R.-G. Ye's recent result (arXi…
The paper develops methods to handle missing data using regularized M-estimation in reproducing kernel Hilbert space.
problem Handling missing data in statistical analysis.
method Kernel ridge regression for imputation and maximum entropy method for propensity score estimation.
result The proposed methods achieve statistical consistency and asymptotic equivalence.
Estimates mixture entropy using pairwise distances.
problem Computing mixture entropy is challenging due to lack of closed-form solutions.
method Proposes a family of estimators based on pairwise distances.
result The proposed estimators are efficient, differentiable, and provide tight bounds on mixture entropy.
This work introduces a novel method to evaluate generative model novelty.
problem Evaluating the novelty of generative models compared to a reference model.
method Spectral approach to differential clustering and Kernel-based Entropic Novelty (KEN) score.
result The KEN score effectively detects novel modes and compares generative models.
Explains various PCA and SPCA methods with theory and applications.
problem No specific problem stated; focuses on explaining methods.
method Explains PCA, SPCA, kernel PCA, and kernel SPCA methods with theory and applications.
result Comprehensive coverage of PCA and SPCA methods with theory and applications.
The paper introduces a new framework to assess generative model uncertainty.
problem Lack of a theoretical framework for assessing generative models' generalization and uncertainty.
method Bias-variance-covariance decomposition for kernel scores, with unbiased and consistent estimators.
result Kernel-based variance and entropy for uncertainty estimation are more predictive than existing methods.
Most simple braids have positive topological entropy.
problem Understanding the topological entropy of simple braids.
method Reduction from simple braids to non-simple 3-strand braids.
result The proportion of simple braids with positive entropy approaches 100% as the number of strands increases.
Theoretical analysis of entropy approximation for Gaussian mixtures.
problem Lack of theoretical guarantees for entropy approximation of Gaussian mixtures.
method Theoretical analysis of the error between true and approximate entropy.
result The error converges to zero as the ratios of means to variances tend to infinity, providing a guarantee for high-dimensional problems.
Graph kernels improve graph similarity and learning tasks.
problem Tackling graph similarity and learning tasks.
method Proposes a message passing scheme framework for designing graph kernels.
result Kernels derived from the framework are competitive with state-of-the-art methods.
Study uses G-BSDEs to decompose pricing kernels under robust G-expectation.
problem Long-term decomposition of robust pricing kernels under G-expectation.
method Proposes and analyzes three types of quadratic G-BSDEs to decompose pricing kernels.
result Pricing kernels decomposed into four components: discounting, transitory, symmetric martingale, and volatility uncertainty.
The paper factors long-term affine pricing kernels into two components.
problem Understanding long-term behavior of affine pricing kernels.
method Long-term factorization into discounting rate and martingale component.
result Explicit identification of long bond volatility and martingale component volatility.