arXiv research

Entropy-based GP adaptive design improves failure probability estimation.

Paper proposes a Renyi entropy-based method for tuning hierarchical topic models.

GMM-HMMs improve malware classification compared to discrete HMMs.

New entropy-based objective for sparse coding improves learning.

ALIEN improves uncertainty estimation of language models by refining entropy-based methods.

Study generalizes Picard iteration for nonlinear PDEs, deriving bounds on error.

Paper models entropy-based impact of soft errors on neural network inference.

We present a procedure for effective estimation of entropy and mutual information from small-sample data, and apply it to the problem of inferring high-dimensional gene association networks. Specifically, we develop a James-Stein-type shrinkage estimator, resulting in a procedure that is highly efficient statistically …

MESSY estimation recovers symbolic density functions from samples using maximum entropy.

We present a new method of generating mixture models for data with categorical attributes. The keys to this approach are an entropy-based density metric in categorical space and annealing of high-entropy/low-density components from an initial state with many components. Pruning of low-density components using the entro…

This paper studies an entropy-based multi-objective Bayesian optimization (MBO). The entropy search is successful approach to Bayesian optimization. However, for MBO, existing entropy-based methods ignore trade-off among objectives or introduce unreliable approximations. We propose a novel entropy-based MBO called Pare…

Entropy-based model for hierarchical learning from multiscale data.

New estimator for joint entropy outperforms existing methods in various distributions.

Diagnostic stroke imaging with C-arm cone-beam computed tomography (CBCT) enables reduction of time-to-therapy for endovascular procedures. However, the prolonged acquisition time compared to helical CT increases the likelihood of rigid patient motion. Rigid motion corrupts the geometry alignment assumed during reconst…

Bipartite networks provide an insightful representation of many systems, ranging from mutualistic networks of species interactions to investment networks in finance. The analysis of their topological structures has revealed the ubiquitous presence of properties which seem to characterize many - apparently different - s…

Unified framework for network model assessment using maximum entropy.

The objective in statistical Optimal Transport (OT) is to consistently estimate the optimal transport plan/map solely using samples from the given source and target marginal distributions. This work takes the novel approach of posing statistical OT as that of learning the transport plan's kernel mean embedding from sam…

We consider the problem of defining the significance of an itemset. We say that the itemset is significant if we are surprised by its frequency when compared to the frequencies of its sub-itemsets. In other words, we estimate the frequency of the itemset from the frequencies of its sub-itemsets and compute the deviatio…

This work improves VAEs using MCMC methods for better variational bounds.

In this paper, we present a machine learning approach for estimating the number of incident wavefronts in a direction of arrival scenario. In contrast to previous works, a multilayer neural network with a cross-entropy objective is trained. Furthermore, we investigate an online training procedure that allows an adaptio…

The scientific method relies on the iterated processes of inference and inquiry. The inference phase consists of selecting the most probable models based on the available data; whereas the inquiry phase consists of using what is known about the models to select the most relevant experiment. Optimizing inquiry involves …

This is full length article (draft version) where problem number of topics in Topic Modeling is discussed. We proposed idea that Renyi and Tsallis entropy can be used for identification of optimal number in large textual collections. We also report results of numerical experiments of Semantic stability for 4 topic mode…

High quality reconstruction with interventional C-arm cone-beam computed tomography (CBCT) requires exact geometry information. If the geometry information is corrupted, e. g., by unexpected patient or system movement, the measured signal is misplaced in the backprojection operation. With prolonged acquisition times of…

Entropy based ideas find wide-ranging applications in finance for calibrating models of portfolio risk as well as options pricing. The abstracted problem, extensively studied in the literature, corresponds to finding a probability measure that minimizes relative entropy with respect to a specified measure while satisfy…

An empirical investigation of the interaction of sample size and discretization - in this case the entropy-based method CAIM (Class-Attribute Interdependence Maximization) - was undertaken to evaluate the impact and potential bias introduced into data mining performance metrics due to variation in sample size as it imp…

A novel outlier score detects new road infrastructure images.

A new active learning method for Gaussian process models.

FEDS distills LIC model knowledge into a lightweight student for efficient compression.

Estimating the entropy based on data is one of the prototypical problems in distribution property testing and estimation. For estimating the Shannon entropy of a distribution on $S$ elements with independent samples, [Paninski2004] showed that the sample complexity is sublinear in $S$, and [Valiant--Valiant2011] showed…

Measures neural network complexity using tangent space diversity.

Distances between probability distributions that take into account the geometry of their sample space,like the Wasserstein or the Maximum Mean Discrepancy (MMD) distances have received a lot of attention in machine learning as they can, for instance, be used to compare probability distributions with disjoint supports. …

This work introduces a novel method to evaluate generative model novelty.

Study predicts price predictability in ultra-high frequency financial data using entropy tests.

New method for uncertainty analysis in TabPFN, a state-of-the-art tabular transformer.

New method shows data-driven causal studies can be misleading.

The study proves compactness and existence of entropy minimizers for self-shrinking surfaces.

Introduces RPU to explain randomization preference in dynamic settings.

Local Sobolev inequality on Ricci flows with applications.

Bayesian active learning improves holistic educational assessments.

A framework uses free probability to analyze Transformer models.

We describe a method for selecting relevant new training data for the LSTM-based domain selection component of our personal assistant system. Adding more annotated training data for any ML system typically improves accuracy, but only if it provides examples not already adequately covered in the existing data. However, …

Distributed descent-based methods are an essential toolset to solving optimization problems in multi-agent system scenarios. Here the agents seek to optimize a global objective function through mutual cooperation. Oftentimes, cooperation is achieved over a wireless communication network that is prone to delays and erro…

The theory of Compressed Sensing (CS) asserts that an unknown signal $x\in\mathbb{R}^p$ can be accurately recovered from an underdetermined set of $n$ linear measurements with $n\ll p$, provided that $x$ is sufficiently sparse. However, in applications, the degree of sparsity $\|x\|_0$ is typically unknown, and the pro…

Paper introduces a new measure combining entropy and Gini index.

CAT framework improves AI medical screening fairness and reliability.

SOAR improves deep networks' robustness against adversarial examples.

Clustering evaluation measures are frequently used to evaluate the performance of algorithms. However, most measures are not properly normalized and ignore some information in the inherent structure of clusterings. We model the relation between two clusterings as a bipartite graph and propose a general component-based …

SSLfmm package improves semi-supervised learning by incorporating informative missingness in finite mixture models.