arXiv research

Entropy-based GP adaptive design improves failure probability estimation.

This work improves VAEs using MCMC methods for better variational bounds.

This paper studies an entropy-based multi-objective Bayesian optimization (MBO). The entropy search is successful approach to Bayesian optimization. However, for MBO, existing entropy-based methods ignore trade-off among objectives or introduce unreliable approximations. We propose a novel entropy-based MBO called Pare…

GMM-HMMs improve malware classification compared to discrete HMMs.

New entropy-based objective for sparse coding improves learning.

Adaptive HMC improves sampling efficiency by optimizing mass matrix.

In this paper, we present a machine learning approach for estimating the number of incident wavefronts in a direction of arrival scenario. In contrast to previous works, a multilayer neural network with a cross-entropy objective is trained. Furthermore, we investigate an online training procedure that allows an adaptio…

A new active learning method for Gaussian process models.

We present a new method of generating mixture models for data with categorical attributes. The keys to this approach are an entropy-based density metric in categorical space and annealing of high-entropy/low-density components from an initial state with many components. Pruning of low-density components using the entro…

Diagnostic stroke imaging with C-arm cone-beam computed tomography (CBCT) enables reduction of time-to-therapy for endovascular procedures. However, the prolonged acquisition time compared to helical CT increases the likelihood of rigid patient motion. Rigid motion corrupts the geometry alignment assumed during reconst…

Entropy-based model for hierarchical learning from multiscale data.

ALIEN improves uncertainty estimation of language models by refining entropy-based methods.

Paper proposes a Renyi entropy-based method for tuning hierarchical topic models.

Bipartite networks provide an insightful representation of many systems, ranging from mutualistic networks of species interactions to investment networks in finance. The analysis of their topological structures has revealed the ubiquitous presence of properties which seem to characterize many - apparently different - s…

High quality reconstruction with interventional C-arm cone-beam computed tomography (CBCT) requires exact geometry information. If the geometry information is corrupted, e. g., by unexpected patient or system movement, the measured signal is misplaced in the backprojection operation. With prolonged acquisition times of…

When a spacetime has boundaries, the entangling surface does not have to be necessarily compact and it may have boundaries as well. Then there appear a new, boundary, contribution to the entanglement entropy due to the intersection of the entangling surface with the boundary of the spacetime. We study the boundary cont…

Effective and intelligent exploration has been an unresolved problem for reinforcement learning. Most contemporary reinforcement learning relies on simple heuristic strategies such as $ε$-greedy exploration or adding Gaussian noise to actions. These heuristics, however, are unable to intelligently distinguish the well …

Paper models entropy-based impact of soft errors on neural network inference.

MPPN network improves long-term time series forecasting accuracy.

New distances measure mixtures of Gaussians, useful in machine learning.

The scientific method relies on the iterated processes of inference and inquiry. The inference phase consists of selecting the most probable models based on the available data; whereas the inquiry phase consists of using what is known about the models to select the most relevant experiment. Optimizing inquiry involves …

This is full length article (draft version) where problem number of topics in Topic Modeling is discussed. We proposed idea that Renyi and Tsallis entropy can be used for identification of optimal number in large textual collections. We also report results of numerical experiments of Semantic stability for 4 topic mode…

Entropy based ideas find wide-ranging applications in finance for calibrating models of portfolio risk as well as options pricing. The abstracted problem, extensively studied in the literature, corresponds to finding a probability measure that minimizes relative entropy with respect to a specified measure while satisfy…

An empirical investigation of the interaction of sample size and discretization - in this case the entropy-based method CAIM (Class-Attribute Interdependence Maximization) - was undertaken to evaluate the impact and potential bias introduced into data mining performance metrics due to variation in sample size as it imp…

Entropy-based decoding improves DLM sampling efficiency.

A novel outlier score detects new road infrastructure images.

Study generalizes Picard iteration for nonlinear PDEs, deriving bounds on error.

Meta-learning approaches have been proposed to tackle the few-shot learning problem.Typically, a meta-learner is trained on a variety of tasks in the hopes of being generalizable to new tasks. However, the generalizability on new tasks of a meta-learner could be fragile when it is over-trained on existing tasks during …

FEDS distills LIC model knowledge into a lightweight student for efficient compression.

Unified framework for network model assessment using maximum entropy.

Enhances model compression with multi-teacher knowledge distillation.

This work introduces a novel method to evaluate generative model novelty.

The paper extends explainability methods to uncertainty-aware models, revealing feature impacts on predictive entropy and likelihood.

The study proves compactness and existence of entropy minimizers for self-shrinking surfaces.

Introduces RPU to explain randomization preference in dynamic settings.

Proposes a new batch selection method for multi-label classification.

Local Sobolev inequality on Ricci flows with applications.

MESSY estimation recovers symbolic density functions from samples using maximum entropy.

A framework uses free probability to analyze Transformer models.

We describe a method for selecting relevant new training data for the LSTM-based domain selection component of our personal assistant system. Adding more annotated training data for any ML system typically improves accuracy, but only if it provides examples not already adequately covered in the existing data. However, …

We present a procedure for effective estimation of entropy and mutual information from small-sample data, and apply it to the problem of inferring high-dimensional gene association networks. Specifically, we develop a James-Stein-type shrinkage estimator, resulting in a procedure that is highly efficient statistically …

Paper introduces a new measure combining entropy and Gini index.

CAT framework improves AI medical screening fairness and reliability.

This work enhances collaborative inference privacy by minimizing conditional entropy and boosting robustness against model inversion attacks.

SOAR improves deep networks' robustness against adversarial examples.

Clustering evaluation measures are frequently used to evaluate the performance of algorithms. However, most measures are not properly normalized and ignore some information in the inherent structure of clusterings. We model the relation between two clusterings as a bipartite graph and propose a general component-based …

Exploration is an extremely challenging problem in reinforcement learning, especially in high dimensional state and action spaces and when only sparse rewards are available. Effective representations can indicate which components of the state are task relevant and thus reduce the dimensionality of the space to explore.…

The study evaluates different probability models for uncertainty visualization using entropy calculations.