arXiv research

Algorithm finds function contours using multiple approximations.

Mathematical analysis of SNE and t-SNE for dimension reduction.

New algorithms for clustering and dimension reduction using relative von Neumann entropy.

A new covariance estimator reduces dimensionality in high-dimensional undersized samples.

Most simple braids have positive topological entropy.

New method adds user constraints to Markov chains for better data reduction.

Introduces a probabilistic framework for dimension reduction methods.

New algorithm estimates semi-continuous data density using entropy maximization.

Study reduces redundant information in multi-modal datasets.

JES optimizes expensive functions by considering joint entropy over input and output spaces.

The paper compares PCA and PP for scRNA sequencing data.

FastICA fails to reveal clear patterns in data.

Extended wMEM approach for MEG inverse problem using wavelet and spatial filters.

The study connects Hilbert entropy to non-differentiability points of limit sets in flag spaces.

We propose a novel information-theoretic approach for Bayesian optimization called Predictive Entropy Search (PES). At each iteration, PES selects the next evaluation point that maximizes the expected information gained with respect to the global maximum. PES codifies this intractable acquisition function in terms of t…

We propose a max-pooling based loss function for training Long Short-Term Memory (LSTM) networks for small-footprint keyword spotting (KWS), with low CPU, memory, and latency requirements. The max-pooling loss training can be further guided by initializing with a cross-entropy loss trained network. A posterior smoothin…

A novel outlier score detects new road infrastructure images.

The ability of many powerful machine learning algorithms to deal with large data sets without compromise is often hampered by computationally expensive linear algebra tasks, of which calculating the log determinant is a canonical example. In this paper we demonstrate the optimality of Maximum Entropy methods in approxi…

DE-QT detects optimal Q-learning stopping points.

Information theory provides principled ways to analyze different inference and learning problems such as hypothesis testing, clustering, dimensionality reduction, classification, among others. However, the use of information theoretic quantities as test statistics, that is, as quantities obtained from empirical data, p…

New RL method nearly optimally learns policies with generative models.

Trust-region methods and natural gradients are equivalent in certain policy search scenarios.

Rate GENERIC extends thermodynamics principles to non-equilibrium systems.

We introduce a new algorithm, called CDER, for supervised machine learning that merges the multi-scale geometric properties of Cover Trees with the information-theoretic properties of entropy. CDER applies to a training set of labeled pointclouds embedded in a common Euclidean space. If typical pointclouds correspondin…

Maximizes coding rate difference for robust, discriminative features.

Regression aims at estimating the conditional mean of output given input. However, regression is not informative enough if the conditional density is multimodal, heteroscedastic, and asymmetric. In such a case, estimating the conditional density itself is preferable, but conditional density estimation (CDE) is challeng…

Direct contextual policy search methods learn to improve policy parameters and simultaneously generalize these parameters to different context or task variables. However, learning from high-dimensional context variables, such as camera images, is still a prominent problem in many real-world tasks. A naive application o…

Unified approach for selecting summary statistics in ABC.

We analyze the problem of learning a single user's preferences in an active learning setting, sequentially and adaptively querying the user over a finite time horizon. Learning is conducted via choice-based queries, where the user selects her preferred option among a small subset of offered alternatives. These queries …

Improved sampling from complex distributions with reduced bias.

NGCA identifies non-Gaussian components in multidimensional data.

Double descent found in DRL, improving generalization with model capacity.

Improves adversarial robustness of DEQ models by regulating neural dynamics.

A quantum circuit designed for efficient statistical model preparation and training.

We consider the problem of group testing with sum observations and noiseless answers, in which we aim to locate multiple objects by querying the number of objects in each of a sequence of chosen sets. We study a probabilistic setting with entropy loss, in which we assume a joint Bayesian prior density on the locations …

Scientific and engineering processes deliver massive high-dimensional data sets that are generated as non-linear transformations of an initial state and few process parameters. Mapping such data to a low-dimensional manifold facilitates better understanding of the underlying processes, and enables their optimization. I…

New sparsity operator reduces variance reduction methods' computational cost.

We study the problem of supervised linear dimensionality reduction, taking an information-theoretic viewpoint. The linear projection matrix is designed by maximizing the mutual information between the projected signal and the class label (based on a Shannon entropy measure). By harnessing a recent theoretical result on…

Predicts trainability of deep neural networks using reconstruction entropy.

Geometric structures help in understanding thermodynamics.

The Ryu-Takayanagi conjecture connects the entanglement entropy in the boundary CFT to the area of open co-dimension two minimal surfaces in the bulk. Especially in AdS(4), the latter are two-dimensional surfaces, and, thus, solutions of a Euclidean non-linear sigma model on a symmetric target space that can be reduced…

Unified probabilistic foundation for fuzzy simplicial sets in dimensionality reduction.

Improved first-order algorithm for entropy regularized OT with faster convergence.

The ultimate goal of optimization is to find the minimizer of a target function.However, typical criteria for active optimization often ignore the uncertainty about the minimizer. We propose a novel criterion for global optimization and an associated sequential active learning strategy using Gaussian processes.Our crit…

Three LF training criteria improve neural network acoustic models without cross-entropy pre-training.

The paper explores how neural networks avoid overfitting by learning from flat minima.

A new AI framework reduces costs and improves performance.

DG improves policy gradients by weighting actions with a sigmoid of advantage and surprisal.