A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
Mathematical analysis of SNE and t-SNE for dimension reduction.
New algorithms for clustering and dimension reduction using relative von Neumann entropy.
We introduce an algorithm to locate contours of functions that are expensive to evaluate. The problem of locating contours arises in many applications, including classification, constrained optimization, and performance analysis of mechanical and dynamical systems (reliability, probability of failure, stability, etc.).…
Most simple braids have positive topological entropy.
Introduces a probabilistic framework for dimension reduction methods.
New algorithm estimates semi-continuous data density using entropy maximization.
Study reduces redundant information in multi-modal datasets.
JES optimizes expensive functions by considering joint entropy over input and output spaces.
In this paper, we explore the limitations of PCA as a dimension reduction technique and study its extension, projection pursuit (PP), which is a broad class of linear dimension reduction methods. We first discuss the relevant concepts and theorems and then apply PCA and PP (with negative standardized Shannon's entropy …
The application of standard sufficient dimension reduction methods for reducing the dimension space of predictors without losing regression information requires inverting the covariance matrix of the predictors. This has posed a number of challenges especially when analyzing high-dimensional data sets in which the numb…
The study connects Hilbert entropy to non-differentiability points of limit sets in flag spaces.
We propose a novel information-theoretic approach for Bayesian optimization called Predictive Entropy Search (PES). At each iteration, PES selects the next evaluation point that maximizes the expected information gained with respect to the global maximum. PES codifies this intractable acquisition function in terms of t…
We propose a max-pooling based loss function for training Long Short-Term Memory (LSTM) networks for small-footprint keyword spotting (KWS), with low CPU, memory, and latency requirements. The max-pooling loss training can be further guided by initializing with a cross-entropy loss trained network. A posterior smoothin…
The wavelet Maximum Entropy on the Mean (wMEM) approach to the MEG inverse problem is revisited and extended to infer brain activity from full space-time data. The resulting dimensionality increase is tackled using a collection of techniques , that includes time and space dimension reduction (using respectively wavelet…
A novel outlier score detects new road infrastructure images.
The ability of many powerful machine learning algorithms to deal with large data sets without compromise is often hampered by computationally expensive linear algebra tasks, of which calculating the log determinant is a canonical example. In this paper we demonstrate the optimality of Maximum Entropy methods in approxi…
DE-QT detects optimal Q-learning stopping points.
Information theory provides principled ways to analyze different inference and learning problems such as hypothesis testing, clustering, dimensionality reduction, classification, among others. However, the use of information theoretic quantities as test statistics, that is, as quantities obtained from empirical data, p…
New RL method nearly optimally learns policies with generative models.
Trust-region methods have yielded state-of-the-art results in policy search. A common approach is to use KL-divergence to bound the region of trust resulting in a natural gradient policy update. We show that the natural gradient and trust region optimization are equivalent if we use the natural parameterization of a st…
Rate GENERIC extends thermodynamics principles to non-equilibrium systems.
We introduce a new algorithm, called CDER, for supervised machine learning that merges the multi-scale geometric properties of Cover Trees with the information-theoretic properties of entropy. CDER applies to a training set of labeled pointclouds embedded in a common Euclidean space. If typical pointclouds correspondin…
Maximizes coding rate difference for robust, discriminative features.
Regression aims at estimating the conditional mean of output given input. However, regression is not informative enough if the conditional density is multimodal, heteroscedastic, and asymmetric. In such a case, estimating the conditional density itself is preferable, but conditional density estimation (CDE) is challeng…
Direct contextual policy search methods learn to improve policy parameters and simultaneously generalize these parameters to different context or task variables. However, learning from high-dimensional context variables, such as camera images, is still a prominent problem in many real-world tasks. A naive application o…
Unified approach for selecting summary statistics in ABC.
We analyze the problem of learning a single user's preferences in an active learning setting, sequentially and adaptively querying the user over a finite time horizon. Learning is conducted via choice-based queries, where the user selects her preferred option among a small subset of offered alternatives. These queries …
Improved sampling from complex distributions with reduced bias.
Double descent found in DRL, improving generalization with model capacity.
Improves adversarial robustness of DEQ models by regulating neural dynamics.
A quantum circuit designed for efficient statistical model preparation and training.
We consider the problem of group testing with sum observations and noiseless answers, in which we aim to locate multiple objects by querying the number of objects in each of a sequence of chosen sets. We study a probabilistic setting with entropy loss, in which we assume a joint Bayesian prior density on the locations …
Scientific and engineering processes deliver massive high-dimensional data sets that are generated as non-linear transformations of an initial state and few process parameters. Mapping such data to a low-dimensional manifold facilitates better understanding of the underlying processes, and enables their optimization. I…
We study the problem of supervised linear dimensionality reduction, taking an information-theoretic viewpoint. The linear projection matrix is designed by maximizing the mutual information between the projected signal and the class label (based on a Shannon entropy measure). By harnessing a recent theoretical result on…
Stochastic kernel based dimensionality reduction approaches have become popular in the last decade. The central component of many of these methods is a symmetric kernel that quantifies the vicinity between pairs of data points and a kernel-induced Markov chain on the data. Typically, the Markov chain is fully specified…
Predicts trainability of deep neural networks using reconstruction entropy.
Geometric structures help in understanding thermodynamics.
The Ryu-Takayanagi conjecture connects the entanglement entropy in the boundary CFT to the area of open co-dimension two minimal surfaces in the bulk. Especially in AdS(4), the latter are two-dimensional surfaces, and, thus, solutions of a Euclidean non-linear sigma model on a symmetric target space that can be reduced…
Unified probabilistic foundation for fuzzy simplicial sets in dimensionality reduction.
Improved first-order algorithm for entropy regularized OT with faster convergence.
Variance reduction methods such as SVRG and SpiderBoost use a mixture of large and small batch gradients to reduce the variance of stochastic gradients. Compared to SGD, these methods require at least double the number of operations per update to model parameters. To reduce the computational cost of these methods, we i…
The ultimate goal of optimization is to find the minimizer of a target function.However, typical criteria for active optimization often ignore the uncertainty about the minimizer. We propose a novel criterion for global optimization and an associated sequential active learning strategy using Gaussian processes.Our crit…
A new AI framework reduces costs and improves performance.
DG improves policy gradients by weighting actions with a sigmoid of advantage and surprisal.
The fastICA method is a popular dimension reduction technique used to reveal patterns in data. Here we show both theoretically and in practice that the approximations used in fastICA can result in patterns not being successfully recognised. We demonstrate this problem using a two-dimensional example where a clear struc…
Paper improves learning efficiency by focusing on effective dimensionality.
LDA-GO improves LDA for high-dimensional data via gradient optimization.
A new method for generative modeling of discrete data using geometric latent subspaces.