Proposes a Gaussian process for Koopman mode decomposition.
problem Estimating Koopman mode decomposition quantities and latent variables.
method Unsupervised Gaussian process for simultaneous estimation.
result Efficient parameter estimation through low-rank approximations.
Study guarantees convergence of mean shift mode estimation.
problem Ensuring reliable mode estimation in KDE using mean shift.
method Utilizes Łojasiewicz inequality to prove convergence rate.
result Extends convergence guarantees to biweight kernel.
New method uses random projections to estimate densities and modes efficiently.
problem Estimating densities and modes from sparse representations.
method Expand-and-sparsify representations followed by linear function and mode recovery algorithms.
result Optimal rates for density and mode estimation achieved.
Proposes MPCA for robust PCA using mode estimation.
problem Outliers sensitivity in PCA.
method Modal Principal Component Analysis (MPCA) based on mode estimation.
result MPCA shows advantages over conventional methods.
Modes and ridges of the probability density function behind observed data are useful geometric features. Mode-seeking clustering assigns cluster labels by associating data samples with the nearest modes, and estimation of density ridges enables us to find lower-dimensional structures hidden in data. A key technical cha…
Estimates mode from partial feedback, improving AI learning pipelines.
problem Estimating the mode of a distribution with partial feedback.
method Entropy coding, coarse sufficient statistics, bandit algorithms.
result Statistically and computationally efficient solution to mode estimation.
Bayesian taut splines estimate modes in probability densities.
problem Estimating the number of modes in probability density functions.
method Bayesian inference with flexible kernel estimators and compositional splines.
result The new method provides more accurate results than traditional approaches.
Develops an MS-inspired algorithm for regression mode finding and space partitioning.
problem Finding local modes of regression functions and partitioning input space.
method Mean-shift-inspired algorithm for iterative gradient ascent.
result Proves convergence and rates of convergence for estimated local modes.
Deep learning helps remove secondary B-mode polarization to detect primordial gravitational waves.
problem Removing secondary B-mode polarization from CMB data to detect primordial gravitational waves. method Applied deep learning (ResUNet-CMB) to estimate and remove multiple sources of secondary B-mode polarization. result Deep learning can produce nearly optimal, unbiased estimates of the amplitude of primordial gravitational waves.
Estimates modes and ridges in mixed Euclidean and directional spaces.
problem Estimating local modes and density ridges in product spaces combining Euclidean and directional metrics.
method Extends mean shift algorithm to product spaces, addressing challenges in generalization.
result Established convergence of the proposed methods and demonstrated effectiveness on real-world datasets.
Study finds the number of modes in Gaussian kernel density estimators scales with sqrt(β log β).
problem Determining the number of clusters in Transformers.
method Used Kac-Rice formula and Edgeworth expansion to prove scaling.
result The expected number of modes scales as Θ(√(β log β)).
For dynamical systems that can be modelled as asymptotically stable linear systems forced by Gaussian noise, this paper develops methods to infer or estimate their modes from observations in real time. The modes can be real or complex. For a real mode, we wish to infer its damping rate and mode shape. For a complex mod…
The paper provides robustness guarantees for mode estimation in bandits.
problem Understanding robustness in mode estimation under adversarial data contamination.
method Simple randomization and theoretical analysis of multi-armed bandits.
result Regret guarantees for various modal bandit problems.
We propose an estimation method for the conditional mode when the conditioning variable is high-dimensional. In the proposed method, we first estimate the conditional density by solving quantile regressions multiple times. We then estimate the conditional mode by finding the maximum of the estimated conditional density…
The paper analyzes zero modes on product manifolds and provides estimates for their norms.
problem Analyzing zero modes on product Riemannian manifolds.
method Using the zero mode equation and non-increasing condition on |\varphi|, the paper derives estimates for the norms of the vector field A.
result The derived estimates are sharp in even dimensions and provide insights into the behavior of zero modes.
The paper studies kernel smoothing and mean shift for directional data, deriving convergence rates and mode estimation.
problem Statistical and computational problems of kernel smoothing for directional data.
method Generalization of mean shift to directional data, derivation of convergence rates, and investigation of mode estimation.
result Statistical convergence rates of directional KDE and its derivatives, ascending property of directional mean shift, and mode estimation.
One of the popular measures of central tendency that provides better representation and interesting insights of the data compared to the other measures like mean and median is the metric mode. If the analytical form of the density function is known, mode is an argument of the maximum value of the density function and o…
Study shows annealing with adaptive schedule reduces mode collapse in NFs for parameter estimation.
problem Mode collapse in normalizing flows for multimodal distributions.
method Annealing with an adaptive schedule based on effective sample size (ESS).
result Our approach reduces mode collapse and converges marginal likelihood faster than MCMC methods.
New model generates data without mode collapse or mixture.
problem Mode collapse and mode mixture in generative models.
method Inspired by AE-OT, improved sample generation algorithm.
result Free from mode collapse and mixture issues.
Density mode clustering is a nonparametric clustering method. The clusters are the basins of attraction of the modes of a density estimator. We study the risk of mode-based clustering. We show that the clustering risk over the cluster cores --- the regions where the density is high --- is very small even in high dimens…
Mode clustering is a nonparametric method for clustering that defines clusters using the basins of attraction of a density estimator's modes. We provide several enhancements to mode clustering: (i) a soft variant of cluster assignment, (ii) a measure of connectivity between clusters, (iii) a technique for choosing the …
Estimates mode of discrete distributions with fewer samples.
problem Identifying the mode of a discrete distribution with high probability.
method Generalizes PPR martingale confidence sequences to handle multiple modes.
result PPR-1v1 stopping rule is asymptotically optimal and significantly more efficient.
In industrial systems, certain process variables that need to be monitored for detecting faults are often difficult or impossible to measure. Soft sensor techniques are widely used to estimate such difficult-to-measure process variables from easy-to-measure ones. Soft sensor modeling requires training datasets includin…
Algorithm identifies nearest mode in noisy data.
problem Identifying the point with the minimum k-th nearest neighbor distance in unknown multivariate probability density.
method Sequential learning algorithm using noisy oracle queries to adaptively decide which points to query.
result Upper bounds on query complexity show significant improvement over baselines.
Learning an optimal policy from a multi-modal reward function is a challenging problem in reinforcement learning (RL). Hierarchical RL (HRL) tackles this problem by learning a hierarchical policy, where multiple option policies are in charge of different strategies corresponding to modes of a reward function and a gati…
BDMBC clusters data with varying densities using a new PLLS measure.
problem Finding clusters with varying densities in data.
method Bagged k-distance with PLLS for mode estimation. result BDMBC achieves optimal convergence rates for mode and level set estimation.
We consider the problem of adaptively PAC-learning a probability distribution P's mode by querying an oracle for information about a sequence of i.i.d. samples X1,X2,… generated from P. We consider two different query models: (a) each query is an index i for which the oracle reveals…
We introduce the functional mean-shift algorithm, an iterative algorithm for estimating the local modes of a surrogate density from functional data. We show that the algorithm can be used for cluster analysis of functional data. We propose a test based on the bootstrap for the significance of the estimated local modes …
Recently, a number of statistical problems have found an unexpected solution by inspecting them through a "modal point of view". These include classical tasks such as clustering or regression. This has led to a renewed interest in estimation and inference for the mode. This paper offers an extensive survey of the tradi…
Optimal kernel improves estimation accuracy in modal statistical methods.
problem Estimation accuracy of kernel-based modal statistical methods depends on the kernel used.
method The study theoretically shows an optimal kernel that minimizes asymptotic error criterion.
result An optimal kernel minimizes the error criterion when using an optimal bandwidth.
Hybrid model for multimodal distributions using diffusion and classification.
problem Sampling from multimodal distributions with correct proportions.
method Divide-and-conquer strategy: identify modes, train classifiers, diffusion models, bridge sampling.
result Framework effectively handles multimodal distributions in high dimensions.
A natural way to characterize the cluster structure of a dataset is by finding regions containing a high density of data. This can be done in a nonparametric way with a kernel density estimate, whose modes and hence clusters can be found using mean-shift algorithms. We describe the theory and practice behind clustering…
Proposes hinge-Wasserstein to improve uncertainty estimation in regression tasks.
problem Estimating multimodal aleatoric uncertainty in regression tasks from images.
method Regression-by-classification paradigm with hinge-Wasserstein loss.
result Hinge-Wasserstein loss improves uncertainty estimation on challenging tasks.
Mean shift clustering finds the modes of the data probability density by identifying the zero points of the density gradient. Since it does not require to fix the number of clusters in advance, the mean shift has been a popular clustering algorithm in various application fields. A typical implementation of the mean shi…
Study uses open data to improve traffic emissions estimation.
problem Estimating accurate link level traffic emissions.
method Data-driven framework integrating MOVES, GPS, OSM, and satellite imagery.
result Neural network reduces RMSE by over 50% for key pollutants.
Improved KL divergence estimators for normalizing flows lead to faster convergence and better approximations.
problem Estimating KL divergences for normalizing flows efficiently and accurately.
method Path-gradient estimators for reverse and forward KL divergences.
result Path-gradient estimators lead to faster convergence and better approximation results.
Framework predicts remaining useful life of DSH subsystems under unknown failure modes.
problem Predicting remaining useful life of DSH subsystems with unknown failure modes.
method Unsupervised framework using mixture of Gaussian regressions and Expectation-Maximization algorithm.
result Improved prediction accuracy and interpretability of RUL.
A new method computes gradients without backpropagation.
problem Optimization of machine learning models.
method Forward mode automatic differentiation to compute gradients.
result Forward gradient is an unbiased estimate of the gradient, eliminating the need for backpropagation.
Backpropagation-free trunk training improves model performance on various benchmarks.
problem Memory inefficiency and noisy gradient estimates in deep network training.
method Split Forward Gradient (Split-FG) method that splits network into trunk and head, estimating only trunk gradient.
result Split-FG achieves better performance than pure forward-gradient training and backpropagation on various benchmarks.
End-to-end training of DBMs with improved gradient estimation.
problem Biased gradient estimation in DBMs, especially with high-dimensional states.
method Unbiased contrastive divergence using MH coupling and local mode initialization.
result End-to-end training of DBMs without greedy pretraining, achieving FID score of 10.33 for MNIST.
DDSME outperforms SME in estimating multimodal distributions.
problem Efficiency of score matching in multimodal distributions.
method Diffusion-based denoising score matching (DDSME) compared to vanilla score matching (SME).
result DDSME avoids the error bound deterioration of SME with increasing mode separation.
Study on conditioning Gaussian measures on nonlinear observations, including representer theorem and mode estimation.
problem Conditioning Gaussian measures on nonlinear observations in Bayesian inference and machine learning.
method Representer theorem, novel mode definition, maximum a posteriori estimation, Laplace approximation.
result Identification of infinite-dimensional Gaussian and finite-dimensional non-Gaussian components in conditioned measures.
A new method constrains PARAFAC2 for better pattern recovery.
problem Challenges in analyzing multi-way measurements with variations across one mode.
method AO-ADMM approach to fit PARAFAC2 model with flexible constraints.
result The proposed method allows for flexible constraints, recovers patterns accurately, and is computationally efficient.
cKAM improves adaptive sampling by incorporating a cyclical stepsize scheme.
problem Adaptive Metropolis algorithms can get stuck in local modes.
method cKAM uses a cyclical stepsize scheme to encourage exploration and escape from local modes.
result cKAM successfully escapes local modes and converges to the true posterior distribution.
We advocate Laplacian K-modes for joint clustering and density mode finding, and propose a concave-convex relaxation of the problem, which yields a parallel algorithm that scales up to large datasets and high dimensions. We optimize a tight bound (auxiliary function) of our relaxation, which, at each iteration, amounts…
Gradient-guided nested sampling improves posterior inference efficiency.
problem Efficiently sampling from complex posterior distributions.
method Gradient-guided nested sampling combining differentiable programming, Hamiltonian slice sampling, clustering, mode separation, dynamic nested sampling, and parallelization.
result Significantly faster mode discovery and more accurate partition function estimates.
Tensor decompositions have rich applications in statistics and machine learning, and developing efficient, accurate algorithms for the problem has received much attention recently. Here, we present a new method built on Kruskal's uniqueness theorem to decompose symmetric, nearly orthogonally decomposable tensors. Unlik…
The most direct way to express arbitrary dependencies in datasets is to estimate the joint distribution and to apply afterwards the argmax-function to obtain the mode of the corresponding conditional distribution. This method is in practice difficult, because it requires a global optimization of a complicated function,…