SDG uses optimal control to improve classifier guidance in low-density regions.
problem Inefficient guidance in low-density regions of posterior distributions.
method Integrates stochastic optimal control with Stein variational inference to compute the steepest descent direction.
result SDG improves guidance in low-density regions, outperforming standard methods.
New algorithm detects anomalies by forcing samples to displace mass in low-density regions.
problem Detecting anomalies in datasets.
method Mass Repulsing Optimal Transport (MROT) approach.
result Our algorithm improves anomaly detection over existing methods.
Neural samplers such as variational autoencoders (VAEs) or generative adversarial networks (GANs) approximate distributions by transforming samples from a simple random source---the latent space---to samples from a more complex distribution represented by a dataset. While the manifold hypothesis implies that the densit…
Improves generation of minority samples using diffusion models.
problem Generating minority samples on low-density regions of a data manifold.
method Introduces minority guidance to focus diffusion models on minority samples.
result Significantly improves generation of high-quality minority samples.
The problem of inhomogeneous cluster densities has been a long-standing issue for distance-based and density-based algorithms in clustering and anomaly detection. These algorithms implicitly assume that all clusters have approximately the same density. As a result, they often exhibit a bias towards dense clusters in th…
Efficiently clusters large datasets using low-density hyperplanes.
problem Clustering large datasets efficiently.
method Incremental estimation of low-density hyperplanes using stochastic gradient descent.
result The method automatically selects an appropriate number of clusters.
This work introduces a novel nonparametric density index defined on graphs, the Sum-over-Forests (SoF) density index. It is based on a clear and intuitive idea: high-density regions in a graph are characterized by the fact that they contain a large amount of low-cost trees with high outdegrees while low-density regions…
CycleCluster uses clustering to improve deep semi-supervised learning.
problem Difficulty in obtaining labelled data for deep learning.
method Proposes a new framework using clustering regularisation and graph-based pseudo-labels.
result Demonstrates improved predictive capability through numerical results.
Density-based clustering is the task of discovering high-density regions of entities (clusters) that are separated from each other by contiguous regions of low-density. DBSCAN is, arguably, the most popular density-based clustering algorithm. However, its cluster recovery capabilities depend on the combination of the t…
We introduce Interpolation Consistency Training (ICT), a simple and computation efficient algorithm for training Deep Neural Networks in the semi-supervised learning paradigm. ICT encourages the prediction at an interpolation of unlabeled points to be consistent with the interpolation of the predictions at those points…
Pantypes improve prototypical models by capturing diverse input distributions.
problem Prototypical models lack sufficient data representation in low density regions.
method Introducing pantypes, a sparse set of diverse objects to represent the full diversity of input distribution.
result Pantypes empower prototypical models to foster high diversity, interpretability, and fairness.
Proposes a framework to balance supervised and unsupervised learning using random matrix theory.
problem Balancing supervised and unsupervised learning in high-dimensional data.
method QLDS model with quadratic margin maximization under low density separation assumption.
result Establishes a smooth bridge between supervised and unsupervised learning methods.
We study optimal estimation for sparse principal component analysis when the number of non-zero elements is small but on the same order as the dimension of the data. We employ approximate message passing (AMP) algorithm and its state evolution to analyze what is the information theoretically minimal mean-squared error …
A new method improves flow matching by dynamically weighting density estimates.
problem High-dimensional integration inefficiency in flow matching.
method Density-weighted Dynamic Stein operators.
result Significant improvement in vector field smoothness and sampling efficiency.
End-to-end pipeline for data-driven decision making in mixed-integer optimization.
problem Data-driven decision making in mixed-integer optimization with uncertainty.
method Exploiting mixed-integer optimization-representability of machine learning methods, characterizing decision trust regions, and ensembling multiple models.
result Framework generates high-quality prescriptions and controls model robustness.
Natural images are virtually surrounded by low-density misclassified regions that can be efficiently discovered by gradient-guided search --- enabling the generation of adversarial images. While many techniques for detecting these attacks have been proposed, they are easily bypassed when the adversary has full knowledg…
We present a new method of generating mixture models for data with categorical attributes. The keys to this approach are an entropy-based density metric in categorical space and annealing of high-entropy/low-density components from an initial state with many components. Pruning of low-density components using the entro…
We present a novel cost function for semi-supervised learning of neural networks that encourages compact clustering of the latent space to facilitate separation. The key idea is to dynamically create a graph over embeddings of labeled and unlabeled samples of a training batch to capture underlying structure in feature …
GCAO improves clustering of high-dimensional data by grouping low-density boundary points.
problem Stability and accuracy of clustering in high-dimensional, non-uniform data.
method Group-level optimization with gravitational attraction and optimization.
result GCAO outperforms 11 clustering methods on multiple datasets.
Discriminatively trained neural classifiers can be trusted, only when the input data comes from the training distribution (in-distribution). Therefore, detecting out-of-distribution (OOD) samples is very important to avoid classification errors. In the context of OOD detection for image classification, one of the recen…
Graph construction is a crucial step in spectral clustering (SC) and graph-based semi-supervised learning (SSL). Spectral methods applied on standard graphs such as full-RBF, ε-graphs and k-NN graphs can lead to poor performance in the presence of proximal and unbalanced data. This is because spectral methods based…
Paper uses RNNs to design LDPC codes for binary erasure channels.
problem Designing capacity-approaching LDPC codes for binary erasure channels.
method Model Density Evolution using RNNs to determine LDPC code coefficients and structure.
result NDE improves LDPC design performance and complexity compared to differential evolution.
Proposes training neural networks to predict uncertainty for out-of-distribution inputs.
problem Poor uncertainty predictions for out-of-distribution inputs limit model robustness.
method Generates pseudo-inputs in low-density regions and trains a Bayesian framework.
result Yields robust and interpretable uncertainty predictions.
Deep learning aids ADMM-based decoding for binary linear codes.
problem Improving decoding efficiency for binary linear codes.
method Designing a decoding network based on ADMM and deep learning.
result Numerical results show improved performance compared to original ADMM.
Boost-and-Skip generates minority samples without guidance, faster and more effectively.
problem Generating minority samples in low-density regions of a data manifold.
method Boost-and-Skip approach with variance-boosted initialization and timestep skipping.
result Boost-and-Skip effectively promotes the emergence of underrepresented minority features.
L-Cool improves image and language translation by cooling low-density samples.
problem Improving translation performance on fringe samples in unsupervised domain translation.
method Performing Langevin dynamics to move low-density samples towards high-density areas.
result L-Cool enhances state-of-the-art methods in image and language translation tasks.
A new method combines MCMC results to avoid failures in parallel computing.
problem Parallel MCMC's sensitivity to subposterior sampling issues leads to failures.
method Parallel Active Inference (PAI) uses Gaussian Process (GP) surrogate modeling and active learning.
result PAI successfully combines MCMC results where previous methods fail.
A framework combines unsupervised and semi-supervised AD using synthetic anomalies.
problem Improving anomaly detection in both unsupervised and semi-supervised settings.
method Proposes a new framework that uses both known and synthetic anomalies for training.
result Synthetic anomalies improve anomaly modeling in low-density regions and provide optimal convergence guarantees.
GANF uses normalizing flows to detect anomalies in multiple time series.
problem Detecting anomalies in multiple time series with interdependencies.
method Bayesian network integration with normalizing flows for unsupervised anomaly detection.
result GANF effectively detects anomalies and identifies distribution drift in time series data.
Efficient algorithms find solutions in a rare well-connected cluster at low constraint densities.
problem Finding solutions in the symmetric binary perceptron at low density.
method Formal proof of existence of a subdominant connected cluster and application of an efficient multiscale majority algorithm.
result An efficient algorithm can find solutions in a subdominant connected cluster with high probability.
Outlier detection amounts to finding data points that differ significantly from the norm. Classic outlier detection methods are largely designed for single data type such as continuous or discrete. However, real world data is increasingly heterogeneous, where a data point can have both discrete and continuous attribute…
Improved VAE for heavy-tailed data using Student's t-distributions.
problem Over-regularization in VAEs with Gaussian priors.
method Proposed t3VAE framework with Student's t-distributions for prior, encoder, and decoder. result Significantly outperforms other models on heavy-tailed datasets.
Proposes Group Loss for deep metric learning to improve clustering and image retrieval.
problem Improving deep metric learning for better clustering and image retrieval.
method Group Loss based on label-propagation method enforcing embedding similarity across all samples of a group.
result Shows state-of-the-art results on clustering and image retrieval on several datasets.
Paper develops a method to create accurate emulators of expensive computer codes.
problem High cost and complexity of running complex computer codes.
method Active learning with Gaussian processes to construct emulators.
result Accurate and compact emulators created for expensive codes.
Random quotients of hyperbolic cubulated groups remain cubulated.
problem Understanding properties of random quotients of hyperbolic cubulated groups.
method Cubical small-cancellation theory, exponential growth of conjugacy classes, and hyperplane stabilizers' growth.
result Low-density random quotients of cubulated hyperbolic groups are cubulated and hyperbolic.
New pruning methods improve dynamic sparse training performance.
problem Improving dynamic sparse training performance.
method Design and empirical analysis of pruning criteria.
result Most pruning methods yield similar results, but magnitude-based pruning performs best in low-density regimes.
New method improves sampling from complex, multi-peaked distributions.
problem Sampling from high-dimensional, multimodal distributions using HMC.
method Combines tempered HMC with automatic tuning strategies.
result Demonstrates more effective scaling with dimension than adaptive methods.
A new method uses string method to explore diffusion models.
problem Understanding the geometry of learned distributions in diffusion models.
method String method to compute continuous paths between samples.
result The string method identifies realistic morphing sequences and transition pathways.
The adversarial training procedure proposed by Madry et al. (2018) is one of the most effective methods to defend against adversarial examples in deep neural networks (DNNs). In our paper, we shed some lights on the practicality and the hardness of adversarial training by showing that the effectiveness (robustness on t…
Study finds a linear lower bound on conformal dimension for random hyperbolic groups.
problem Understanding conformal dimension in random hyperbolic groups.
method Building undistorted round trees from lower density groups.
result Achieves a linear lower bound in l at all densities 0<d<1/2. MCBP detects boundaries in high-dimensional data using curvature.
problem Boundary detection in high-dimensional data.
method MCBP uses mean curvature to model data manifold curvature.
result MCBP improves clustering performance in complex scenarios.
Proposes a novel model-agnostic training procedure for anomaly detection incorporating known anomalies.
problem Challenges of anomaly detection, especially when only a few anomalous samples are available.
method Reformulates one-class classification as a binary classification problem, using pseudo-anomalous samples drawn from a normalizing flow model.
result Demonstrates comparable or superior performance on tasks with variable amounts of known anomalies.
In this paper we consider the problem of semi-supervised learning with deep Convolutional Neural Networks (ConvNets). Semi-supervised learning is motivated on the observation that unlabeled data is cheap and can be used to improve the accuracy of classifiers. In this paper we propose an unsupervised regularization term…
Proposes VAE-KRnet for density estimation and variational Bayes.
problem Challenges of density estimation and posterior approximation in high dimensions.
method Combines VAE and KRnet for density estimation, incorporating mutual information maximization.
result Improves posterior approximation in high-dimensional cases.
Super-AND improves unsupervised embedding learning with 89.2% accuracy on CIFAR-10.
problem Extracting good representations from data without labels.
method Super-AND uses unique losses to gather similar samples and maintain features.
result Super-AND achieves 89.2% accuracy on CIFAR-10 image classification.
The paper exposes common misconceptions about OOD detection and proposes a new framework.
problem Density-based OOD detection fails in deep learning settings.
method Proposes the OOD proxy framework to unify likelihood-ratio-based methods.
result Likelihood ratio is a principled method for OOD detection.
Develops a local Fokker--Planck geometric framework for more accurate score estimation.
problem Inaccurate estimation of score function in non-linear, state-dependent drifts.
method Local Fokker--Planck geometric framework, time change to cumulative-variance coordinate, heat-ball mean-value representations, exact high-dimensional sampling.
result Exact local mean-value representations for the score and density, improved accuracy in low-density regions.
Anomalies are intuitively easy for human experts to understand, but they are hard to define mathematically. Therefore, in order to have performance guarantees in unsupervised anomaly detection, priors need to be assumed on what the anomalies are. By contrast, active learning provides the necessary priors through approp…