New discrepancy function compares discrete probability measures considering space geometry.
problem Comparing discrete probability measures in a geometrically meaningful way.
method Proposes the Fourier Discrepancy Function, proving convexity, differentiability, and providing gradient formula.
result Proves the Fourier Discrepancy is convex, twice differentiable, and provides an explicit gradient formula.
Optimizes kernel discrepancies by selecting subsets efficiently.
problem Improving kernel discrepancies for QMC methods.
method Introduces a novel subset selection algorithm for kernel discrepancies.
result Efficiently generates low-discrepancy samples from various distributions.
This paper defines the notion of class discrepancy for families of functions. It shows that low discrepancy classes admit small offline and streaming coresets. We provide general techniques for bounding the class discrepancy of machine learning problems. As corollaries of the general technique we bound the discrepancy …
Semi-parametric framework for nonlinear system identification
problem Nonlinear system identification
method Orthogonal Gaussian process regression
result Interpretable models from incomplete physics
Sliced kernelized Stein discrepancy improves goodness-of-fit tests and model learning in high dimensions.
problem The curse-of-dimensionality in kernelized Stein discrepancy (KSD).
method Sliced Stein discrepancy and its scalable variants using optimal one-dimensional projections.
result Significantly outperforms KSD and baselines in goodness-of-fit tests and improves model learning.
Generative neural network simulates characteristic functions.
problem Simulating from characteristic functions inaccessible in closed form.
method Generative neural network with Maximum-Mean-Discrepancy loss.
result Universal algorithm independent of dimensionality and function properties.
Stein discrepancy improves UDA performance in low-data scenarios.
problem Improving model performance on unlabeled target domains with limited data.
method Proposes a novel UDA framework using Stein discrepancy, an asymmetric measure that depends on the target distribution through its score function.
result Consistently outperforms prior UDA approaches under limited target data across multiple benchmarks.
Expands Bayesian experiment design framework to account for model discrepancies.
problem Model misspecification in Bayesian optimal experiment design.
method Introduces Expected General Information Gain and Expected Discriminatory Information criteria.
result Demonstrates improved robustness and detection capabilities in experiment design.
Bayesian inference uses Stein discrepancy for robustness in intractable likelihoods.
problem Intractable likelihoods in Bayesian inference.
method Generalised Bayesian inference with Stein discrepancy as the loss function.
result Robust generalised posteriors with closed form or accessible using MCMC.
We present a new method for evaluating and training unnormalized density models. Our approach only requires access to the gradient of the unnormalized model's log-density. We estimate the Stein discrepancy between the data density p(x) and the model density q(x) defined by a vector function of the data. We paramete…
A new framework improves kernel Stein discrepancy tests for validating distributions.
problem Improving goodness-of-fit testing for non-normal distributions.
method Introducing Sf-KSD, a unifying framework for studying Stein operators in KSD-based tests.
result Sf-KSD guides the development of new tests and outperforms existing methods.
We show in this note that the Sobolev Discrepancy introduced in Mroueh et al in the context of generative adversarial networks, is actually the weighted negative Sobolev norm ∣∣.∣∣H˙−1(νq), that is known to linearize the Wasserstein W2 distance and plays a fundamental role in the dynamic formulation of…
In supervised clustering, standard techniques for learning a pairwise dissimilarity function often suffer from a discrepancy between the training and clustering objectives, leading to poor cluster quality. Rectifying this discrepancy necessitates matching the procedure for training the dissimilarity function to the clu…
We propose a framework for synthesis of geological images based on an exemplar image. We synthesize new realizations such that the discrepancy in the patch distribution between the realizations and the exemplar image is minimized. Such discrepancy is quantified using a kernel method for two-sample test called maximum m…
Generative adversarial networks (GANs) generate data based on minimizing a divergence between two distributions. The choice of that divergence is therefore critical. We argue that the divergence must take into account the hypothesis set and the loss function used in a subsequent learning task, where the data generated …
Two methods using low-discrepancy points improve data compression for neural networks.
problem Efficiently compress large datasets for neural network training.
method Two methods based on low-discrepancy points: digital nets with averaging and clustering.
result Second method outperforms supercompress in compression error and neural network accuracy.
USD algorithm transports distributions with or without mass conservation.
problem Transporting distributions with different masses.
method Particle descent algorithm using Sobolev-Fisher discrepancy.
result USD converges to target distribution in MMD sense.
A new method trains discrete EBMs without sampling.
problem Training EBMs on discrete spaces is hard.
method Energy Discrepancy (ED), a contrastive loss.
result ED offers theoretical guarantees for various perturbation types.
We consider the problem of improving the efficiency of randomized Fourier feature maps to accelerate training and testing speed of kernel methods on large datasets. These approximate feature maps arise as Monte Carlo approximations to integral representations of shift-invariant kernel functions (e.g., Gaussian kernel).…
Deep learning model approximates stochastic responses.
problem Approximating stochastic responses using neural networks.
method Generative neural network with conditional maximum mean discrepancy (CMMD) loss.
result Excellent performance on benchmark problems.
A new ensemble filter uses transport maps and MMD optimization for high-dimensional data assimilation.
problem High-dimensional data assimilation challenges in ensemble filtering.
method Optimized Maximum Mean Discrepancy (MMD) for transport map construction.
result Significant improvement in robustness and posterior approximation.
This paper introduces localized discrepancy theories for unsupervised domain adaptation.
problem Improving generalization bounds for unsupervised domain adaptation.
method Localized discrepancies defined on the hypothesis space after localization, leading to smaller and asymmetric values.
result Improved generalization bounds and sample complexity reduction.
A new metric detects non-Markovian states in partially observable environments.
problem Learning state representations in partially observable environments.
method Introducing the λ-discrepancy metric to detect non-Markovian states. result The λ-discrepancy is zero for Markov processes and non-zero for partially observable environments. New algorithm improves knowledge transfer in dynamic decision-making.
problem Utilizing data from existing ventures to improve decision-making in new ventures.
method Proposes Transferred Fitted Q-Iteration algorithm for estimating optimal action-state function Q∗. result Significantly improved final learning error of Q∗ function. A test for comparing function samples using MMD.
problem Testing if two functional data samples come from the same distribution.
method Maximum Mean Discrepancy (MMD) for functional data, with theoretical scaling analysis.
result The proposed test is effective and robust to functional reconstructions.
A new test statistic measures discrepancy between conditional distributions.
problem Measuring the discrepancy between two conditional distributions.
method Proposes a Bregman matrix divergence-based statistic that avoids explicit distribution estimation.
result The new statistic inherits high-order statistics and demonstrates utility in multi-task learning, concept drift detection, and feature selection.
Given i.i.d samples from some unknown continuous density on hyper-rectangle [0,1]d, we attempt to learn a piecewise constant function that approximates this underlying density non-parametrically. Our density estimate is defined on a binary split of [0,1]d and built up sequentially according to discrepancy crite…
Fourier representation improves KSD for infinite-dimensional data.
problem Applying KSD to infinite-dimensional data.
method Combining measure equations with kernel methods for a Fourier representation of KSD.
result KSD can separate measures in infinite-dimensional Hilbert spaces.
The article introduces practical estimators for kernel discrepancies.
problem Estimating kernel discrepancies accurately and efficiently.
method Presented various estimators for MMD, HSIC, and KSD, including V-statistics, U-statistics, and incomplete U-statistics. Stressed the importance of kernel bandwidth and introduced adaptive estimators.
result Adaptive estimators combining multiple estimators with various kernels address the problem of kernel selection.
Paper proposes approximate Stein classes for efficient truncated density estimation.
problem Difficulties in estimating truncated density models due to intractable normalising constants and boundary conditions.
method Adapts score matching to solve the problem, introduces approximate Stein classes and a novel discrepancy measure, TKSD.
result TKSD does not require a fixed weighting function and can be evaluated using only boundary samples, leading to improved accuracy.
Given iid observations from an unknown absolute continuous distribution defined on some domain Ω, we propose a nonparametric method to learn a piecewise constant function to approximate the underlying probability density function. Our density estimate is a piecewise constant function defined on a binary partition o…
This work proposes a new method to match distributions across different spaces using cycle-consistent maps.
problem Matching distributions across different spaces with consistent bidirectional maps.
method A novel unbalanced Monge optimal transport formulation for matching distributions on different spaces, employing cycle-consistent maps.
result The proposed discrepancy captures the cycle-consistent GAN framework and provides theoretical support.
A new strategy selects k in k-NN regression without hold-out data.
problem Choosing optimal k in k-NN regression without hold-out data.
method Iterative procedure over k, minimum discrepancy principle.
result Minimax-optimal over smoothness function classes.
A new loss function ED simplifies training energy-based models without scores.
problem Training energy-based models is computationally expensive.
method Energy Discrepancy (ED) loss function that does not rely on scores or MCMC.
result ED effectively interpolates between score matching and negative log-likelihood.
SENA-discrepancy-VAE interprets latent causal factors in biological pathways.
problem Interpreting latent causal factors in biological pathways.
method SENA-discrepancy-VAE, a model based on discrepancy-VAE, that produces interpretable latent causal factors.
result Sena-discrepancy-VAE achieves comparable predictive performance with non-interpretable counterparts while providing biologically meaningful causal factors.
Though machine learning algorithms excel at minimizing the average loss over a population, this might lead to large discrepancies between the losses across groups within the population. To capture this inequality, we introduce and study a notion we call maximum weighted loss discrepancy (MWLD), the maximum (weighted) d…
Proposes DWMD for better matching of hidden representations across domains.
problem Measuring data distribution discrepancy between semantically related domains for feature representation matching.
method DWMD, a moment-based probability distribution metric that explicitly orders and weights higher-order moments.
result DWMD is error-free and can strictly reflect distribution differences without feature distribution assumptions.
MPMC generates low-discrepancy points using graph neural networks.
problem Generating efficient low-discrepancy point sets.
method Leveraging Graph Neural Networks to model geometric properties.
result Achieves state-of-the-art performance in generating low-discrepancy points.
New method optimizes non-linear functionals over probability measures.
problem Optimizing non-linear functionals defined over probability measures.
method N-particle underdamped Langevin algorithm with spacetime discretization.
result Converges globally in total variation distance.
Much of machine learning relies on comparing distributions with discrepancy measures. Stein's method creates discrepancy measures between two distributions that require only the unnormalized density of one and samples from the other. Stein discrepancies can be combined with kernels to define kernelized Stein discrepanc…
Study shows the corrected Akaike criterion is inadmissible for estimating Kullback-Leibler discrepancy.
problem Inadmissibility of the corrected Akaike information criterion for estimating Kullback-Leibler discrepancy.
method Loss estimation framework to demonstrate inadmissibility and provide improved estimators.
result Improved estimators of Kullback-Leibler discrepancy are provided and perform well in reduced-rank situations.
Improved algorithm for low-discrepancy colorings with practical time complexity.
problem Finding near-optimal colorings for set systems with low discrepancy.
method Randomized algorithm using primal-dual reweighing and matchings with low crossing number.
result Improved time complexity for constructing colorings and approximations.
For any prime power q and any dimension s≥1, we present a construction of (t,s)-sequences in base q with finite-row generating matrices such that, for fixed q, the quality parameter t is asymptotically optimal as a function of s as s→∞. This is the first construction of (t,s)-sequences th…
TMDA aligns subdomain data distribution discrepancies across domains using manifold representations.
problem Transfer learning challenges due to domain divergence.
method TMDA uses low-dimensional manifolds to represent subdomains and aligns local data distribution discrepancies across domains using M3D.
result TMDA is a promising method for various transfer learning tasks.
The performance of standard learning procedures has been observed to differ widely across groups. Recent studies usually attribute this loss discrepancy to an information deficiency for one group (e.g., one group has less data). In this work, we point to a more subtle source of loss discrepancy---feature noise. Our mai…
New partition designs reduce star discrepancy in high-dimensional sampling.
problem Improving the expected star discrepancy in high-dimensional sampling.
method Developed non-equal volume partitions to achieve lower expected star discrepancy.
result Explicit upper bounds for expected star discrepancy under non-equal volume partitions.
Improved likelihood-free inference for high-dimensional models.
problem Challenges in likelihood-free inference for high-dimensional parameter spaces.
method Bayesian optimization-based approach with misspecification-robust characterisation.
result Efficient inference in 100-dimensional space with real data application.
A new method for density estimation using mixture discrepancy and moments.
problem Generalizing histogram statistics to higher dimensions.
method Density estimation via mixture discrepancy and moments (DSP-mix and MSP).
result DSP-mix and MSP are computationally tractable and maintain accuracy with increased speed.