This paper introduces localized discrepancy theories for unsupervised domain adaptation.
problem Improving generalization bounds for unsupervised domain adaptation.
method Localized discrepancies defined on the hypothesis space after localization, leading to smaller and asymmetric values.
result Improved generalization bounds and sample complexity reduction.
Active learning algorithms propose which unlabeled objects should be queried for their labels to improve a predictive model the most. We study active learners that minimize generalization bounds and uncover relationships between these bounds that lead to an improved approach to active learning. In particular we show th…
Study shows boundedness of klt singularities in 3D or with bounded Kollár components.
problem Boundedness of klt singularities in algebraic geometry.
method Analysis of Kollár components and local volumes.
result Minimal log discrepancies of Kollár components are bounded in dimension 3.
New discrepancy function compares discrete probability measures considering space geometry.
problem Comparing discrete probability measures in a geometrically meaningful way.
method Proposes the Fourier Discrepancy Function, proving convexity, differentiability, and providing gradient formula.
result Proves the Fourier Discrepancy is convex, twice differentiable, and provides an explicit gradient formula.
New partition designs reduce star discrepancy in high-dimensional sampling.
problem Improving the expected star discrepancy in high-dimensional sampling.
method Developed non-equal volume partitions to achieve lower expected star discrepancy.
result Explicit upper bounds for expected star discrepancy under non-equal volume partitions.
This paper defines the notion of class discrepancy for families of functions. It shows that low discrepancy classes admit small offline and streaming coresets. We provide general techniques for bounding the class discrepancy of machine learning problems. As corollaries of the general technique we bound the discrepancy …
Proposes an efficient lower bound for Gromov-Wasserstein discrepancy.
problem Comparing structured data from different metric-measure spaces.
method Orthogonal Gromov-Wasserstein (OGW) discrepancy with efficient closed-form lower bound.
result Efficient and tight lower bounds for Gromov-Wasserstein discrepancy.
Unsupervised domain adaptation is the problem setting where data generating distributions in the source and target domains are different, and labels in the target domain are unavailable. One important question in unsupervised domain adaptation is how to measure the difference between the source and target domains. A pr…
Study confirms boundedness of certain singularities in log Fano geometry.
problem Boundedness of log Fano cone singularities and minimal log discrepancies.
method Analyzing local volumes and minimal log discrepancies of Kollár components.
result Boundedness of K-semistable log Fano cone singularities confirmed in dimension three.
Bayes-consistent disagreement discrepancy loss improves model robustness.
problem Distribution shift in real-world neural network deployment.
method Introducing a novel disagreement loss that is Bayes consistent.
result Proves existing surrogates for disagreement discrepancy are not Bayes consistent.
Study on discrepancy principle for learning algorithms in nonparametric regression.
problem Determining optimal iteration number in nonparametric regression with unknown optimal iteration.
method Investigates discrepancy principle and modified principles for kernelized spectral filters, using deviation inequalities and change-of-norm arguments.
result Classical discrepancy principle is adaptive for slow rates, while modified principles are adaptive for faster rates.
Appropriately evaluating the discrepancy between domains is essential for the success of unsupervised domain adaptation. In this paper, we first point out that existing discrepancy measures are less informative when complex models such as deep neural networks are used, in addition to the facts that they can be computat…
Validates composite systems using discrepancy propagation.
problem Validation of industrial systems with costly real-world tests.
method Propagates bounds on distributional discrepancy measures through a composite system.
result Derives upper bound on real system failure probability from simulations.
New research sets the minimax lower bound for KSD estimation at sqrt(n).
problem Estimating goodness-of-fit using Kernel Stein Discrepancy (KSD) on high-dimensional spaces.
method Two complementary results proving the minimax lower bound of KSD estimation.
result The minimax lower bound of KSD estimation is n^(-1/2), indicating exponential difficulty with dimensionality.
Stein discrepancy improves UDA performance in low-data scenarios.
problem Improving model performance on unlabeled target domains with limited data.
method Proposes a novel UDA framework using Stein discrepancy, an asymmetric measure that depends on the target distribution through its score function.
result Consistently outperforms prior UDA approaches under limited target data across multiple benchmarks.
Residual flows are shown to approximate MMD well.
problem Lack of theoretical understanding of normalizing flows' expressiveness.
method Proved residual flows are universal approximators in MMD.
result Residual flows can approximate MMD with a bounded number of blocks.
The paper shows how MMD metrizes weak convergence for certain kernels.
problem Characterizing MMD metrizing weak convergence for a wide class of kernels.
method Proving MMD metrizes weak convergence for specific kernels on a locally compact space.
result Corrected prior results and identified new kernels metrizing weak convergence.
We present a new analysis of the problem of learning with drifting distributions in the batch setting using the notion of discrepancy. We prove learning bounds based on the Rademacher complexity of the hypothesis set and the discrepancy of distributions both for a drifting PAC scenario and a tracking scenario. Our boun…
PolyGraph Discrepancy improves graph generative model evaluation.
problem Inability of existing metrics to provide an absolute performance measure and comparability across different graph descriptors.
method Approximates Jensen-Shannon distance using binary classifiers trained to distinguish between real and generated graphs.
result PGD provides a more robust and insightful evaluation compared to MMD metrics.
New conditions ensure MMDs separate and converge to target distributions.
problem Ensuring MMDs separate and converge to target distributions.
method Deriving new sufficient and necessary conditions for MMDs on separable metric spaces.
result First KSDs that exactly metrize weak convergence to P.
In this paper, we bound the error induced by using a weighted skeletonization of two data sets for computing a two sample test with kernel maximum mean discrepancy. The error is quantified in terms of the speed in which heat diffuses from those points to the rest of the data, as well as how at the weights on the refere…
New method detects changes online with bounds on delay.
problem Detecting changes in data streams efficiently.
method Maximizes discrepancy between pre-change and post-change distributions.
result Non-asymptotic bounds on average running length and detection delay.
Improved algorithm for low-discrepancy colorings with practical time complexity.
problem Finding near-optimal colorings for set systems with low discrepancy.
method Randomized algorithm using primal-dual reweighing and matchings with low crossing number.
result Improved time complexity for constructing colorings and approximations.
Given i.i.d samples from some unknown continuous density on hyper-rectangle [0,1]d, we attempt to learn a piecewise constant function that approximates this underlying density non-parametrically. Our density estimate is defined on a binary split of [0,1]d and built up sequentially according to discrepancy crite…
Generation of pseudorandom numbers from different probability distributions has been studied extensively in the Monte Carlo simulation literature. Two standard generation techniques are the acceptance-rejection and inverse transformation methods. An alternative approach to Monte Carlo simulation is the quasi-Monte Carl…
Optimizes kernel discrepancies by selecting subsets efficiently.
problem Improving kernel discrepancies for QMC methods.
method Introduces a novel subset selection algorithm for kernel discrepancies.
result Efficiently generates low-discrepancy samples from various distributions.
New lower bounds for linear classification problems in high dimensions.
problem Linear classification problems in high-dimensional spaces.
method Reduction from hardness conjectures for Affine Degeneracy testing and k-Sum problems.
result Matching lower bounds of Ω(n^d) and respectively Ω(1/ε^d) for Maximum Halfspace Discrepancy problem.
The paper analyzes greedy algorithms for MMD minimization, showing their efficiency and approximation error.
problem Minimizing Maximum Mean Discrepancy (MMD) for probability measure quantization.
method Iterative algorithms including kernel herding, greedy MMD minimization, and Sequential Bayesian Quadrature (SBQ).
result The greedy algorithms have a lower approximation error than SBQ, but are significantly faster.
Deep neural networks can approximate any target probability distribution given certain conditions.
problem Approximating complex probability distributions with deep neural networks.
method Proving the existence of a deep neural network mapping that approximates a target distribution under various integral probability metrics.
result Upper bounds on the size of the neural network in terms of dimension and approximation error for different metrics.
Supervised learning results typically rely on assumptions of i.i.d. data. Unfortunately, those assumptions are commonly violated in practice. In this work, we tackle such problem by focusing on domain generalization: a formalization where the data generating process at test time may yield samples from never-before-seen…
For any prime power q and any dimension s≥1, we present a construction of (t,s)-sequences in base q with finite-row generating matrices such that, for fixed q, the quality parameter t is asymptotically optimal as a function of s as s→∞. This is the first construction of (t,s)-sequences th…
The article introduces practical estimators for kernel discrepancies.
problem Estimating kernel discrepancies accurately and efficiently.
method Presented various estimators for MMD, HSIC, and KSD, including V-statistics, U-statistics, and incomplete U-statistics. Stressed the importance of kernel bandwidth and introduced adaptive estimators.
result Adaptive estimators combining multiple estimators with various kernels address the problem of kernel selection.
Study on Wasserstein gradient flow for MMD between Coulomb measures.
problem Analyzing the long-time behavior of MMD between probability and target measures using Coulomb kernels.
method Existence of global weak solutions, ultracontractive estimate, regularity analysis, exponential decay proof, defective Polyak-Lojasiewicz inequality.
result Exponential decay of squared MMD toward a uniformly positive target measure on flat torus.
The paper explores multidimensional critic output in GANs, improving convergence and diversity.
problem Underexplored in GANs literature, multidimensional critic output.
method Generalized Wasserstein GAN framework, SRVT block, maximal p-centrality discrepancy.
result High-dimensional critic output improves GAN performance in convergence and diversity.
Improved UDA framework using f-divergence measures.
problem Addressing distribution shifts in machine learning.
method Refined f-divergence-based discrepancy and f-domain discrepancy. result Novel target error and sample complexity bounds.
In this work, we present a novel upper bound of target error to address the problem for unsupervised domain adaptation. Recent studies reveal that a deep neural network can learn transferable features which generalize well to novel tasks. Furthermore, a theory proposed by Ben-David et al. (2010) provides a upper bound …
Though machine learning algorithms excel at minimizing the average loss over a population, this might lead to large discrepancies between the losses across groups within the population. To capture this inequality, we introduce and study a notion we call maximum weighted loss discrepancy (MWLD), the maximum (weighted) d…
Stein's method for measuring convergence to a continuous target distribution relies on an operator characterizing the target and Stein factor bounds on the solutions of an associated differential equation. While such operators and bounds are readily available for a diversity of univariate targets, few multivariate targ…
MPMC generates low-discrepancy points using graph neural networks.
problem Generating efficient low-discrepancy point sets.
method Leveraging Graph Neural Networks to model geometric properties.
result Achieves state-of-the-art performance in generating low-discrepancy points.
Sliced kernelized Stein discrepancy improves goodness-of-fit tests and model learning in high dimensions.
problem The curse-of-dimensionality in kernelized Stein discrepancy (KSD).
method Sliced Stein discrepancy and its scalable variants using optimal one-dimensional projections.
result Significantly outperforms KSD and baselines in goodness-of-fit tests and improves model learning.
Much of machine learning relies on comparing distributions with discrepancy measures. Stein's method creates discrepancy measures between two distributions that require only the unnormalized density of one and samples from the other. Stein discrepancies can be combined with kernels to define kernelized Stein discrepanc…
Proves a conjecture about manifolds and scalar curvature.
problem Determining when a manifold admits a positive scalar curvature metric.
method Uses a geometric bound to measure discrepancies between vector fields.
result Proves the conjecture in codimension two.
Study shows the corrected Akaike criterion is inadmissible for estimating Kullback-Leibler discrepancy.
problem Inadmissibility of the corrected Akaike information criterion for estimating Kullback-Leibler discrepancy.
method Loss estimation framework to demonstrate inadmissibility and provide improved estimators.
result Improved estimators of Kullback-Leibler discrepancy are provided and perform well in reduced-rank situations.
A new measure scales MMD to assess distribution closeness.
problem Testing statistical significance of distribution closeness.
method Norm-adaptive MMD (NAMMD) for distributional discrepancy.
result NAMMD-based DCT has higher test power than MMD-based DCT.
In Bayesian inference, the posterior distributions are difficult to obtain analytically for complex models such as neural networks. Variational inference usually uses a parametric distribution for approximation, from which we can easily draw samples. Recently discrete approximation by particles has attracted attention …
A new method trains deep neural networks for open set domain adaptation without negative open set difference.
problem Training deep neural networks for open set domain adaptation without negative open set difference.
method Proposes a new upper bound of target-domain risk, including source-domain risk, ε-open set difference (Δε), distributional discrepancy, and constant. Uses gradient descent for source-domain risk and Δε, and adversarial training for distributional discrepancy. Trains DNNs via minimizing the new upper bound. result Shows state-of-the-art performance on benchmark datasets.
Transfer learning has been demonstrated to be successful and essential in diverse applications, which transfers knowledge from related but different source domains to the target domain. Online transfer learning(OTL) is a more challenging problem where the target data arrive in an online manner. Most OTL methods combine…
New method gives provable error bounds for neural nets under distribution shift.
problem Proving reliable error bounds for neural networks under distribution shift.
method Optimizing a classifier to disagree with another, using a new 'disagreement loss'.
result Valid error bounds with comparable accuracy to competitive methods.