Partial model averaging improves Federated Learning performance.
problem Periodic model averaging causes significant model discrepancy in Federated Learning.
method Proposes a partial model averaging framework that encourages local models to stay close to each other.
result Partial averaging achieves up to 2.2% higher validation accuracy than full averaging.
Two methods using low-discrepancy points improve data compression for neural networks.
problem Efficiently compress large datasets for neural network training.
method Two methods based on low-discrepancy points: digital nets with averaging and clustering.
result Second method outperforms supercompress in compression error and neural network accuracy.
Uncertainty quantification (UQ) is a vital step in using mathematical models and simulations to take decisions. The field of cardiac simulation has begun to explore and adopt UQ methods to characterise uncertainty in model inputs and how that propagates through to outputs or predictions. In this perspective piece we dr…
Generative model for TPPs using signatures and distributional discrepancies.
problem Limitations of signature methods for TPPs and lack of global sequence-level loss in neural models.
method Introduce interarrival embedding to lift jump paths to continuous paths of bounded variation, enabling signature methods for discrete event sequences. Develop sigTPP, a signature-based generative model trained on path-level loss.
result sigTPP achieves the best average rank across multiple metrics and outperforms or is within a standard error of the strongest baseline in 64% of dataset-metric pairs.
New method prunes classification trees for biased data.
problem Pruning classification trees in imbalanced training data.
method Optimal pruning procedure for inhomogeneous data.
result First efficient procedure for optimal pruning under covariate shift.
Optimal model averaging for conditional generative models improves performance across various data types.
problem Multiple plausible generators for conditional distributions can vary in performance.
method Sample-based maximum mean discrepancy, static model averaging, and mixture-of-experts model averaging.
result MoEMA improves over competing baselines across various data types.
New method detects changes online with bounds on delay.
problem Detecting changes in data streams efficiently.
method Maximizes discrepancy between pre-change and post-change distributions.
result Non-asymptotic bounds on average running length and detection delay.
Generalizes causal inference to high-dimensional outcomes.
problem Limited causal inference methods for multivariate outcomes.
method Formulates causal discrepancy tests for nominal variables, uses conditional independence tests.
result Causal CDcorr method improves finite sample validity and power.
New principle controls graph-informed adversarial discrepancies.
problem Graph-informed adversarial learning for interpolative divergences.
method Proves infimal subadditivity for interpolative divergences.
result Graph-informed adversarial learning is justified for interpolative divergences.
Estimates and tests treatment effects on entire outcome distributions.
problem Treatment effects on entire outcome distributions, not just averages.
method Proposes a novel estimand and doubly robust estimator, develops a test.
result First test with provably valid type 1 error guarantees in this setting.
LMC improves sampling from complex distributions using quasi-random sequences.
problem Sampling from complex high-dimensional distributions with high accuracy.
method Using completely uniformly distributed (CUD) sequences in Langevin Monte Carlo (LMC) to generate Gaussian perturbations.
result LMC with low-discrepancy CUD sequences achieves smaller estimation error than standard LMC.
Though machine learning algorithms excel at minimizing the average loss over a population, this might lead to large discrepancies between the losses across groups within the population. To capture this inequality, we introduce and study a notion we call maximum weighted loss discrepancy (MWLD), the maximum (weighted) d…
In supervised clustering, standard techniques for learning a pairwise dissimilarity function often suffer from a discrepancy between the training and clustering objectives, leading to poor cluster quality. Rectifying this discrepancy necessitates matching the procedure for training the dissimilarity function to the clu…
A new inference method using regression and batched discrepancies.
problem Simulating parameters from simulator outputs.
method Regression-based projection and batched discrepancy weighting.
result Method produces a self-normalized pseudo-posterior.
Obtaining more accurate equity value estimates is the starting point for stock selection, value-based indexing in a noisy market, and beating benchmark indices through tactical style rotation. Unfortunately, discounted cash flow, method of comparables, and fundamental analysis typically yield discrepant valuation estim…
Paper analyzes SVGD algorithm for non-asymptotic convergence.
problem Optimizing a set of particles to approximate a target probability distribution.
method Finite time analysis of SVGD algorithm, providing descent lemma and convergence rates.
result SVGD algorithm decreases the objective at each iteration and converges to the target distribution.
Proposes DR-ME test for interpretable distributional treatment effects.
problem Detects invisible differences in treatment effects on distributional outcomes.
method Semiparametrically efficient finite-location test using kernel witnesses and orthogonal features.
result DR-ME reveals causal-discrepancy coordinates and has noncentral chi-square local power.
Optimizes kernel discrepancies by selecting subsets efficiently.
problem Improving kernel discrepancies for QMC methods.
method Introduces a novel subset selection algorithm for kernel discrepancies.
result Efficiently generates low-discrepancy samples from various distributions.
Proposes a new method to analyze the distributional effects of treatments.
problem Analyzing the full distributional impact of treatments beyond just the mean.
method Uses kernel conditional mean embeddings and U-statistic regression to investigate the CoDiTE.
result Demonstrates the effectiveness of the proposed method through experiments.
New method gives provable error bounds for neural nets under distribution shift.
problem Proving reliable error bounds for neural networks under distribution shift.
method Optimizing a classifier to disagree with another, using a new 'disagreement loss'.
result Valid error bounds with comparable accuracy to competitive methods.
New discrepancy function compares discrete probability measures considering space geometry.
problem Comparing discrete probability measures in a geometrically meaningful way.
method Proposes the Fourier Discrepancy Function, proving convexity, differentiability, and providing gradient formula.
result Proves the Fourier Discrepancy is convex, twice differentiable, and provides an explicit gradient formula.
This paper introduces localized discrepancy theories for unsupervised domain adaptation.
problem Improving generalization bounds for unsupervised domain adaptation.
method Localized discrepancies defined on the hypothesis space after localization, leading to smaller and asymmetric values.
result Improved generalization bounds and sample complexity reduction.
We study the information content of nuclear masses from the perspective of global models of nuclear binding energies. To this end, we employ a number of statistical methods and diagnostic tools, including Bayesian calibration, Bayesian model averaging, chi-square correlation analysis, principal component analysis, and …
Gromov-Wasserstein (GW) is a powerful tool to compare probability measures whose supports are in different metric spaces. GW suffers however from a computational drawback since it requires to solve a complex non-convex quadratic program. We consider in this work a specific family of cost metrics, namely \textit{tree me…
Proposes a method to improve CATE estimation by imputing missing potential outcomes.
problem Statistical discrepancy between distinct treatment groups in CATE estimation.
method Contrastive learning approach to reliably impute missing potential outcomes for a subset of individuals.
result Improves the accuracy and robustness of CATE estimation models.
The article introduces practical estimators for kernel discrepancies.
problem Estimating kernel discrepancies accurately and efficiently.
method Presented various estimators for MMD, HSIC, and KSD, including V-statistics, U-statistics, and incomplete U-statistics. Stressed the importance of kernel bandwidth and introduced adaptive estimators.
result Adaptive estimators combining multiple estimators with various kernels address the problem of kernel selection.
Model selection for time series forecasting can be biased by the distribution of scores.
problem Model selection for probabilistic forecasting on time series data.
method Using proper scoring rules to aggregate scores across multiple time series.
result The mean score is immune to the skewness of the score distribution.
We consider discriminative dictionary learning in a distributed online setting, where a network of agents aims to learn a common set of dictionary elements of a feature space and model parameters while sequentially receiving observations. We formulate this problem as a distributed stochastic program with a non-convex o…
MPMC generates low-discrepancy points using graph neural networks.
problem Generating efficient low-discrepancy point sets.
method Leveraging Graph Neural Networks to model geometric properties.
result Achieves state-of-the-art performance in generating low-discrepancy points.
Sliced kernelized Stein discrepancy improves goodness-of-fit tests and model learning in high dimensions.
problem The curse-of-dimensionality in kernelized Stein discrepancy (KSD).
method Sliced Stein discrepancy and its scalable variants using optimal one-dimensional projections.
result Significantly outperforms KSD and baselines in goodness-of-fit tests and improves model learning.
Much of machine learning relies on comparing distributions with discrepancy measures. Stein's method creates discrepancy measures between two distributions that require only the unnormalized density of one and samples from the other. Stein discrepancies can be combined with kernels to define kernelized Stein discrepanc…
A new differentiable divergence for time series comparison.
problem Computing discrepancies between time series of varying lengths.
method Proposed a new divergence, soft-DTW divergence, addressing issues of differentiability and positivity.
result Showed that the new divergence is a valid divergence: non-negative and minimized when time series are equal.
Study shows the corrected Akaike criterion is inadmissible for estimating Kullback-Leibler discrepancy.
problem Inadmissibility of the corrected Akaike information criterion for estimating Kullback-Leibler discrepancy.
method Loss estimation framework to demonstrate inadmissibility and provide improved estimators.
result Improved estimators of Kullback-Leibler discrepancy are provided and perform well in reduced-rank situations.
This paper defines the notion of class discrepancy for families of functions. It shows that low discrepancy classes admit small offline and streaming coresets. We provide general techniques for bounding the class discrepancy of machine learning problems. As corollaries of the general technique we bound the discrepancy …
Semi-parametric framework for nonlinear system identification
problem Nonlinear system identification
method Orthogonal Gaussian process regression
result Interpretable models from incomplete physics
Approximate Bayesian computation (ABC) has become an essential part of the Bayesian toolbox for addressing problems in which the likelihood is prohibitively expensive or entirely unknown, making it intractable. ABC defines a pseudo-posterior by comparing observed data with simulated data, traditionally based on some su…
TMDA aligns subdomain data distribution discrepancies across domains using manifold representations.
problem Transfer learning challenges due to domain divergence.
method TMDA uses low-dimensional manifolds to represent subdomains and aligns local data distribution discrepancies across domains using M3D.
result TMDA is a promising method for various transfer learning tasks.
The performance of standard learning procedures has been observed to differ widely across groups. Recent studies usually attribute this loss discrepancy to an information deficiency for one group (e.g., one group has less data). In this work, we point to a more subtle source of loss discrepancy---feature noise. Our mai…
New partition designs reduce star discrepancy in high-dimensional sampling.
problem Improving the expected star discrepancy in high-dimensional sampling.
method Developed non-equal volume partitions to achieve lower expected star discrepancy.
result Explicit upper bounds for expected star discrepancy under non-equal volume partitions.
FAST selects coresets more efficiently by matching distributions in the frequency domain.
problem Efficiently selecting representative subsets of large datasets for deep learning.
method FAST uses spectral graph theory and CFD to match distributions, addressing limitations of existing methods.
result FAST significantly outperforms state-of-the-art coreset selection methods in accuracy and energy efficiency.
We derive a dimensionally-reduced limit theory for an n-dimensional nonlinear elastic body that is slender along k dimensions. The starting point is to view an elastic body as an n-dimensional Riemannian manifold together with a not necessarily isometric W1,2-immersion in n-dimensional Euclidean space. The…
A new method for density estimation using mixture discrepancy and moments.
problem Generalizing histogram statistics to higher dimensions.
method Density estimation via mixture discrepancy and moments (DSP-mix and MSP).
result DSP-mix and MSP are computationally tractable and maintain accuracy with increased speed.
Stochastic Stein Discrepancies improve inference efficiency.
problem Intractable computation of Stein discrepancies.
method Subsampled approximations of Stein operators.
result Stochastic Stein Discrepancies inherit convergence properties of standard SDs.
Bayes-consistent disagreement discrepancy loss improves model robustness.
problem Distribution shift in real-world neural network deployment.
method Introducing a novel disagreement loss that is Bayes consistent.
result Proves existing surrogates for disagreement discrepancy are not Bayes consistent.
SVM used for estimating treatment effects without confounding.
problem Estimating average treatment effects in the presence of confounding variables.
method Adapts SVM classifier as a kernel-based weighting procedure to balance covariates and estimate causal effects.
result SVM provides a continuous relaxation of the quadratic integer program for balancing covariates and maximizing effective sample size.
Maximum mean discrepancy (MMD) has been widely adopted in domain adaptation to measure the discrepancy between the source and target domain distributions. Many existing domain adaptation approaches are based on the joint MMD, which is computed as the (weighted) sum of the marginal distribution discrepancy and the condi…
Unsupervised domain adaptation is the problem setting where data generating distributions in the source and target domains are different, and labels in the target domain are unavailable. One important question in unsupervised domain adaptation is how to measure the difference between the source and target domains. A pr…
MCRapper efficiently computes patterns in data using Monte-Carlo Rademacher Averages.
problem Finding statistically significant patterns in data with limited samples.
method Monte-Carlo Empirical Rademacher Averages (MCERA) for poset families.
result MCRapper provides upper bounds to the discrepancy of functions, enabling efficient pattern mining.