The paper proposes a method to produce well-calibrated predictions in regression tasks using maximum mean discrepancy.
problem The need for accurate uncertainty quantification in machine learning predictions.
method The method uses maximum mean discrepancy to minimize the kernel embedding measure and calibrate predictions.
result The method produces well-calibrated and sharp prediction intervals, outperforming state-of-the-art methods.
The study examines how hyperparameters affect prediction discrepancies in machine learning models.
problem Prediction inconsistencies across different machine learning models trained on the same dataset.
method Investigation of six models (Elastic Net, Decision Tree, k-NN, SVM, RF, XGBoost) on 21 benchmark datasets, focusing on key hyperparameters.
result Hyperparameter tuning improves model performance but increases prediction discrepancies, especially in Extreme Gradient Boosting.
This work investigates how GCNs should handle local structure discrepancies in testing nodes.
problem GCNs assume homophily but real graphs often have discrepancies in local structure.
method Using causal graph analysis, the study intervenes the graph structure to assess the local structure's impact on predictions.
result The method effectively enhances GCN predictions by eliminating local structure discrepancies.
Uncertainty quantification (UQ) is a vital step in using mathematical models and simulations to take decisions. The field of cardiac simulation has begun to explore and adopt UQ methods to characterise uncertainty in model inputs and how that propagates through to outputs or predictions. In this perspective piece we dr…
SENA-discrepancy-VAE interprets latent causal factors in biological pathways.
problem Interpreting latent causal factors in biological pathways.
method SENA-discrepancy-VAE, a model based on discrepancy-VAE, that produces interpretable latent causal factors.
result Sena-discrepancy-VAE achieves comparable predictive performance with non-interpretable counterparts while providing biologically meaningful causal factors.
Study on discrepancy principle for learning algorithms in nonparametric regression.
problem Determining optimal iteration number in nonparametric regression with unknown optimal iteration.
method Investigates discrepancy principle and modified principles for kernelized spectral filters, using deviation inequalities and change-of-norm arguments.
result Classical discrepancy principle is adaptive for slow rates, while modified principles are adaptive for faster rates.
Study finds discrepancies in model explanations for clean vs. adversarial inputs.
problem Identifying adversarial attacks on neural networks.
method Utilized experimental results to propose a framework for detecting adversarial inputs.
result Proposed framework can identify adversarial inputs based on model explanations.
Active learning algorithms propose which unlabeled objects should be queried for their labels to improve a predictive model the most. We study active learners that minimize generalization bounds and uncover relationships between these bounds that lead to an improved approach to active learning. In particular we show th…
New measures quantify uncertainty in survival models for maintenance tasks.
problem Uncertainty in survival models for maintenance tasks.
method Formal measures of ambiguity, discrepancy, and obscurity introduced.
result Multiple accurate survival models may yield conflicting risk estimates.
Framework corrects model form errors in structural dynamics predictions.
problem Model form errors in parametric models of structural dynamics.
method Gaussian Process Latent Force Model (GPLFM) for non-parametric discrepancy representation, linear Bayesian filtering for state and discrepancy estimation, modal reduction for computational tractability.
result Significant reduction of displacement and rotation prediction errors under unseen excitations.
A new measure helps compute suboptimality in entropy-regularized methods.
problem Computing suboptimality in entropy-regularized variational objectives when unnormalised densities are unavailable.
method Introduced 'kernel gradient discrepancy' (KGD) to compute suboptimality explicitly.
result KGD characterizes kernel Stein discrepancy (KSD) in the standard Bayesian context and measures variational gradient size.
Building performance discrepancies between building design and operation are one of the causes that lead many new designs fail to achieve their goals and objectives. One of main factors contributing to the discrepancy is occupant behaviors. Occupants responding to a new design are influenced by several factors. Existin…
Proposes an efficient lower bound for Gromov-Wasserstein discrepancy.
problem Comparing structured data from different metric-measure spaces.
method Orthogonal Gromov-Wasserstein (OGW) discrepancy with efficient closed-form lower bound.
result Efficient and tight lower bounds for Gromov-Wasserstein discrepancy.
A new embedding method extracts dataset-scale metric distribution into vectorial representation for graph data.
problem Classifying graph-structured data based on overall dataset-scale discrepancies.
method MetricDistribution2vec embedding strategy.
result Significant improvement in supervised prediction tasks on real-world graph datasets.
Optimizes kernel discrepancies by selecting subsets efficiently.
problem Improving kernel discrepancies for QMC methods.
method Introduces a novel subset selection algorithm for kernel discrepancies.
result Efficiently generates low-discrepancy samples from various distributions.
New discrepancy function compares discrete probability measures considering space geometry.
problem Comparing discrete probability measures in a geometrically meaningful way.
method Proposes the Fourier Discrepancy Function, proving convexity, differentiability, and providing gradient formula.
result Proves the Fourier Discrepancy is convex, twice differentiable, and provides an explicit gradient formula.
This paper introduces localized discrepancy theories for unsupervised domain adaptation.
problem Improving generalization bounds for unsupervised domain adaptation.
method Localized discrepancies defined on the hypothesis space after localization, leading to smaller and asymmetric values.
result Improved generalization bounds and sample complexity reduction.
The article introduces practical estimators for kernel discrepancies.
problem Estimating kernel discrepancies accurately and efficiently.
method Presented various estimators for MMD, HSIC, and KSD, including V-statistics, U-statistics, and incomplete U-statistics. Stressed the importance of kernel bandwidth and introduced adaptive estimators.
result Adaptive estimators combining multiple estimators with various kernels address the problem of kernel selection.
Deep learning model approximates stochastic responses.
problem Approximating stochastic responses using neural networks.
method Generative neural network with conditional maximum mean discrepancy (CMMD) loss.
result Excellent performance on benchmark problems.
Building accurate language models that capture meaningful long-term dependencies is a core challenge in natural language processing. Towards this end, we present a calibration-based approach to measure long-term discrepancies between a generative sequence model and the true distribution, and use these discrepancies to …
Framework achieves fairness in predictions using partially known causal graph over clusters of variables.
problem Achieving fairness in algorithmic decisions when causal graph knowledge is limited.
method Leverages a causal graph over clusters of variables to train a prediction model, reducing interventional distribution discrepancies.
result Framework strikes a better balance between fairness and accuracy than existing approaches under limited causal graph knowledge.
MPMC generates low-discrepancy points using graph neural networks.
problem Generating efficient low-discrepancy point sets.
method Leveraging Graph Neural Networks to model geometric properties.
result Achieves state-of-the-art performance in generating low-discrepancy points.
Sliced kernelized Stein discrepancy improves goodness-of-fit tests and model learning in high dimensions.
problem The curse-of-dimensionality in kernelized Stein discrepancy (KSD).
method Sliced Stein discrepancy and its scalable variants using optimal one-dimensional projections.
result Significantly outperforms KSD and baselines in goodness-of-fit tests and improves model learning.
Much of machine learning relies on comparing distributions with discrepancy measures. Stein's method creates discrepancy measures between two distributions that require only the unnormalized density of one and samples from the other. Stein discrepancies can be combined with kernels to define kernelized Stein discrepanc…
Study shows the corrected Akaike criterion is inadmissible for estimating Kullback-Leibler discrepancy.
problem Inadmissibility of the corrected Akaike information criterion for estimating Kullback-Leibler discrepancy.
method Loss estimation framework to demonstrate inadmissibility and provide improved estimators.
result Improved estimators of Kullback-Leibler discrepancy are provided and perform well in reduced-rank situations.
This paper defines the notion of class discrepancy for families of functions. It shows that low discrepancy classes admit small offline and streaming coresets. We provide general techniques for bounding the class discrepancy of machine learning problems. As corollaries of the general technique we bound the discrepancy …
Semi-parametric framework for nonlinear system identification
problem Nonlinear system identification
method Orthogonal Gaussian process regression
result Interpretable models from incomplete physics
TMDA aligns subdomain data distribution discrepancies across domains using manifold representations.
problem Transfer learning challenges due to domain divergence.
method TMDA uses low-dimensional manifolds to represent subdomains and aligns local data distribution discrepancies across domains using M3D.
result TMDA is a promising method for various transfer learning tasks.
The performance of standard learning procedures has been observed to differ widely across groups. Recent studies usually attribute this loss discrepancy to an information deficiency for one group (e.g., one group has less data). In this work, we point to a more subtle source of loss discrepancy---feature noise. Our mai…
Industry datasets used for text classification are rarely created for that purpose. In most cases, the data and target predictions are a by-product of accumulated historical data, typically fraught with noise, present in both the text-based document, as well as in the targeted labels. In this work, we address the quest…
New partition designs reduce star discrepancy in high-dimensional sampling.
problem Improving the expected star discrepancy in high-dimensional sampling.
method Developed non-equal volume partitions to achieve lower expected star discrepancy.
result Explicit upper bounds for expected star discrepancy under non-equal volume partitions.
A new method for density estimation using mixture discrepancy and moments.
problem Generalizing histogram statistics to higher dimensions.
method Density estimation via mixture discrepancy and moments (DSP-mix and MSP).
result DSP-mix and MSP are computationally tractable and maintain accuracy with increased speed.
TopInG improves graph interpretability using persistent homology.
problem Lack of interpretability in Graph Neural Networks (GNNs).
method TopInG uses persistent homology to identify persistent rationale subgraphs in graphs.
result TopInG improves predictive accuracy and interpretability compared to state-of-the-art methods.
Stochastic Stein Discrepancies improve inference efficiency.
problem Intractable computation of Stein discrepancies.
method Subsampled approximations of Stein operators.
result Stochastic Stein Discrepancies inherit convergence properties of standard SDs.
Bayes-consistent disagreement discrepancy loss improves model robustness.
problem Distribution shift in real-world neural network deployment.
method Introducing a novel disagreement loss that is Bayes consistent.
result Proves existing surrogates for disagreement discrepancy are not Bayes consistent.
Maximum mean discrepancy (MMD) has been widely adopted in domain adaptation to measure the discrepancy between the source and target domain distributions. Many existing domain adaptation approaches are based on the joint MMD, which is computed as the (weighted) sum of the marginal distribution discrepancy and the condi…
NOVI improves deep Gaussian process inference with neural generators and regularized Stein discrepancy.
problem Intractable exact inference in deep Gaussian processes.
method NOVI uses a neural generator to approximate the posterior distribution and minimizes Regularized Stein Discrepancy.
result NOVI achieves 93.56% classification accuracy on CIFAR10, outperforming state-of-the-art methods.
Efficiently marginalizes over Gaussian Process kernels for better model flexibility and uncertainty.
problem Inefficient marginalization over Gaussian Process kernels for large datasets.
method Bayesian Quadrature scheme with maximum mean discrepancies and invariances between Spectral Mixture kernels.
result Achieves more accurate predictions and better calibrated uncertainty than state-of-the-art baselines.
Unsupervised domain adaptation is the problem setting where data generating distributions in the source and target domains are different, and labels in the target domain are unavailable. One important question in unsupervised domain adaptation is how to measure the difference between the source and target domains. A pr…
Appropriately evaluating the discrepancy between domains is essential for the success of unsupervised domain adaptation. In this paper, we first point out that existing discrepancy measures are less informative when complex models such as deep neural networks are used, in addition to the facts that they can be computat…
Energy distance measures feature heterogeneity in federated learning.
problem Heterogeneity across data sources hinders model aggregation in federated learning.
method Introduced Taylor approximations of energy distance for efficient computation.
result Taylor approximations accurately capture feature discrepancies, improving convergence.
Framework identifies discrepancies in physics models, improving sensor accuracy.
problem Model inaccuracies leading to poor control algorithms.
method Learning systematic state-space residuals and deterministic dynamical errors.
result Improved quantification of system dynamics and control algorithms.
Machine learning (ML) and artificial intelligence (AI) algorithms are now being used to automate the discovery of physics principles and governing equations from measurement data alone. However, positing a universal physical law from data is challenging without simultaneously proposing an accompanying discrepancy model…
BRPC online Bayesian calibration handles gradual and abrupt system changes.
problem Aligning model outputs with field observations in evolving systems.
method Bayesian Recursive Projected Calibration (BRPC) for streaming data under simulator mismatch and nonstationarity.
result Improves calibration accuracy under gradual changes and robustness under abrupt regime shifts.
Stein discrepancy improves UDA performance in low-data scenarios.
problem Improving model performance on unlabeled target domains with limited data.
method Proposes a novel UDA framework using Stein discrepancy, an asymmetric measure that depends on the target distribution through its score function.
result Consistently outperforms prior UDA approaches under limited target data across multiple benchmarks.
Inequalities linking entropy, Fisher info, Stein discrepancy, and Wasserstein distance on Riemannian manifolds.
problem Linking entropy, Fisher info, Stein discrepancy, and Wasserstein distance on Riemannian manifolds.
method Deriving inequalities linking these measures on Riemannian manifolds.
result Strengthening and extending existing inequalities to Riemannian manifolds.
We explore whether useful temporal neural generative models can be learned from sequential data without back-propagation through time. We investigate the viability of a more neurocognitively-grounded approach in the context of unsupervised generative modeling of sequences. Specifically, we build on the concept of predi…
SRRM improves recursive transport surrogates in the small-discrepancy regime.
problem Insufficient understanding of recursive partitioning methods' statistical behavior and resolution in the small-discrepancy regime.
method Introduced Selective Recursive Rank Matching (SRRM) to improve the resolution of Recursive Rank Matching (RRM).
result SRRM yields a higher-fidelity practical surrogate for the Wasserstein distance at moderate additional computational cost.