The paper quantifies and attributes uncertainty in complex system simulations.
problem Uncertainty in complex system simulations due to unknown or approximated subprocesses.
method Developed a framework for quantifying and attributing submodel uncertainty using bootstrapping, Bayesian model averaging, and tree-based methods.
result Individual submodels contribute to overall uncertainty, and their importance can be quantified.
A GAN method for stochastic boundary conditions in fast dynamics.
problem Uncertainty quantification in fast dynamics and wave propagation.
method Physics-informed GANs for stochastic boundary conditions.
result Improved convergence and better stochastic boundary conditions.
Federated learning was proposed with an intriguing vision of achieving collaborative machine learning among numerous clients without uploading their private data to a cloud server. However, the conventional framework requires each client to leverage the full model for learning, which can be prohibitively inefficient fo…
A new method for handling missing values in data.
problem Handling missing values in machine learning models.
method Sharing pattern submodels with sparsity-inducing regularization.
result Sharing pattern submodels provide robust predictions and maintain/improve pattern submodel performance.
PostNet predicts uncertainty without OOD data, improving OOD detection and calibration.
problem Accurate uncertainty estimation for safe systems.
method PostNet uses Normalizing Flows to learn individual posterior distributions over predicted probabilities.
result PostNet achieves state-of-the-art results in OOD detection and uncertainty calibration.
E3 combines sparse MoEs and ensembles to improve model efficiency and performance.
problem Improving model efficiency and performance in machine learning.
method Combining sparse mixture of experts and ensembles, presenting Efficient Ensemble of Experts (E3). result E3 achieves better accuracy, log-likelihood, few-shot learning, robustness, and uncertainty estimates than baselines. Data is one of the essential ingredients to power deep learning research. Small datasets, especially specific to medical institutes, bring challenges to deep learning training stage. This work aims to develop a practical deep multimodal that can classify patients into abnormal and normal categories accurately as well a…
We study the mixtures of factorizing probability distributions represented as visible marginal distributions in stochastic layered networks. We take the perspective of kernel transitions of distributions, which gives a unified picture of distributed representations arising from Deep Belief Networks (DBN) and other netw…
ULFS-KDPE estimates parameters efficiently without influence functions.
problem Estimating pathwise differentiable parameters in nonparametric models.
method Kernel debiased plug-in estimator based on universal least favorable submodel.
result Semiparametric efficiency achieved without influence function derivation.
Temperature scaling improves model uncertainty but not diversity in LLMs.
problem Improving the calibration and stochasticity of probabilistic models.
method Investigates theoretical properties of temperature scaling in classification and LLMs.
result Temperature scaling increases model uncertainty but not diversity in LLMs.
Many powerful machine learning models are based on the composition of multiple processing layers, such as deep nets, which gives rise to nonconvex objective functions. A general, recent approach to optimise such "nested" functions is the method of auxiliary coordinates (MAC). MAC introduces an auxiliary coordinate for …
We applied machine learning to predict whether a gene is involved in axon regeneration. We extracted 31 features from different databases and trained five machine learning models. Our optimal model, a Random Forest Classifier with 50 submodels, yielded a test score of 85.71%, which is 4.1% higher than the baseline scor…
CNNs identify stock market trend endpoints based on expert opinion.
problem Finding optimal entry and exit points for stock market trends.
method Three CNN submodels sequentially identify changepoints, locate them, and classify trends as upward, downward, or flat.
result CNNs can identify long-term trends based on expert opinion, offering a new approach to stock market analysis.
Recommendations are broadly used in marketplaces to match users with items relevant to their interests and needs. To understand user intent and tailor recommendations to their needs, we use deep learning to explore various heterogeneous data available in marketplaces. This paper focuses on the challenge of measuring re…
Latent tree models are graphical models defined on trees, in which only a subset of variables is observed. They were first discussed by Judea Pearl as tree-decomposable distributions to generalise star-decomposable distributions such as the latent class model. Latent tree models, or their submodels, are widely used in:…
Proposes a neural network for contextual regression.
problem Improving model efficiency and interpretability in regression with contextual features.
method Simple contextual neural network (SCtxtNN) that separates context identification from context-specific regression.
result SCtxtNN achieves lower excess mean squared error and more stable performance than feed-forward neural networks.
Gaussian latent tree models, or more generally, Gaussian latent forest models have Fisher-information matrices that become singular along interesting submodels, namely, models that correspond to subforests. For these singularities, we compute the real log-canonical thresholds (also known as stochastic complexities or l…
We unify and extend a number of approaches related to constructing multivariate Variance-Gamma (V.G.) models for option pricing. An overarching model is derived by subordinating multivariate Brownian motion to a subordinator from the Thorin (1977) class of generalised Gamma convolution subordinators. A class of models …
Ever since the proof of asymptotic normality of maximum likelihood estimator by Cramer (1946), it has been understood that a basic technique of the Taylor series expansion suffices for asymptotics of M-estimators with smooth/differentiable loss function. Although the Taylor series expansion is a purely deterministic …
To find efficient screening methods for high dimensional linear regression models, this paper studies the relationship between model fitting and screening performance. Under a sparsity assumption, we show that a subset that includes the true submodel always yields smaller residual sum of squares (i.e., has better model…
Estimates hybrid dynamical systems with polynomial expansions and Markovian switching.
problem Identifying hybrid dynamical systems with nonlinear autoregressive exogenous (NARX) components and Markovian switching.
method Probabilistic framework using Expectation Maximization for parameter estimation, including submodel coefficients, hidden state values, and transition probabilities. Disentangles mode classification and NARX regression tasks. Uses soft-labels and coordinate descent approach for parameter fitting.
result Demonstrated on a SMNARX problem with three nonlinear sub-models, achieving parsimonious models through l1-norm bridge estimation and hard-thresholding.
Recent success in deep learning has generated immense interest among practitioners and students, inspiring many to learn about this new technology. While visual and interactive approaches have been successfully developed to help people more easily learn deep learning, most existing tools focus on simpler models. In thi…
In mixtures-of-experts (ME) model, where a number of submodels (experts) are combined, there have been two longstanding problems: (i) how many experts should be chosen, given the size of the training data? (ii) given the total number of parameters, is it better to use a few very complex experts, or is it better to comb…
The paper establishes convergence rates for MoE models in classification problems.
problem Understanding the behavior of MoE models in classification settings.
method Established convergence rates for density and parameter estimation in softmax gating multinomial logistic MoE models.
result Parameter estimation rates are significantly improved with a novel modified softmax gating function.
A new framework models multi-state events and biomarkers.
problem Limited representation of complex multi-state trajectories.
method General multi-state joint modeling framework.
result Accurate parameter recovery and personalized predictions.
A hybrid strategy forecasts short-term loads using Warm-start Gradient Tree Boosting.
problem Lack of effective short-term load forecasting methods.
method Hybrid strategy integrating four different inference models: tree-based ensemble method Warm-start Gradient Tree Boosting (WGTB).
result Demonstrates effectiveness of hybrid strategy on real datasets.
Paper introduces geometry-aware normalizing flows for improved causal inference.
problem Disparity between sample and population distributions in causal inference.
method Integrates continuous normalizing flows with parametric submodels, employing Wasserstein gradient flows and optimal transport.
result Significantly reduces parameter estimation bias and variance in finite-sample settings.
A method for identifying NPWARX models with arbitrary domains using probabilistic mixture models.
problem Identifying hybrid system models with discontinuous maps.
method Probabilistic mixture model with a neural network for nonlinear partitioning and Expectation Maximization for parameter estimation.
result Demonstrated on a nonlinear piece-wise problem with discontinuous maps.
NASirt automates CNN architecture design for spectral data.
problem Designing optimal neural architectures for complex data.
method Neural Architecture Search (NAS) with Item Response Theory (IRT) for instance-level complexity.
result NASirt achieves high accuracy (97.40%) on spectral datasets.
TSC improves causal effect estimation in panel data.
problem Estimating causal effects in panel data with a single treated unit.
method Targeted synthetic control method that refines initial weights through a one-dimensional targeted update.
result TSC consistently improves estimation accuracy over state-of-the-art SCM baselines.
LFM learns a sequence of smaller models to generate data from noise.
problem Learning continuous, invertible flows between distributions.
method Stepwise Local Flow Matching (LFM) model, matching diffusion processes up to time-step size.
result LFM achieves competitive generative performance compared to Flow Matching.
Novel method for learning Gaussian graphical models from paired data.
problem Learning Gaussian graphical models for dependent groups.
method Introducing twin order to explore the search space more efficiently.
result The twin order makes the model space a distributive lattice, leading to more efficient model exploration.
Study tackles causal structure learning in linear models with unobserved variables and measurement error.
problem Challenges of unobserved common causes and measurement error in causal structure learning.
method Introduces LV-SEM-ME model with four types of variables and characterizes identifiability under separability condition.
result Establishes form of identification robustness for target effect in broader LV-SEM-ME model.
The concept of mathematical modeling is widespread across almost all of the fields of contemporary science and engineering. Because of the existing necessity of predictions the behavior of natural phenomena, the researchers develop more and more complex models. However, despite their ability to better forecasting, the …
In this paper we discuss the variable selection method from \ell0-norm constrained regression, which is equivalent to the problem of finding the best subset of a fixed size. Our study focuses on two aspects, consistency and computation. We prove that the sparse estimator from such a method can retain all of the importa…
New PG samplers improve inference in coupled state-space models.
problem Bayesian inference from multiple time series with shared parameters.
method Marginalized Particle Gibbs samplers for coupled state-space models.
result Improved parameter inference through shared information.
Unified method for input, data, and model uncertainty in neural networks.
problem Uncertainty in neural network inputs and outputs.
method Propagating uncertainty through inputs using a unified formulation.
result More stable decision boundaries with input noise, and propagation of input uncertainty to model outputs.
This paper benchmarks uncertainty disentanglement across various tasks.
problem Disentangling multiple sources of uncertainty for specialized tasks.
method Reimplemented and evaluated a wide range of uncertainty estimators.
result No existing approach provides disentangled uncertainty estimators in practice.
Unified Bayesian framework for quantifying GNN uncertainty.
problem Quantifying uncertainty in GNN predictions due to modeling errors and measurement uncertainty.
method Unified Bayesian framework with aleatoric uncertainty from probabilistic links and feature noise, and epistemic uncertainty from model parameter distribution. Uses Assumed Density Filtering for aleatoric uncertainty and Monte Carlo dropout for model parameter uncertainty.
result Bayesian model performs similarly to frequentist model and provides additional uncertainty information.
Paper recovers uncertainty from dynamic valuation rules.
problem Recovering latent uncertainty from observable valuation rules.
method Developed procedures to identify and characterize uncertainty structures from valuation rules.
result Valuation rules contain sufficient information to identify and recover uncertainty structures.
Proposes a new criterion for reliable uncertainty estimation in deep neural networks.
problem Inability of existing approaches to provide reliable uncertainty estimates for deep neural networks.
method Develops a density uncertainty layer architecture that satisfies the proposed criterion.
result Density uncertainty layers provide more reliable uncertainty estimates and robust out-of-distribution detection.
Proposes a method to quantify uncertainty in graph neural networks for node classification.
problem Uncertainty in graph neural networks for node classification.
method Bayesian uncertainty propagation (BUP) method embedding GNNs in a Bayesian framework.
result Demonstrates superior performance of the proposed method on benchmark datasets.
New ensemble methods improve time series forecasting accuracy.
problem Global Forecasting Models (GFM) lack localisation for heterogeneous datasets.
method Ensemble techniques with clustering and varied GFM models.
result Significantly higher accuracy achieved compared to baseline models.
Survey on uncertainty in ML and DL, covering sources, quantification, and decision-making.
problem Understanding and quantifying uncertainty in ML and DL for risk-sensitive applications.
method Structured review of literature, categorizing uncertainty, assessing uncertainty quantification techniques.
result Broadened scope of uncertainty discussion and updated DL uncertainty quantification methods.
Unified Uncertainty Calibration improves AI predictions by combining different types of uncertainty.
problem AI classifiers struggle with uncertainty, leading to miscalibrated predictions and poor performance.
method Unified Uncertainty Calibration (U2C) combines aleatoric and epistemic uncertainties to improve prediction quality.
result U2C outperforms traditional reject-or-classify methods across various ImageNet benchmarks.
New method combines ODE filters and numerical quadrature to propagate model uncertainty.
problem Propagation of model uncertainty in ODE solutions with uncertain parameters.
method Combining ODE filters with numerical quadrature.
result Effective propagation of both numerical and parametric uncertainty.
Paper decomposes risk into aleatoric and epistemic uncertainties and generates predictive uncertainty measures.
problem Unclear relationships between various predictive uncertainty measures in literature.
method Bayesian estimation to decompose risk into aleatoric and epistemic uncertainties, generating different predictive uncertainty measures.
result Experimental validation confirms usefulness of derived predictive uncertainty measures for detecting out-of-distribution and misclassified instances.
Estimating how uncertain an AI system is in its predictions is important to improve the safety of such systems. Uncertainty in predictive can result from uncertainty in model parameters, irreducible data uncertainty and uncertainty due to distributional mismatch between the test and training data distributions. Differe…