Large models collapse epistemic uncertainty, challenging traditional wisdom.
problem Epistemic uncertainty collapse in large models.
method Implicit ensembling and decomposition techniques.
result Larger models can collapse epistemic uncertainty, contrary to expectations.
A new method improves Bayesian deep learning by balancing scalability and accuracy.
problem Scalability issues in Bayesian neural networks.
method Collapsed inference scheme that performs Bayesian model averaging using collapsed samples.
result Significant improvements over existing methods in predictive performance and uncertainty estimation.
Proposes PSCs for UQ in deep nets without retraining.
problem Estimating uncertainty in deep nets with a single pass.
method Identifies sensitive, smooth intermediate layer, fits probabilistic model.
result PSCs achieve UQ and OOD detection performance matching existing methods.
Bayesian Neural Networks show unexpected collapse of epistemic uncertainty with large models and little data.
problem Unexpected collapse of epistemic uncertainty in Bayesian Neural Networks.
method Experiments with varying model size and training data size.
result Epistemic uncertainty collapses in the presence of large models and sometimes little data.
Framework disentangles deep feature uncertainty for efficient inference.
problem Inference-time uncertainty estimation for reliable decision-making.
method Uncertainty-Guided Inference-Time Selection framework.
result Significantly tighter prediction intervals and 60% compute reduction.
DVE uses GPs on DNN outputs to provide UQ without retraining.
problem Feature collapse in DNNs affects UQ methods.
method Deep Vecchia ensemble (DVE) of GPs on DNN hidden layers.
result Deterministic UQ possible in feature-collapsed DNNs.
SMI uses mixture models to improve SVGD's performance in Bayesian inference.
problem Variance collapse in SVGD for Bayesian inference, especially with small models.
method Generalizes SVGD to Stein mixture models, optimizing an ELBO lower bound.
result SMI avoids variance collapse and accurately estimates uncertainty for small BNNs.
Proposes a new method for uncertainty estimation in neural networks.
problem Uncertainty quantification in neural networks for high-risk applications.
method Intuitive framework based on signal-to-noise ratio and variance-gated measure.
result Demonstrates a collapse in diversity of committee machines.
New bandit model for healthcare intervention planning.
problem Maximizing patient health with limited monitoring resources.
method Developed Collapsing Bandits model and derived optimal policies.
result 3-order-of-magnitude speedup in algorithm performance.
COME replaces entropy minimization to prevent model collapse.
problem Overconfidence in entropy minimization leads to model collapse.
method COME explicitly models uncertainty with a Dirichlet prior distribution.
result COME achieves state-of-the-art performance on various TTA settings.
The standard taxonomy of predictive uncertainty is inconsistent with standard measures.
problem Uncertainty taxonomy and measure inconsistency
method Proof of inconsistency
result Uncertainty is not reducible to data collection
A new meta-learning method using shared variational inference.
problem Meta-learning with uncertainty over model parameters.
method Shared amortized variational inference network for conditional prior and posterior.
result Prevents collapse of conditional prior to Dirac delta function.
We propose and analyze numerically a simple dynamical model that describes the firm behaviors under uncertainty of demand forecast. Iterating this simple model and varying some parameters values we observe a wide variety of market dynamics such as equilibria, periodic and chaotic behaviors. Interestingly the model is a…
Improved uncertainty estimation in neural networks with VBLL.
problem Improving uncertainty estimation in neural networks.
method Deterministic variational formulation for training Bayesian last layer neural networks.
result Improves predictive accuracy, calibration, and out-of-distribution detection.
Bayesian analysis reveals epistemic uncertainty as a key diagnostic for delayed generalization in in-context learning.
problem Delayed generalization in in-context learning from few examples.
method Bayesian perspective, modular arithmetic tasks, approximate Bayesian techniques, spectral mechanism analysis.
result Epistemic uncertainty collapses sharply when the model groks, indicating a practical diagnostic of generalization.
A new model improves uncertainty estimation in deep learning.
problem Deep Kernel Learning (DKL) produces unreliable uncertainty estimates.
method Proposed a bi-Lipschitz constraint to preserve distances in feature space.
result DUE model outperforms previous DKL and other methods in uncertainty quality.
Framework estimates multiple plausible solutions with uncertainty measures.
problem Machine learning models need to propose multiple plausible solutions with meaningful uncertainty.
method Discrete latent variables model one-to-many mappings, allowing effective conditional probability estimation.
result Framework outperforms state-of-the-art in uncertainty estimation and is practical.
LLMs generate answers under incomplete context, and their uncertainty should scale with missing information.
problem Evaluating the quality of LLM answers under incomplete context.
method A controlled framework with varying context availability, and two uncertainty measures (sampling-based confidence and response entropy) evaluated on SQuAD.
result Response entropy increases with context removal and explains more variance in accuracy than confidence, suggesting it is a more responsive uncertainty measure.
Repulsive ensembles improve uncertainty estimates in PINNs for differential equations.
problem Improving uncertainty estimates in PINNs for differential equations.
method Employing repulsive ensembles (RE-PINN) with a repulsive term in the loss function.
result Repulsive ensembles produce more accurate uncertainty estimates and higher sample diversity.
Combines neural networks with variational inference for better uncertainty quantification.
problem Overconfident predictions from traditional neural networks and time-consuming Bayesian optimization.
method VIFO (Variational Inference on the Final-Layer Output) using neural networks to learn mean and variance.
result VIFO provides a good tradeoff in run time and uncertainty quantification, especially for out of distribution data.
Deep generative model for healthcare data identifies coherent substructures and mutational clusters.
problem Analytical challenges in healthcare data, including sparsity, missingness, and small sample sizes.
method Proposes a deep generative Bayesian model with collapsed Gibbs sampling for multinomial count data.
result Identifies coherent substructures and biologically meaningful mutational clusters in cancer data.
VCL adds uncertainty to contrastive learning models.
problem Lack of uncertainty quantification in contrastive learning methods.
method VCL uses a decoder-free framework that maximizes ELBO with InfoNCE loss and KL divergence.
result VCL provides meaningful uncertainty estimates and matches deterministic baselines in accuracy.
NE-GMM uses ES and GMM to improve uncertainty quantification.
problem Challenges in estimating mean and variance of complex distributions.
method Integrates Gaussian Mixture Model with Energy Score.
result NE-GMM outperforms in predictive accuracy and uncertainty quantification.
New method recovers clean data from corrupted samples.
problem Recovering clean data from corrupted samples with uncertainty.
method Probabilistic Tomographic Auto-Encoder method that derives reduced entropy condition approximate inference.
result Superior performance in imputation and de-noising compared to existing methods.
New model detects hidden group structures in criminal networks.
problem Challenges in identifying group structures in criminal networks with noisy data.
method Developed an extended stochastic block model (ESBM) to infer group structures.
result Unveiled complex block structures in an Italian mafia network.
In image classification tasks, the ability of deep CNNs to deal with complex image data has proven to be unrivalled. However, they require large amounts of labeled training data to reach their full potential. In specialised domains such as healthcare, labeled data can be difficult and expensive to obtain. Active Learni…
Bayesian deep learning faces posterior collapse due to likelihood vs. prior competition.
problem Posterior collapse in Bayesian deep learning models.
method Identified competition between likelihood and prior regularization in a linear latent variable model.
result Posterior collapse is related to neural and dimensional collapse, suggesting a broader learning issue.
Proves weakly non-collapsed RCD spaces are strongly non-collapsed.
problem Proving the equivalence of weakly non-collapsed and strongly non-collapsed RCD spaces.
method Analyzes properties of RCD spaces and uses auxiliary results.
result Confirms conjecture about RCD spaces being strongly non-collapsed.
Study on Neural Collapse limits in deep learning.
problem Understanding the limits of Neural Collapse in deep learning.
method Investigated Neural Collapse in the context of generalization and feature learning, refining conjectures and conducting experiments.
result Neural Collapse primarily occurs on the train set and not on the test set, suggesting it is an optimization phenomenon with unclear connections to generalization.
AdaDEM decouples EM into two parts to improve class overlap and uncertainty.
problem Improper EM limits its effectiveness in various machine learning tasks.
method Decouple EM into CADF and GMC, and AdaDEM normalizes CADF reward and uses MEC.
result AdaDEM outperforms classical EM and improves performance in noisy and dynamic environments.
A new framework reduces inconsistencies in chaotic surrogate modeling.
problem Consistency issues between probabilistic objectives and dynamical system dynamics.
method KAFFEE (Kalman-Aware Framework For Ergodic Emulation), a differentiable extended Kalman filter.
result KAFFEE mitigates the dynamic-probabilistic consistency gap, improving reconstruction and predictive scores.
Ricci flow smooths locally collapsing manifolds with controlled curvature.
problem Locally collapsing manifolds with controlled Ricci curvature.
method Ricci flow for a definite period of time, detecting collapsing infranil fiber bundles.
result Topological conditions detect collapsing infranil fiber bundles.
Proposes new loss functions for better handling bimodal predictive uncertainty.
problem Bimodal predictive uncertainty in machine learning models.
method Family of distribution-aware loss functions integrating normalized RMSE with Wasserstein and Cramér distances.
result Proposed loss functions reduce predictive uncertainty estimation error by 45% on complex bimodal datasets.
Physics Informed Deep Kernel Learning improves prediction accuracy and uncertainty quantification.
problem Limited performance of deep kernel learning due to scarce or insufficient data.
method Integrates physics knowledge represented by differential equations with latent sources into deep kernel learning.
result Advantages in prediction accuracy and uncertainty quantification on synthetic and real-world datasets.
Paper uses virtual big data to improve autoencoder training and address imbalanced data classification.
problem Imbalanced data classification and autoencoder over-fitting.
method Cross-concatenation using Virtual Big Data.
result Cross-concatenation method effectively balances imbalanced class distributions.
Mathematical analysis shows annealing prevents mode collapse in Gaussian mixtures.
problem Mode collapse in variational inference for multimodal distributions.
method Analyzed annealing strategies for Gaussian mixtures, derived formulas, and tested on neural networks.
result Appropriately chosen annealing schemes can robustly prevent mode collapse.
The study characterizes and rules out collapsing in convex ancient mean curvature flow.
problem Characterizing and ruling out collapsing in convex ancient mean curvature flow.
method Characterization and counterexamples.
result Collapsing occurs if and only if the flow is asymptotic to at least one Grim hyperplane.
We introduce the Pitman Yor Diffusion Tree (PYDT) for hierarchical clustering, a generalization of the Dirichlet Diffusion Tree (Neal, 2001) which removes the restriction to binary branching structure. The generative process is described and shown to result in an exchangeable distribution over data points. We prove som…
Despite excellent progress in recent years, mode collapse remains a major unsolved problem in generative adversarial networks (GANs).In this paper, we present spectral regularization for GANs (SR-GANs), a new and robust method for combating the mode collapse problem in GANs. Theoretical analysis shows that the optimal …
New method controls posterior collapse in VAEs without network architecture constraints.
problem Posterior collapse in VAEs reduces diversity of generated samples.
method Introduces Latent Reconstruction (LR) loss to control posterior collapse.
result Controls posterior collapse on various datasets without architectural constraints.
Special Lagrangian submanifolds emerge from K3 surface collapse.
problem Understanding special Lagrangian submanifolds in K3 surface collapse.
method Lifting affine lines to degenerating sequences of special Lagrangian submanifolds.
result Constructing special Lagrangian two-spheres connecting Taub-NUT bubbles.
Collapsibility is a combinatorial strengthening of contractibility. We relate this property to metric geometry by proving the collapsibility of any complex that is CAT(0) with a metric for which all vertex stars are convex. This strengthens and generalizes a result by Crowley. Further consequences of our work are: (1) …
Two-dimensional collapsed spaces with lower Ricci bounds are topological surfaces.
problem Topology of collapsed spaces with lower Ricci bounds
method Prove that collapsed spaces are topological surfaces
result Collapsed spaces are topological surfaces
Prove that collapsing CSC metrics can be perturbed to invariant collapsing CSC metrics.
problem Prove that collapsing constant scalar curvature metrics can be perturbed to invariant collapsing constant scalar curvature metrics.
method Prove that a sequence of constant scalar curvature metrics which is collapsing with bounded curvature to a manifold can be perturbed to a sequence of invariant collapsing constant scalar curvature metrics.
result Prove that a sequence of constant scalar curvature metrics which is collapsing with bounded curvature to a manifold can be perturbed to a sequence of invariant collapsing constant scalar curvature metrics.
The paper proposes a new risk model for foundation models in finance.
problem Understanding how foundation models affect trading strategies' risk and return.
method An extension of the CAPM, separating systematic and idiosyncratic risks.
result Monte Carlo dropout measures the epistemic risk of foundation models.
We will simplify the earlier proofs of Perelman's collapsing theorem of 3-manifolds given by Shioya-Yamaguchi and Morgan-Tian. Among other things, we use Perelman's semi-convex analysis of distance functions to construct the desired local Seifert fibration structure on collapsed 3-manifolds. The verification of Perelma…
Estimate collapsibility of causal effects in CPDAGs via strong d-convex hulls.
problem Estimate causal effects in CPDAGs.
method Use strong d-convex hulls to characterize minimal collapsible sets.
result Efficient algorithm for obtaining collapsible sets in DAGs and CPDAGs.
We introduce the theory of strong homotopy types of simplicial complexes. Similarly to classical simple homotopy theory, the strong homotopy types can be described by elementary moves. An elementary move in this setting is called a strong collapse and it is a particular kind of simplicial collapse. The advantage of usi…