This paper uses reference priors to improve deep learning models with unlabeled and labeled data.
problem Improving deep learning models with limited labeled data and unlabeled data from the same or related tasks.
method Develops and applies generalizations of reference priors for deep networks to exploit unlabeled and labeled data.
result Demonstrates new semi-supervised learning and pretraining methods for transfer learning.
Informative Bayesian priors are often difficult to elicit, and when this is the case, modelers usually turn to noninformative or objective priors. However, objective priors such as the Jeffreys and reference priors are not tractable to derive for many models of interest. We address this issue by proposing techniques fo…
Optimality of TS with noninformative priors proven for Pareto model.
problem Optimality of Thompson Sampling with noninformative priors for Pareto bandits.
method Proved optimality of TS with certain probability matching priors, showed suboptimality with others, and found effectiveness of truncation procedures.
result TS with certain probability matching priors achieves optimal regret bound for Pareto model.
We present an iterative Markov chainMonte Carlo algorithm for computingreference priors and minimax risk forgeneral parametric families. Ourapproach uses MCMC techniques based onthe Blahut-Arimoto algorithm forcomputing channel capacity ininformation theory. We give astatistical analysis of the algorithm,bounding the n…
New method detects if data points were used in training models with low cost and high power.
problem Detecting if a particular data point was used in training a model.
method Fine-grained modeling of null hypothesis in likelihood ratio tests, leveraging reference models and population data.
result RMIA has superior test power compared to prior methods, even at extremely low false positive rates.
Two data-dependent information metrics are developed to quantify the information of the prior and likelihood functions within a parametric Bayesian model, one of which is closely related to the reference priors from Berger, Bernardo, and Sun, and information measure introduced by Lindley. A combination of theoretical, …
A new method samples from multi-modal distributions without hyperparameter tuning.
problem Sampling from multi-modal distributions is challenging and requires tuning hyperparameters.
method Learned Reference-based Diffusion Sampler (LRDS) that learns a reference model on high-density regions and uses it to train a diffusion-based sampler.
result LRDS best exploits prior knowledge on multi-modal distributions compared to competing algorithms.
Proposes a new prior for complex models to improve prediction accuracy.
problem Difficulty in specifying priors for complex models like neural networks.
method Predictive complexity priors defined by comparing model predictions to a reference model, transferred to parameters via change of variables.
result Improves model predictions by reducing unintuitive effects of traditional priors.
Unified definition of hallucinations in language models.
problem Persistent hallucinations despite mitigation efforts.
method Unified definition of hallucination as inaccurate world modeling.
result Unified framework distinguishes hallucinations from other errors.
A salient approach to interpretable machine learning is to restrict modeling to simple models. In the Bayesian framework, this can be pursued by restricting the model structure and prior to favor interpretable models. Fundamentally, however, interpretability is about users' preferences, not the data generation mechanis…
Bayesian PROCOVA uses AI to adjust for covariates in RCTs.
problem Unbiased and precise treatment effect inferences from RCTs.
method Generative AI constructs digital twins for covariate adjustment, using an additive mixture prior.
result Efficiency gains in smaller RCTs compared to frequentist methods.
Improves AI-prior reliability for Bayesian inference.
problem Error propagation from predictive models into posterior inference.
method Rectified AI-informed prior elicitation framework.
result Significant reduction in bias and improvement in predictive performance.
RP-WNO extends WNO with uncertainty quantification, useful for scientists and engineers.
problem Uncertainty in predictions of deep learning models.
method Randomized Prior Wavelet Neural Operator (RP-WNO) with uncertainty quantification module.
result RP-WNO effectively estimates uncertainty in predictions.
In this paper, we develop a Bayesian evidence maximization framework to solve the sparse non-negative least squares (S-NNLS) problem. We introduce a family of probability densities referred to as the Rectified Gaussian Scale Mixture (R- GSM) to model the sparsity enforcing prior distribution for the solution. The R-GSM…
Adaptive optimal transport priors improve few-shot learning robustness.
problem Limited supervision and distribution shifts in few-shot learning.
method Prototype-Guided Distributionally Robust Optimization (PG-DRO) framework.
result PG-DRO achieves stronger robust generalization in few-shot scenarios.
This paper presents a Bayesian image segmentation model based on Potts prior and loopy belief propagation. The proposed Bayesian model involves several terms, including the pairwise interactions of Potts models, and the average vectors and covariant matrices of Gauss distributions in color image modeling. These terms a…
We compute the expected value of the Kullback-Leibler divergence to various fundamental statistical models with respect to canonical priors on the probability simplex. We obtain closed formulas for the expected model approximation errors, depending on the dimension of the models and the cardinalities of their sample sp…
Method recovers complex-valued signals from speckle-noised measurements.
problem Recovering complex-valued signals from speckle-noised measurements.
method Bagged Deep Image Priors integrated with projected gradient descent and Newton-Schulz algorithm.
result Achieves state-of-the-art performance in MSE reduction.
New algorithms achieve logarithmic regret in KL-regularized Markov games.
problem Improving sample efficiency in game-theoretic settings with KL regularization.
method Developed OMG and SOMG algorithms for matrix and Markov games, using best response sampling and superoptimistic bonuses.
result Logarithmic regret in T that scales inversely with KL regularization strength β. Enhances CMS with Bayesian nonparametrics for better low-frequency token estimation.
problem Improving frequency estimation of low-frequency tokens in data streams.
method Integrates Bayesian nonparametrics (Pitman-Yor process) into CMS for more accurate frequency estimation.
result CMS-PYP outperforms CMS and CMS-DP in estimating low-frequency tokens.
New method uses quotient predictor space for better PAC-Bayes bounds, reducing KL divergence and improving model performance.
problem Overparameterized models with continuous symmetries can lead to biased predictions.
method Perform PAC-Bayesian analysis on quotient predictor space, constructing a canonical prior that reflects model's implicit bias.
result The new prior reduces KL divergence and improves model performance in experiments.
This study explores how choosing noninformative priors affects Thompson Sampling in multiparameter bandit models.
problem The optimality of Thompson Sampling (TS) in multiparameter bandit models depends on the choice of priors, especially when models are complex.
method The study extends regret analysis to uniform distributions and proposes a modified TS policy, TS-T, to achieve asymptotic optimality.
result Changing noninformative priors can significantly affect the expected regret in multiparameter bandit models.
Unified sampling approach for Bayesian imaging problems.
problem Sampling from complex prior and posterior distributions in Bayesian imaging.
method Gaussian latent machine model for efficient prior and posterior sampling.
result Unified and generalized sampling algorithms for various imaging problems.
Develops probabilistic models for gene regulatory network inference.
problem Challenges in reconstructing gene regulatory networks from genome-wide data.
method Two complementary frameworks: PMF-GRN and GLM-Prior.
result Probabilistic inference refines regulatory estimates with quantified uncertainty.
Decentralized learning achieves centralized performance via Gibbs measures.
problem Achieving centralized performance in decentralized machine learning.
method ERM-RER learning framework with Gibbs measures and relative-entropy regularization.
result Achieving centralized performance with Gibbs measures and specific scaling of regularization factors.
Deep learning tackles low-photon nanoscale holographic phase retrieval.
problem Low-photon imaging challenges at nanoscale.
method Dataset-free deep learning framework with physical model integration.
result Significantly improves signal recovery from higher noise levels.
Smooth Schrödinger Bridges improve trajectory inference by smoothing Gaussian processes.
problem Improving trajectory inference in applications like particle tracking.
method Generalizes Schrödinger Bridge problem to smooth Gaussian processes, solving the problem on phase space.
result The method outperforms existing methods on real datasets.
Symbolic regression improved by incorporating prior knowledge.
problem Insufficient guidance from training data alone for model accuracy.
method Multi-objective symbolic regression combining training data and prior constraints.
result Models that fit training data well and comply with prior knowledge.
Study improves fractional posterior for 1-bit matrix completion.
problem Estimating a binary matrix from observed entries.
method Fractional posterior approach with low-rank factorization and spectral scaled Student priors.
result Concentration results for fractional posterior, demonstrating effectiveness in matrix recovery.
Physics-informed kernel learning integrates physical priors into machine learning models.
problem Tackles the integration of physical laws into machine learning models for improved accuracy and efficiency.
method Uses Fourier methods to approximate the kernel and minimizes a physics-informed risk function.
result Demonstrates PIKL outperforms physics-informed neural networks and traditional PDE solvers in various scenarios.
Survey of integrating physics knowledge into machine learning models.
problem Mitigating data shortage and ensuring physical plausibility.
method Combining physics knowledge with machine learning models.
result Summarizes recent works in physics-informed machine learning.
GLIME improves LIME's stability and local fidelity.
problem LIME's instability and low local fidelity.
method Introducing GLIME, an enhanced framework that derives an equivalent formulation of LIME with faster convergence and improved stability.
result GLIME generates explanations with higher local fidelity and is independent of reference choice.
Quality-designed consumer products are easy to recognize. Wouldn't it be great if the quality of financial products became just as apparent? This paper is addressed to financial practitioners. It provides an informal introduction to Quantitative Structuring -- a technology of manufacturing quality financial products (i…
Method identifies low-dimensional structure in high-dimensional probability measures.
problem Identifying low-dimensional structure in high-dimensional probability measures.
method Extends prior work on minimizing majorizations of the Kullback-Leibler divergence to identify optimal approximations within a specific class of measures.
result Connection between dimensional logarithmic Sobolev inequality and approximations with the ansatz.
Neural model predicts object states and physical parameters from visual observations.
problem Computational models struggle with physical reasoning and adapting to new environments.
method Visual prior predicts particle-based system from visual observations; inference module refines estimates subject to dynamics constraints.
result Model can infer physical properties within a few observations and adapt to unseen scenarios.
This paper introduces a new semi-parametric approach to the pricing and risk management of bespoke CDO tranches, with a particular attention to bespokes that need to be mapped onto more than one reference portfolio. The only user input in our framework is a multi-factor model (a "prior" model hereafter) for index portf…
Proposes scale mixture of NNGPs for more flexible stochastic processes.
problem Limited focus on broadening the class of stochastic processes from NNGPs.
method Scale mixture of NNGPs with scale priors on last-layer parameters.
result Turns neural networks into a richer class of stochastic processes.
Non-negative blind source separation (non-negative BSS), which is also referred to as non-negative matrix factorization (NMF), is a very active field in domains as different as astrophysics, audio processing or biomedical signal processing. In this context, the efficient retrieval of the sources requires the use of sig…
New method learns disentangled representations using Gromov-Monge maps.
problem Learning disentangled representations from unlabelled data.
method Introduces a novel approach based on Gromov-Monge maps to preserve geometric features while aligning data distributions.
result Demonstrates effectiveness on four benchmarks, outperforming other methods.
Most policy search algorithms require thousands of training episodes to find an effective policy, which is often infeasible with a physical robot. This survey article focuses on the extreme other end of the spectrum: how can a robot adapt with only a handful of trials (a dozen) and a few minutes? By analogy with the wo…
The paper tackles fairness in data and algorithms, expanding on prior work.
problem Discrimination and disparate treatment in data and algorithms.
method Targeted learning for nonparametric inference of fairness in the data generating process.
result Derivation and validation of estimators for fairness metrics like demographic parity and equal opportunity.
We propose a Bayesian approximate inference method for learning the dependence structure of a Gaussian graphical model. Using pseudo-likelihood, we derive an analytical expression to approximate the marginal likelihood for an arbitrary graph structure without invoking any assumptions about decomposability. The majority…
In many problem settings, parameter vectors are not merely sparse but dependent in such a way that non-zero coefficients tend to cluster together. We refer to this form of dependency as "region sparsity." Classical sparse regression methods, such as the lasso and automatic relevance determination (ARD), which model par…
In prior work the authors introduced a parabolic flow for pluriclosed metrics, referred to as pluriclosed flow. We also demonstrated that this flow, after certain gauge transformations, gives a class of solutions to the renormalization group flow of the nonlinear sigma model with B-field. Using these transformations, w…
A new ensemble filter uses transport maps and MMD optimization for high-dimensional data assimilation.
problem High-dimensional data assimilation challenges in ensemble filtering.
method Optimized Maximum Mean Discrepancy (MMD) for transport map construction.
result Significant improvement in robustness and posterior approximation.
Paper proposes conditional multidimensional scaling for better data reduction.
problem Mapping high-dimensional data to low-dimensional space with known features.
method Developed a broad class of methods called conditional multidimensional scaling (MDS) with an optimization algorithm.
result Conditional MDS improves estimation quality and simplifies visualization and knowledge discovery.
T-LoHo model detects structured sparsity and smoothness on graph data.
problem Detecting structured sparsity and smoothness in graph-structured data.
method Tree-based Low-rank Horseshoe (T-LoHo) prior for multivariate parameters.
result Improves anomaly detection on road networks compared to other methods.
New DEC variant improves sample complexity bounds in decision making.
problem Understanding sample-efficient learning guarantees in decision making.
method Introducing a new Constrained Decision-Estimation Coefficient (DEC) and using it to derive improved lower bounds.
result New lower bounds improve upon prior work in three aspects: expectation, global applicability, and improper reference models.