Bayesian quadrature improves integration efficiency with invariant priors.
problem Efficient numerical integration with known structure.
method Invariance priors for bijective transformations in input domain.
result Superior performance in synthetic and real-world applications.
Statsformer validates and adapts LLM-derived semantic priors for improved supervised learning.
problem Unreliable semantic priors from LLMs can degrade supervised learning performance.
method Adapts LLM-derived feature scores into a family of learner-specific prior-injection mechanisms, calibrating their influence using out-of-fold validation.
result Improves prediction performance by adaptively downweighting unreliable LLM priors, ensuring a guardrailed statistical learning system.
The paper proposes a method to integrate prior information into penalized regression.
problem Improving predictive performance in high-dimensional tasks with prior information.
method Integrating multiple sources of prior information into penalized regression.
result The method improves predictive performance, as shown by simulations and applications.
Proposes a method to integrate prior knowledge into trajectory prediction models.
problem Improving accuracy and robustness in trajectory prediction models.
method Continual learning approach that allows integration of arbitrary prior knowledge and probabilistic predictions.
result Outperforms non-informed and informed learning methods, using half as many observation examples.
Transformer-based method for causal discovery with prior knowledge integration.
problem Complex nonlinear dependencies and spurious correlations in time series data.
method Multi-layer Transformer forecaster with gradient-based causal structure extraction and attention masking for prior knowledge integration.
result Significant improvement in causal discovery and causal lag estimation compared to state-of-the-art methods.
Thermodynamic integration (TI) for computing marginal likelihoods is based on an inverse annealing path from the prior to the posterior distribution. In many cases, the resulting estimator suffers from high variability, which particularly stems from the prior regime. When comparing complex models with differences in a …
We propose a novel approach for nonlinear regression using a two-layer neural network (NN) model structure with sparsity-favoring hierarchical priors on the network weights. We present an expectation propagation (EP) approach for approximate integration over the posterior distribution of the weights, the hierarchical s…
InfoSEM infers gene regulatory networks without GT labels, improving performance.
problem Inferring GRNs from gene expression data with high accuracy and avoiding biases.
method InfoSEM uses deep generative models with informative priors (textual gene embeddings).
result InfoSEM outperforms existing models by 38.5% across four datasets.
This paper improves Gaussian process predictions by integrating prior knowledge.
problem Gaussian processes lack predictive power when prior information is ignored.
method Derive mean and covariance functions from previous data using weighted sums of basis functions.
result Integrating prior knowledge significantly increases look-ahead time and accuracy.
Bayesian neural networks with functional priors improve surrogate modeling in mechanics.
problem Challenges in integrating prior knowledge and quantifying uncertainties in high-dimensional NN parameter spaces.
method Anchored ensembling to integrate a priori information and learn low-rank correlations between NN parameters.
result Effective transfer of knowledge between function-space and parameter-space priors improves surrogate model accuracy and uncertainty estimation.
VampPrior Mixture Model improves clustering in DLVMs.
problem Simplicity of standard priors in DLVMs leads to poor clustering performance.
method Leverages VampPrior concepts to fit a Bayesian GMM prior in a VAE.
result VMM achieves highly competitive clustering performance on benchmark datasets.
Paper examines stability of Bayesian posterior measures using integral probability metrics.
problem Stability of Bayesian inference in large-scale inverse problems.
method New families of integral probability metrics for likelihood and prior perturbations.
result Constructs new stability results for Bayesian posterior measures.
We add prior knowledge to deep networks to make them invariant to transformations.
problem Creating deep networks invariant to transformations like rotation.
method A novel layer based on invariant integration to enforce feature space invariances.
result State-of-the-art performance on the Rotated-MNIST dataset.
A new data-adaptive prior stabilizes kernel learning in operators.
problem Learning kernels in operators from data is ill-posed due to nonlocal dependence.
method Introduces a data-adaptive prior to stabilize the Bayesian posterior mean.
result The data-adaptive prior achieves a stable posterior with small noise limits.
Novel method combines physics priors for energy-conserving dynamics.
problem Learning long-term dynamics of complex physical systems from noisy data.
method Variational Integrator Graph Networks integrating energy constraint, high-order symplectic integrators, and graph neural networks.
result Improves predictive performance across single and many-body problems.
We introduce a model-based reconstruction framework with deep learned (DL) and smoothness regularization on manifolds (STORM) priors to recover free breathing and ungated (FBU) cardiac MRI from highly undersampled measurements. The DL priors enable us to exploit the local correlations, while the STORM prior enables us …
New bounds use IPMs to improve generalization in machine learning.
problem Improving generalization bounds in machine learning.
method PAC-Bayes bounds with Integral Probability Metrics (IPM).
result Natural interpolation between worst-case and favorable cases.
Survey of integrating physics knowledge into machine learning models.
problem Mitigating data shortage and ensuring physical plausibility.
method Combining physics knowledge with machine learning models.
result Summarizes recent works in physics-informed machine learning.
Residual Prior Diffusion integrates coarse latent priors with diffusion models for better generative tasks.
problem Diffusion models struggle with representing both large-scale and fine-scale details in data distributions.
method Two-stage framework: first a coarse prior model captures large-scale structure, then a diffusion model represents the residual.
result RPD accurately captures fine-scale details while preserving large-scale structure, outperforming standard diffusion models.
The Fredholm integral equation of the first kind improves solutions for ill-posed supervised learning problems with limited data.
problem Ill-posed supervised learning problems with insufficient data.
method Using the Fredholm integral equation of the first kind (FIFK) with semi-supervised assumptions and MSDF methods.
result Improved accuracy and stability in solutions for ill-posed problems.
Bayesian Probabilistic Integration uses BART for high-dimensional, non-smooth functions.
problem Bayesian quadrature's limitations in high-dimensional or non-smooth functions.
method Bayesian Additive Regression Trees (BART) priors for numerical integration.
result Explicit convergence rates can be obtained in various settings.
This paper improves image super-resolution by integrating cross-scale non-local attention.
problem Improving image super-resolution by leveraging long-range and cross-scale feature correlations.
method Proposes a Cross-Scale Non-Local (CS-NL) attention module integrated into a recurrent neural network.
result Significantly improved performance on SISR benchmarks.
DABS uses a policy network to select experiments in high-dimensional design spaces.
problem Adaptive factorial screening in high-dimensional discrete design spaces.
method DABS learns a policy network offline to sequentially select experiments, incorporating sparsity and interactions via a spike-and-slab prior.
result DABS achieves superior accuracy and scalability over classical and Bayesian baselines under tight experimental budgets.
Gaussian processes are ubiquitous in nature and engineering. A case in point is a class of neural networks in the infinite-width limit, whose priors correspond to Gaussian processes. Here we perturbatively extend this correspondence to finite-width neural networks, yielding non-Gaussian processes as priors. The methodo…
Paper introduces a method to generate physically feasible dynamics with physical priors.
problem Challenges in generating physically feasible dynamics under physical priors.
method Seamlessly incorporates physical priors into diffusion-based generative models.
result Efficient generation of physically realistic dynamics across various physical phenomena.
Probabilistic solvers improve stability for stiff systems.
problem Performance penalties for small steps in stiff systems.
method Probabilistic exponential integrators that include fast linear dynamics in the prior.
result Proven L-stability and probabilistic error accounting.
Bayesian models use hyperparameters to indirectly assign priors, and this work shows how these priors can be derived from maximum entropy principles.
problem Understanding the assumptions and dependencies in Bayesian hierarchical models.
method Demonstrates how canonical distributions and maximum entropy principles can be used to derive marginal priors in hierarchical models.
result Marginal priors in hierarchical models derived from maximum entropy principles have different constraints compared to the original priors.
This paper analyzes and guarantees convergence of prior-guided ZO algorithms.
problem Understanding convergence properties of prior-guided zeroth-order optimization algorithms.
method Analysis of convergence under a greedy descent framework with various gradient estimators, and development of ARS algorithm.
result Convergence guarantee for prior-guided random gradient-free (PRGF) algorithms and accelerated random search (ARS) algorithm.
Unified framework for data-driven priors in Bayesian inverse problems
problem Bayesian inverse problems
method Unified framework using score functions
result Evaluation of four data-driven priors
Bayesian approach improves rain field reconstruction using CMLs and DMs.
problem Challenges in accurately reconstructing ground-level rainfall from CML path-integrated measurements.
method Bayesian inverse problem with Diffusion Models as priors.
result Improved performance in rainfall estimation compared to existing methods.
Bayesian Hierarchical Invariant Prediction refines ICP for better scalability and prior integration.
problem Improving computational scalability and invariance testing for causal inference.
method Bayesian Hierarchical structure to test invariance under heterogeneous data.
result Demonstrated improved scalability and potential as an alternative to ICP.
Physics-informed kernel learning integrates physical priors into machine learning models.
problem Tackles the integration of physical laws into machine learning models for improved accuracy and efficiency.
method Uses Fourier methods to approximate the kernel and minimizes a physics-informed risk function.
result Demonstrates PIKL outperforms physics-informed neural networks and traditional PDE solvers in various scenarios.
Bayesian neural networks incorporate domain knowledge through variational inference.
problem Specifying priors for Bayesian neural networks that capture domain knowledge is challenging.
method Proposes a framework for integrating domain knowledge into BNN priors through variational inference.
result BNNs with proposed domain knowledge priors outperform those with standard priors, achieving better predictive performance.
Boltzmann machines are powerful distributions that have been shown to be an effective prior over binary latent variables in variational autoencoders (VAEs). However, previous methods for training discrete VAEs have used the evidence lower bound and not the tighter importance-weighted bound. We propose two approaches fo…
We study the problem of generating adversarial examples in a black-box setting in which only loss-oracle access to a model is available. We introduce a framework that conceptually unifies much of the existing work on black-box attacks, and we demonstrate that the current state-of-the-art methods are optimal in a natura…
Bayesian framework for sphere regression using Gaussian fields.
problem Nonparametric regression on the sphere with Gaussian priors.
method Isotropic Gaussian field priors, harmonic structure, exact posterior distributions, optimal spectral truncation, posterior contraction rates.
result Sharp posterior contraction rates for Gaussian priors with polynomially decaying angular power spectra.
Imagine that measurements are made at times t0 and t1 of the trajectory of a physical system whose governing laws are given approximately by a class A of so-called {\em prior vector fields}. Because the physical laws are not known precisely, it might be that the measurements are not realised by the integ…
Variational dropout (VD) is a generalization of Gaussian dropout, which aims at inferring the posterior of network weights based on a log-uniform prior on them to learn these weights as well as dropout rate simultaneously. The log-uniform prior not only interprets the regularization capacity of Gaussian dropout in netw…
Proposes robust ITRs integrating multiple datasets to handle posterior shift.
problem Posterior shift in conditional outcome distributions between source and target populations.
method Distributionally robust approach with closed-form solution and adaptive uncertainty tuning.
result Achieves superior performance compared to existing methods in simulations and real-data applications.
In this paper we present decomposable priors, a family of priors over structure and parameters of tree belief nets for which Bayesian learning with complete observations is tractable, in the sense that the posterior is also decomposable and can be completely determined analytically in polynomial time. This follows from…
Unified framework for convergence of discrete diffusion models without state space size dependence.
problem Fundamental limitations in existing convergence theory for discrete diffusion models, especially under singular priors and large vocabularies.
method Unified adjoint-equation-based framework that establishes dimension-free convergence guarantees in any integral probability metric (IPM).
result First dimension-free convergence bounds applicable to both masked and uniform priors, free of state space size S. Autonomous terrestrial vehicles must be capable of perceiving traffic lights and recognizing their current states to share the streets with human drivers. Most of the time, human drivers can easily identify the relevant traffic lights. To deal with this issue, a common solution for autonomous cars is to integrate recog…
The lack of interpretability often makes black-box models difficult to be applied to many practical domains. For this reason, the current work, from the black-box model input port, proposes to incorporate data-based prior information into the black-box soft-margin SVM model to enhance its interpretability. The concept …
PRISM-VQ combines financial priors with vector quantization for better stock prediction.
problem Predicting cross-sectional stock returns is hard due to low signal-to-noise ratios and changing market conditions.
method Integrates expert priors, vector-quantized latent factors, and dynamic factor loadings.
result Consistent improvements in cross-sectional return prediction and portfolio performance.
New framework uses score-based priors to solve ill-conditioned polynomial equations, improving signal recovery from noisy data.
problem Recovering signals from low-order moments in inverse problems, especially ill-conditioned polynomial equations.
method Integrates score-based diffusion priors with moment-based estimators to regularize and solve nonlinear inverse problems.
result Diffusion priors improve recovery from third-order moments and make super-resolution MTD feasible.
Regularization methods, specifically those which directly alter weights like L1 and L2, are an integral part of many learning algorithms. Both the regularizers mentioned above are formulated by assuming certain priors in the parameter space and these assumptions, in some cases, induce sparsity in the parameter sp…
Introduces DF framework for sampling decisions from target distributions.
problem Sampling from target distributions with additional guidance.
method DF framework based on MDP and Path Integral Diffusion.
result DF enhances guided sampling across various applications.
Adaptive optimal transport priors improve few-shot learning robustness.
problem Limited supervision and distribution shifts in few-shot learning.
method Prototype-Guided Distributionally Robust Optimization (PG-DRO) framework.
result PG-DRO achieves stronger robust generalization in few-shot scenarios.