Bayesian model improves image completion accuracy by automatically learning low rank structure.
problem Improving image completion accuracy with limited data and avoiding overfitting.
method Developed a Bayesian low rank tensor ring model with multiplicative interaction and Student-T distribution for sparse core factors.
result The proposed method outperforms state-of-the-art image completion techniques, especially in recovery accuracy.
TFB simplifies Bayesian LLM uncertainty estimation without extra training.
problem Estimating uncertainty in LLM responses remains challenging.
method Training-Free Bayesianization (TFB) that transforms low-rank adapters into Bayesian ones without additional training.
result TFB achieves superior uncertainty estimation and generalization compared to existing methods.
The problem of low-rank matrix estimation recently received a lot of attention due to challenging applications. A lot of work has been done on rank-penalized methods and convex relaxation, both on the theoretical and applied sides. However, only a few papers considered Bayesian estimation. In this paper, we review the …
This paper presents a Bayesian method for estimating the rank of a low-rank tensor model of joint PMF.
problem Estimating the rank of a low-rank tensor model of joint PMF from observed data.
method Bayesian framework for estimating low-rank components and rank simultaneously, using variational inference.
result Automatic rank detection and improved estimation accuracy compared to cross-validation methods.
BRTR improves robust tensor completion with automatic rank detection.
problem Robust tensor completion from incomplete data with outliers.
method Bayesian robust tensor ring decomposition (BRTR) with variational Bayesian (VB) algorithm.
result Automatic detection of TR rank and improved performance over state-of-the-art methods.
Rank-based Bayesian Optimization improves molecule selection in chemical systems.
problem Optimizing chemical compounds using traditional regression models.
method Introducing Rank-based Bayesian Optimization (RBO) using ranking models.
result RBO outperforms regression-based BO, especially for rough landscapes and activity cliffs.
QS-BO optimizes functions using only rank-based feedback.
problem Optimizing expensive functions with unreliable or unavailable metric values.
method Quantile-scaling pipeline to convert ranks into Gaussian targets.
result QS-BO consistently achieves lower objective values and is statistically significant.
Bayesian TNKMs automatically infer model complexity and feature relevance.
problem Manual tuning of TN rank and feature dimensions is error-prone and computationally expensive.
method Bayesian approach with hierarchical priors on TN factors for automatic rank and feature selection.
result Superior performance in prediction accuracy, uncertainty quantification, interpretability, and scalability.
Bayesian approach improves AdaLoRA's performance and efficiency.
problem Improving the efficiency and performance of adaptive low-rank adaptation.
method Utilized Bayesian metrics and the Improved Variational Online Newton (IVON) optimizer for adaptive parameter budget allocation.
result Bayesian counterpart outperforms sensitivity-based importance metric and is faster than AdaLoRA.
The problem of low rank matrix completion is considered in this paper. To exploit the underlying low-rank structure of the data matrix, we propose a hierarchical Gaussian prior model, where columns of the low-rank matrix are assumed to follow a Gaussian distribution with zero mean and a common precision matrix, and a W…
We develop latent variable models for Bayesian learning based low-rank matrix completion and reconstruction from linear measurements. For under-determined systems, the developed methods are shown to reconstruct low-rank matrices when neither the rank nor the noise power is known a-priori. We derive relations between th…
Paper introduces MPES for top-k ranking BO with preferential observations.
problem Handling top-k ranking and tie/indifference observations in Bayesian optimization.
method Designs a surrogate model and introduces MPES acquisition function.
result MPES outperforms existing acquisition functions in handling preferential observations.
Bayesian model ranks treatments in multi-response experiments.
problem Identifying the best treatment among competing ideal properties.
method Bayesian approach with Markov Chain Monte Carlo algorithm.
result Reliable inference of treatment ranks in practice.
Study the distribution for low-rank matrix learning, improving inference methods.
problem Lack of understanding of underlying probability distributions in low-rank matrix learning.
method Analyze the distribution f(X)∝e−λ∥X∥∗, using differential geometry to design an improved MCMC algorithm and learn penalty parameter λ. result Improved MCMC algorithm and penalty parameter learning for low-rank Bayesian inference.
In this paper, we provide a method to learn the directed structure of a Bayesian network using data. The data is accessed by making conditional probability queries to a black-box model. We introduce a notion of simplicity of representation of conditional probability tables for the nodes in the Bayesian network, that we…
We consider the problem of modeling multivariate time series with parsimonious dynamical models which can be represented as sparse dynamic Bayesian networks with few latent nodes. This structure translates into a sparse plus low rank model. In this paper, we propose a Gaussian regression approach to identify such a mod…
Proposes a new method to improve Bayesian computation accuracy using flexible classification.
problem Bayesian computations accuracy check using rank-based simulation-based calibration has limitations.
method Replaces marginal rank test with a flexible classification approach that learns from data.
result Improves statistical power and provides an interpretable divergence measure of miscalibration.
Compact parameterization improves Bayesian neural network performance.
problem Improving performance of Bayesian neural networks using variational methods.
method Restricting variational distribution to a k-tied Normal distribution with low-rank factorization.
result Compact parameterization improves signal-to-noise ratio and convergence speed.
Bayesian model infers strengths from noisy tennis match outcomes.
problem Ranking tennis players from match outcomes.
method Bayesian approach to infer unobserved strengths and mapping function.
result Bayesian approach robust to different model specifications.
Rank-1 BNNs improve efficiency and scalability of Bayesian neural nets.
problem Underfitting and lack of scalability in Bayesian neural networks.
method Propose a rank-1 parameterization of BNNs and use mixture approximate posteriors.
result Rank-1 BNNs achieve state-of-the-art performance across various datasets.
Bayesian neural networks can be simplified by parameterizing weights as rank-r matrices, reducing parameter count and improving performance.
problem High parameter count in standard Bayesian neural networks.
method Parameterize weights as W=ABop with A∈Rmimesr, B∈Rnimesr, inducing a singular posterior. result PAC-Bayes generalization bounds and loss bounds show improved performance with fewer parameters.
New method calibrates LLMs for safety-critical tasks with scalable Bayesian inference.
problem Overconfidence in LLMs after fine-tuning for specific tasks.
method Orthogonalized Low-Rank Adapters (PoLAR) with variational Bayesian inference.
result Scalable and well-calibrated uncertainty estimation for LLMs.
This paper is devoted to the bipartite ranking problem, a classical statistical learning task, in a high dimensional setting. We propose a scoring and ranking strategy based on the PAC-Bayesian approach. We consider nonlinear additive scoring functions, and we derive non-asymptotic risk bounds under a sparsity assumpti…
Machine learning provides algorithms that can learn from data and make inferences or predictions on data. Bayesian networks are a class of graphical models that allow to represent a collection of random variables and their condititional dependencies by directed acyclic graphs. In this paper, an inference algorithm for …
Paper compares Bayesian and de-biased estimators for low-rank matrix completion.
problem Predict missing entries in partially observed matrices.
method Bayesian and de-biased estimators comparison.
result De-biased estimator performs similarly to Bayesian estimators but is more stable and can outperform in small samples.
This paper develops a matrix-variate adaptive Markov chain Monte Carlo (MCMC) methodology for Bayesian Cointegrated Vector Auto Regressions (CVAR). We replace the popular approach to sampling Bayesian CVAR models, involving griddy Gibbs, with an automated efficient alternative, based on the Adaptive Metropolis algorith…
BoRA finetunes multi-task LLMs by sharing information through hierarchical priors.
problem Limited data for some tasks in multi-task LLMs.
method Bayesian hierarchical low-rank adaption.
result BoRA outperforms individual and unified model approaches.
Bayesian framework for sequential learning tasks with low-rank approximations.
problem Balancing knowledge retention and adaptability in sequential neural networks.
method Bayesian framework with diagonal plus low-rank approximations of the precision matrix.
result Unlocking capabilities to encode task relationships and incorporate prior knowledge from later tasks.
Develops a method to infer partial rankings from sparse comparisons.
problem Challenges in ranking items with limited and noisy comparisons.
method Nonparametric Bayesian approach for learning partial rankings.
result Finds partial rankings that distinguish meaningful differences only when data supports it.
Tensor decomposition is an effective approach to compress over-parameterized neural networks and to enable their deployment on resource-constrained hardware platforms. However, directly applying tensor compression in the training process is a challenging task due to the difficulty of choosing a proper tensor rank. In o…
BLoB fine-tunes LLMs with Bayesian methods to improve uncertainty estimation.
problem Overconfidence in LLMs during inference for domain-specific tasks.
method Continuous Bayesian low-rank adaptation during fine-tuning.
result BLoB improves generalization and uncertainty estimation in LLMs.
Recovery of low-rank matrices has recently seen significant activity in many areas of science and engineering, motivated by recent theoretical results for exact reconstruction guarantees and interesting practical applications. A number of methods have been developed for this recovery problem. However, a principled meth…
CANDECOMP/PARAFAC (CP) tensor factorization of incomplete data is a powerful technique for tensor completion through explicitly capturing the multilinear latent factors. The existing CP algorithms require the tensor rank to be manually specified, however, the determination of tensor rank remains a challenging problem e…
SetRank tackles collaborative ranking from implicit feedback using setwise Bayesian approach.
problem Challenges in pairwise and listwise approaches for implicit feedback.
method SetRank is a novel setwise Bayesian approach that accommodates implicit feedback characteristics.
result SetRank outperforms state-of-the-art baselines on real-world datasets.
Bayesian model compares ML algorithms on various datasets.
problem Comparing multiple machine learning algorithms across multiple datasets.
method Bayesian Bradley-Terry model, defining regions of practical equivalence (ROPE).
result Allows nuanced statements and ROPE definitions for algorithm comparison.
This paper tackles ranking-based performance normalization for optimization algorithms.
problem Ranking optimization algorithms across diverse numerical scales disrupts performance comparisons.
method Introduces absolute ranking and a sampling-based computational method to address numerical scale variation.
result Provides a more robust framework for assessing performance across multiple algorithms and problems.
Algorithm ranks assets in fluctuating markets.
problem Ranking assets in nonstationary time series.
method Naive Bayes asset ranker that adjusts weights based on performance.
result Outperforms traditional methods and S&P 500 index.
New Bayesian matrix completion method using Stiefel manifolds.
problem Efficient Bayesian matrix completion with uncertainty quantification.
method Geodesic Hamiltonian Monte Carlo on Stiefel manifolds.
result Improved sampling performance and accuracy on real-world problems.
Streaming tensor factorization is a powerful tool for processing high-volume and multi-way temporal data in Internet networks, recommender systems and image/video data analysis. Existing streaming tensor factorization algorithms rely on least-squares data fitting and they do not possess a mechanism for tensor rank dete…
We consider the problem of discriminative factor analysis for data that are in general non-Gaussian. A Bayesian model based on the ranks of the data is proposed. We first introduce a new {\em max-margin} version of the rank-likelihood. A discriminative factor model is then developed, integrating the max-margin rank-lik…
Bayesian method reduces misclassification errors in ranking Pareto-optimal solutions.
problem Identifying true Pareto-optimal solutions in noisy multiobjective optimization.
method Sequential allocation of extra samples using stochastic kriging to build predictive distributions.
result The proposed method outperforms existing algorithms in reducing misclassification errors.
StatFEM uses low-rank approximations to scale Bayesian statFEM for high-dimensional problems.
problem Model misspecification and scalability in high-dimensional physical systems.
method Low-rank approximation of covariance matrix, Bayesian filtering, sparse data reconstruction.
result Reconstructs sparsely observed data-generating processes with minimal loss of information.
This paper develops a method to train compact neural networks with reduced memory and computational costs.
problem Training large neural networks consumes excessive resources and energy.
method End-to-end training framework using Bayesian tensor decomposition with automatic rank determination.
result The method achieves significant parameter reduction and maintains or improves accuracy.
Rank aggregation based on pairwise comparisons over a set of items has a wide range of applications. Although considerable research has been devoted to the development of rank aggregation algorithms, one basic question is how to efficiently collect a large amount of high-quality pairwise comparisons for the ranking pur…
A new Bayesian optimization method using Poisson process for better noise robustness.
problem Estimating relative rankings of candidates in noisy environments.
method Poisson process-based ranking surrogate model and tailored acquisition functions.
result PoPBO framework shows lower computation costs and better robustness to noise compared to GP-BO.
Bayesian principles improve neural additive models for better feature selection and uncertainty.
problem Lack of calibrated uncertainties and feature selection in neural additive models.
method Augmenting NAMs with Bayesian principles to provide credible intervals, feature selection, and interaction ranking.
result Improved performance on tabular datasets and real-world medical tasks.
Bayesian framework improves LLM evaluation stability and transparency.
problem Pass@k and avg@N are unstable and misleading for LLMs.
method Bayesian evaluation with posterior estimates and credible intervals.
result Posterior-based evaluation yields stable and transparent rankings.
A new method for Bayesian inference tackles high-dimensional problems.
problem Bayesian inference in high-dimensional settings with kernel density estimation issues.
method Projected Wasserstein gradient descent (pWGD) method to overcome curse of dimensionality.
result pWGD method effectively addresses high-dimensional Bayesian inference problems.