Transformers interpret as probabilistic mixtures, offering new insights.
problem Understanding Transformers from a probabilistic perspective.
method Modeling Transformers as mixtures of Gaussian models.
result Transformers can be seen as maximum posterior probability estimators.
Proposes a probabilistic method for generating semantically-aware adversarial examples.
problem Generating adversarial examples that are difficult for humans to detect while preserving semantics.
method Embeds subjective understanding of semantics as a distribution into adversarial example generation.
result Achieves higher success rates in circumventing adversarial defense mechanisms.
The paper investigates model collapse in language models from a probabilistic perspective.
problem Understanding and preventing model collapse in language model training.
method Investigates recursive parametric model training from a probabilistic standpoint, characterizing conditions for model collapse and proposing mitigation strategies.
result Progressively increasing sample size is necessary to prevent model collapse, with a superlinear growth rate required in the asymptotic regime.
We present a new algorithm for approximate inference in probabilistic programs, based on a stochastic gradient for variational programs. This method is efficient without restrictions on the probabilistic program; it is particularly practical for distributions which are not analytically tractable, including highly struc…
Surveying probabilistic real algebraic geometry.
problem Classical problems in real algebraic geometry.
method Probabilistic perspective on classical topics.
result Modern approach to Hilbert's Sixteenth Problem.
We present a general probabilistic perspective on Gaussian filtering and smoothing. This allows us to show that common approaches to Gaussian filtering/smoothing can be distinguished solely by their methods of computing/approximating the means and covariances of joint probabilities. This implies that novel filters and …
This work connects LLE, factor analysis, and probabilistic PCA through a stochastic perspective.
problem Exploring the theoretical connection between LLE, factor analysis, and probabilistic PCA.
method Solving the stochastic linear reconstruction of LLE using expectation maximization.
result LLE, factor analysis, and probabilistic PCA are shown to be connected through a stochastic perspective.
CAVs reveal latent concept distributions, but are vulnerable to adversarial attacks.
problem Understanding latent concept encodings in AI models.
method Probabilistic perspective on CAVs, deriving mean and covariance.
result CAVs can be adversarially manipulated, highlighting a vulnerability.
Paper proposes Monarch matrices for scalable probabilistic circuits.
problem Improving scalability of probabilistic circuits.
method Sparse Monarch matrices for sum blocks in PCs.
result Significantly reduces memory and computation costs, enabling unprecedented scaling.
BNs extract sparse, non-redundant features from climate data networks.
problem Redundant information in correlation networks limits physical feature extraction.
method Construct data-driven complex networks using Bayesian Networks.
result Sparse, non-redundant features reveal generalizable physical features.
New method encodes function preferences into neural nets for better generalization.
problem Challenges in encoding explicit function preferences in neural network training.
method Function-space empirical Bayes (FSEB) regularization.
result FSEB leads to near-perfect semantic shift detection and improved generalization.
Normalizing flows simplify complex distributions through bijective transformations.
problem Defining expressive probability distributions efficiently.
method Bijective transformations on a base distribution.
result Unified perspective on normalizing flows for modeling and inference.
Probabilistic programming languages represent complex data with intermingled models in a few lines of code. Efficient inference algorithms in probabilistic programming languages make possible to build unified frameworks to compute interesting probabilities of various large, real-world problems. When the structure of mo…
Bayesian deep learning improves neural network accuracy and generalization.
problem Improving accuracy and calibration of deep neural networks.
method Bayesian marginalization and deep ensembles to approximate marginalization, and tempering for calibrating predictive distributions.
result Bayesian approaches improve deep neural networks' accuracy and generalization.
Unified framework for learning flexible probabilistic programs using DPP and PAC-Bayes bounds.
problem Learning and generalizing from complex probabilistic models.
method Unified DPP representation and PAC-Bayes bounds for stochastic programs.
result Improved performance and generalization prediction using flexible DPP model representations and learned complexity measures.
We prove that the Euler form of a metric connection on real oriented vector bundle E over a compact oriented manifold M can be identified, as a current, with the expectation of the random current defined by the zero-locus of a certain random section of the bundle. We also explain how to reconstruct probabilisticall…
Unified framework for Gaussian process methods in differential equations.
problem Fragmented approaches to Gaussian process methods in differential equations.
method Unified Bayesian perspective integrating differential equation constraints.
result Consolidation of existing methods and foundation for future research.
This work offers a broad perspective on probabilistic modeling and inference in light of recent advances in probabilistic programming, in which models are formally expressed in Turing-complete programming languages. We consider a typical workflow and how probabilistic programming languages can help to automate this wor…
Adversarial attacks on probabilistic state-space models affect latent state and policy decisions.
problem Robust reinforcement learning under adversarial observability.
method Analyzing adversarial attacks on linear probabilistic state-space models.
result Demonstrating the influence of adversarial observations on latent state and policy decisions.
We introduce a toy probabilistic model to analyze job-matching processes in recent Japanese labor markets for university graduates by means of statistical physics. We show that the aggregation probability of each company is rewritten by means of non-linear map under several conditions. Mathematical treatment of the map…
A new probabilistic mixup framework improves deep learning generalization.
problem Improving deep learning generalization performance.
method Introduces a novel probabilistic mixup framework for conditional density estimation.
result Empirical results show benefits over existing mixup variants.
Simplified explanation of DDPMs for machine learning.
problem Complexity in explaining DDPMs and missing rationale.
method Six simple steps to explain DDPMs.
result Clear rationale for each step of DDPM formulation.
Develops probabilistic safety regions for scalable classifiers.
problem Minimizing misclassification errors in supervised classification.
method Introduces probabilistic safety regions and scalable classifiers.
result Probabilistic certifications for classifier performance.
New algorithm improves graph inference tasks.
problem Complex graph reasoning and prediction tasks.
method Policy Message Passing algorithm reformulates graph inference as stochastic sequential processes.
result Consistently outperforms state-of-the-art models.
Study on stability of GCNNs under graph perturbations.
problem Limited theoretical understanding of GCNN stability.
method Proposes a probabilistic framework to analyze GCNN stability under various graph perturbations.
result Demonstrates the importance of data distribution in stability analysis.
Paper introduces probabilistic approach to CO layers in ML.
problem Challenges in training CO layers with stochastic gradient descent.
method Probabilistic perspective, approximate differentiation, structured losses.
result InferOpt.jl package for training CO layers.
Bayesian symbolic regression automates model discovery from data.
problem Learning closed-form mathematical models from data using heuristic methods.
method Probabilistic approach to symbolic regression, connecting to information theory and statistical physics.
result Probabilistic approach provides model plausibility and performance guarantees.
Capsule models enforce object pose relationships for robustness, explored with probabilistic generative and variational methods.
problem Enforcing object pose relationships for robustness to viewpoint changes.
method Probabilistic generative model with variational bound, exploring capsule assumptions and inference mechanisms.
result Unified objective and test time optimisation demonstrated for capsule models.
ProGen improves spatiotemporal forecasting with SDEs and diffusion models.
problem Complex spatial and temporal dependencies in spatiotemporal data.
method ProGen uses Stochastic Differential Equations and diffusion-based generative models.
result ProGen outperforms state-of-the-art models on traffic datasets.
The paper studies batch decompositions of random datasets with probabilistic similarity constraints.
problem Understanding how to optimally split large datasets into batches for better model learning.
method Assumes independent data points from a space, defines similarity, and uses probabilistic and martingale methods to find bounds on batch sizes.
result Demonstrates an inherent tradeoff between relaxing similarity constraints and batch size, and provides bounds for maximum similarity subsets.
This paper investigates how synthetic data and overparameterization improve VAE generalization.
problem Improving generalization performance of Variational Autoencoders (VAEs).
method Investigates the effectiveness of synthetic data and overparameterization in VAEs.
result Training on synthetic data and using more parameters improves VAE generalization, inference, and robustness.
The paper connects disentanglement to manifold charts and commutativity.
problem Discovering local charts of the data manifold for disentanglement.
method Interpreting disentanglement as local charts of the data manifold and studying commutativity.
result Commutativity is a central property in disentanglement, as shown in manifold, group theoretic, and probabilistic frameworks.
In this work, we introduce a novel probabilistic representation of deep learning, which provides an explicit explanation for the Deep Neural Networks (DNNs) in three aspects: (i) neurons define the energy of a Gibbs distribution; (ii) the hidden layers of DNNs formulate Gibbs distributions; and (iii) the whole architec…
Unified probabilistic foundation for fuzzy simplicial sets in dimensionality reduction.
problem Lack of clear probabilistic interpretation in fuzzy simplicial sets.
method Introducing a probabilistic framework explaining fuzzy simplicial sets as marginals of probability measures on simplicial sets.
result Unified probabilistic theoretical foundation for fuzzy simplicial sets.
In this paper we explore the "vector semantics" problem from the perspective of "almost orthogonal" property of high-dimensional random vectors. We show that this intriguing property can be used to "memorize" random vectors by simply adding them, and we provide an efficient probabilistic solution to the set membership …
The framework of reinforcement learning or optimal control provides a mathematical formalization of intelligent decision making that is powerful and broadly applicable. While the general form of the reinforcement learning problem enables effective reasoning about uncertainty, the connection between reinforcement learni…
This paper introduces a neural operator for probabilistic conditioning.
problem Probabilistic conditioning of random variables X given Y. method Develops a single operator that maps any joint density to its conditional, approximated by neural operators.
result Neural operators can approximate the conditioning operator to arbitrary accuracy.
The performance of many machine learning techniques depends on the choice of an appropriate similarity or distance measure on the input space. Similarity learning (or metric learning) aims at building such a measure from training data so that observations with the same (resp. different) label are as close (resp. far) a…
ICON learns differential equation operators from examples, revealing probabilistic inference.
problem Learning operators for differential equations from limited examples.
method Probabilistic operator learning using ICON architectures trained on diverse datasets.
result ICON implicitly performs Bayesian inference on solution operators.
Quantum kernel improves probabilistic time series forecasting.
problem Quantifying uncertainty in probabilistic time series predictions.
method Integrates quantum kernel with Gaussian process regression.
result Quantum kernel enhances forecasting performance.
Probabilistic atlases provide essential spatial contextual information for image interpretation, Bayesian modeling, and algorithmic processing. Such atlases are typically constructed by grouping subjects with similar demographic information. Importantly, use of the same scanner minimizes inter-group variability. Howeve…
Transfer learning assumes classifiers of similar tasks share certain parameter structures. Unfortunately, modern classifiers uses sophisticated feature representations with huge parameter spaces which lead to costly transfer. Under the impression that changes from one classifier to another should be ``simple'', an effi…
We consider probabilistic PCA and related factor models from a Bayesian perspective. These models are in general not identifiable as the likelihood has a rotational symmetry. This gives rise to complicated posterior distributions with continuous subspaces of equal density and thus hinders efficiency of inference as wel…
A new perspective on self-attention models using MLPs.
problem Improving sequence modeling with self-attention mechanisms.
method Introducing HyperMLP and HyperGLU, which use dynamic two-layer MLPs with reverse-offset layout.
result HyperMLP/HyperGLU consistently outperform softmax-attention baselines.
Treating neural network inputs and outputs as random variables, we characterize the structure of neural networks that can be used to model data that are invariant or equivariant under the action of a compact group. Much recent research has been devoted to encoding invariance under symmetry transformations into neural n…
Recent advances in analysis of subband amplitude envelopes of natural sounds have resulted in convincing synthesis, showing subband amplitudes to be a crucial component of perception. Probabilistic latent variable analysis is particularly revealing, but existing approaches don't incorporate prior knowledge about the ph…
Study proposes a multi-agent framework to mitigate bias in sentiment analysis.
problem Bias in sentiment analysis models.
method Integrates multiple LLMs, incorporates dialogue sessions, and uses probabilistic prediction.
result KCS+IBC reduces entropy and increases variance, suggesting improved balance between aggregation and diversity.
Generative flow networks use RL to learn probabilistic models efficiently.
problem Training generative models with RL for compositional discrete objects.
method Reformulate GFlowNet training as entropy-regularized RL with specific reward and regularizer.
result Entropy-regularized RL can be competitive with established GFlowNet training methods.