This paper improves DNN generalization by accurately estimating mutual information.
problem Intractability of estimating mutual information in DNNs.
method Introduces a probabilistic representation of DNNs to accurately estimate mutual information.
result Derives a tighter generalization bound than previous relaxations.
CMDNet simplifies MAP detection for large systems with probabilistic relaxation.
problem High complexity of MAP detection in large systems.
method Probabilistic Continuous relaxation of discrete variables, iterative CMD algorithm, CMDNet with online optimization.
result CMDNet achieves a promising accuracy-complexity trade-off in MIMO systems.
We present sparse topical coding (STC), a non-probabilistic formulation of topic models for discovering latent representations of large collections of data. Unlike probabilistic topic models, STC relaxes the normalization constraint of admixture proportions and the constraint of defining a normalized likelihood functio…
Efficiently solves MRF inference problems with semidefinite programming.
problem Computing partition function or MAP estimate in binary and multi-class MRFs.
method Coordinate-descent-based fast semidefinite solver for SDPs.
result Substantially outperforms existing state-of-the-art methods in approximate inference.
We present a probabilistic viewpoint to multiple kernel learning unifying well-known regularised risk approaches and recent advances in approximate Bayesian inference relaxations. The framework proposes a general objective function suitable for regression, robust regression and classification that is lower bound of the…
Unified framework for differentiable graph partitioning with probabilistic cuts.
problem Lack of general guarantees and principled gradients in prior probabilistic relaxations of graph cuts.
method Unified probabilistic framework covering a wide class of cuts, including Normalized Cut, with tight analytic upper bounds.
result Rigorous, numerically stable foundation for scalable, differentiable graph partitioning.
This work interprets SFA through variational inference, relaxing linearity constraints.
problem Recover non-linear SFA from variational inference.
method Probabilistic interpretation of SFA through variational inference, relaxing linearity constraints.
result Reinterprets SFA as a variational framework, allowing slowness as a regularizer to reconstruction loss.
VaSST uses soft symbolic trees for probabilistic symbolic regression.
problem Efficiently recover symbolic expressions from noisy data.
method Variational inference with soft symbolic trees.
result Superior performance in structural recovery and predictive accuracy.
We give a novel formal theoretical framework for unsupervised learning with two distinctive characteristics. First, it does not assume any generative model and based on a worst-case performance metric. Second, it is comparative, namely performance is measured with respect to a given hypothesis class. This allows to avo…
In this paper, we present a local information theoretic approach to explicitly learn probabilistic clustering of a discrete random variable. Our formulation yields a convex maximization problem for which it is NP-hard to find the global optimum. In order to algorithmically solve this optimization problem, we propose tw…
Variable selection is a fundamental task in statistical data analysis. Sparsity-inducing regularization methods are a popular class of methods that simultaneously perform variable selection and model estimation. The central problem is a quadratic optimization problem with an l0-norm penalty. Exactly enforcing the l0-no…
Paper introduces MLRH, a probabilistic model for multilabel classification.
problem Multilabel classification challenges in various domains.
method Introduces hidden variables to relax one-hot-encoding in logistic regression.
result Probabilistic model achieves competitive performance compared to other algorithms.
The paper develops sum-of-squares relaxations for computing f-divergences.
problem Computing f-divergences from non-centered covariance matrices. method Sum-of-squares relaxations for convex optimization.
result Sum-of-squares relaxations make computations tractable.
This text explores strategies for learning discrete latent structures in neural networks.
problem Learning discrete latent structures in neural networks is challenging.
method Continuous relaxation, surrogate gradients, and probabilistic estimation.
result Many latent structure learning strategies use the same fundamental building blocks but apply them differently.
We consider a network scenario in which agents can evaluate each other according to a score graph that models some interactions. The goal is to design a distributed protocol, run by the agents, that allows them to learn their unknown state among a finite set of possible values. We propose a Bayesian framework in which …
Study probabilistic safety of BNNs under adversarial attacks.
problem Evaluate vulnerability of BNNs to adversarial attacks.
method Relaxation techniques from non-convex optimization to compute probabilistic safety bounds.
result Certify probabilistic safety of BNNs with millions of parameters.
Bayesian hierarchical clustering (BHC) is an agglomerative clustering method, where a probabilistic model is defined and its marginal likelihoods are evaluated to decide which clusters to merge. While BHC provides a few advantages over traditional distance-based agglomerative clustering algorithms, successive evaluatio…
The paper studies batch decompositions of random datasets with probabilistic similarity constraints.
problem Understanding how to optimally split large datasets into batches for better model learning.
method Assumes independent data points from a space, defines similarity, and uses probabilistic and martingale methods to find bounds on batch sizes.
result Demonstrates an inherent tradeoff between relaxing similarity constraints and batch size, and provides bounds for maximum similarity subsets.
Proposes a new Gaussian factor for probabilistic inference with degenerate settings.
problem Handling linear dependencies among random variables in Gaussian networks.
method Introduces a parametrised factor that relaxes the positive-definite constraint of the covariance matrix.
result Accurately accommodates degeneracies in probabilistic inference without significant computational overhead.
Matching correlated VAR time series databases by recovering matching permutations.
problem Matching perturbed and permuted correlated VAR time series.
method Probabilistic framework modeling, maximum likelihood estimator (MLE), linear assignment, convex relaxations.
result Recovery guarantees for perfect or partial recovery of matching permutations, thresholds for σ. In Bayesian classification, it is important to establish a probabilistic model for each class for likelihood estimation. Most of the previous methods modeled the probability distribution in the whole sample space. However, real-world problems are usually too complex to model in the whole sample space; some fundamental …
Proposes a recursive MPC scheme with probabilistic safety guarantees for uncertain dynamic systems.
problem Probabilistic safety guarantees for MPC in dynamic environments with unknown stochastic agents.
method Uses conformal prediction to derive high-confidence prediction regions and gradually relax safety constraints online.
result Ensures recursive feasibility of MPC schemes by relaxing safety constraints over time.
Using the linear Gaussian latent variable model as a starting point we relax some of the constraints it imposes by deriving a nonparametric latent feature Gaussian variable model. This model introduces additional discrete latent variables to the original structure. The Bayesian nonparametric nature of this new model al…
Discrete random variables are natural components of probabilistic clustering models. A number of VAE variants with discrete latent variables have been developed. Training such methods requires marginalizing over the discrete latent variables, causing training time complexity to be linear in the number clusters. By appl…
We provide a dynamic programming principle for stochastic optimal control problems with expectation constraints. A weak formulation, using test functions and a probabilistic relaxation of the constraint, avoids restrictions related to a measurable selection but still implies the Hamilton-Jacobi-Bellman equation in the …
Efficiently certifies global robustness of large neural networks with probabilistic guarantees.
problem Certifying robustness of large neural networks in a scalable and efficient manner.
method Sampling an ε-net and invoking a local robustness oracle.
result Certifies a probabilistic relaxation of robustness efficiently and globally.
Deep-PrAE improves rare-event simulation for black-box systems.
problem Evaluating rare safety-critical events in learning-based systems.
method Combines deep neural networks with IS to create statistically guaranteed estimations.
result Deep-PrAE provides accurate bounds on safety-critical event probabilities.
Geometric approach improves probabilistic robustness in neural networks.
problem Widespread lack of robustness in deep neural networks to adversarial examples.
method Geometric view on Probabilistically Robust Learning (PRL) and introduction of new perimeters.
result Existence of solutions and properties of modified PRL models.
Paper introduces PRMs to learn non-Markovian stochastic rewards for reinforcement learning.
problem Lack of structured representation for non-Markovian stochastic rewards in reinforcement learning.
method Introduces probabilistic reward machines (PRMs) and presents an algorithm to learn them from decision processes.
result Algorithm proves correct and convergent for learning PRMs from decision processes.
This work improves understanding of dimension reduction algorithms and their probabilistic embeddings.
problem Improving theoretical understanding of non-linear dimension reduction algorithms.
method Analytical investigation of a generalized multidimensional scaling optimization problem.
result Probabilistic formulation of the problem leads to deterministic embeddings, contrary to standard implementations.
The probabilistic bisection algorithm (PBA) solves a class of stochastic root-finding problems in one dimension by successively updating a prior belief on the location of the root based on noisy responses to queries at chosen points. The responses indicate the direction of the root from the queried point, and are incor…
Improved hierarchical discrete VAEs for better stability and performance.
problem Training stable and efficient hierarchical discrete VAEs with numerous latent variables.
method Introducing Relaxed-Responsibility Vector-Quantisation to parameterise discrete latent variables in a hierarchical structure.
result Achieved state-of-the-art bits-per-dim results for various standard datasets.
Enhances traffic forecasting with dynamic regression incorporating error modeling.
problem Improving accuracy of traffic forecasts using deep spatiotemporal models.
method Integrates matrix-variate autoregressive (AR) model into loss function for error series of base model.
result Improved traffic forecasting performance on SOTA models with interpretable AR coefficients.
Generative AI improves surrogate models by blending LF and HF data.
problem Data scarcity between high-fidelity and low-fidelity simulations.
method Probabilistic multi-fidelity surrogate framework using generative transfer learning.
result The model achieves HF accuracy with fewer HF evaluations.
New method improves robust point matching under probabilistic settings.
problem Insufficient theoretical understanding of existing point matching methods.
method Distance profiles and modified matching procedure.
result Improved robustness under probabilistic settings.
Paper proposes DAG-DB for learning discrete DAGs via backpropagation.
problem Learning Directed Acyclic Graphs (DAGs) from data.
method DAG-DB uses Discrete Backpropagation with I-MLE and Straight-Through Estimation.
result DAG-DB learns DAGs effectively using probabilistic sampling and backpropagation.
Normalizing flows are shown to be equivalent to Bayesian networks, revealing new insights.
problem Understanding the limitations and capabilities of normalizing flows.
method Revisiting normalizing flows as probabilistic graphical models and analyzing their structure.
result Normalizing flows can be reduced to Bayesian networks, revealing new insights into their structure and capabilities.
A grand challenge in machine learning is the development of computational algorithms that match or outperform humans in perceptual inference tasks that are complicated by nuisance variation. For instance, visual object recognition involves the unknown object position, orientation, and scale in object recognition while …
Paper studies Adam's convergence under relaxed assumptions, proving a rate of O(poly(log T)/sqrt(T)).
problem Understanding Adam's convergence in non-convex, stochastic optimization with unbounded gradients and noise.
method Introduced a comprehensive noise model and used it to prove Adam's convergence rate.
result Adam finds a stationary point with a rate of O(poly(log T)/sqrt(T)) in high probability.
Method improves microbial biomass yield estimation from noisy data.
problem Estimating microbial biomass yields from noisy cell counts and substrate measurements.
method Probabilistic macrochemical modeling to relax cell weight assumptions and improve robustness.
result Model provides accurate uncertainty estimates of key parameters.
Unified framework for information-theoretic bounds on learning algorithms.
problem Deriving generalization bounds for learning algorithms.
method Probabilistic decorrelation lemma, symmetrization, couplings, chaining, Young's inequality.
result New upper bounds on generalization error in expectation and high probability.
A user-friendly interface constructs effective background knowledge from ER diagrams.
problem Inefficient construction of background knowledge by domain experts in ILP systems.
method Design of a graphical user interface to interact with Entity Relationship diagrams to construct modes for a probabilistic logic learning system.
result Domain experts can construct effective background knowledge on par with experts using the graphical interface.
Proposes DILATE and STRIPE++ for precise time series forecasting.
problem Non-stationary signals with sudden changes.
method Incorporates shape and temporal criteria in deep learning models.
result Improves precision in deterministic and probabilistic forecasting.
Improves convergence speed in compressive sensing with a new probabilistic approach.
problem Efficiently solving the best subset selection problem in compressive sensing.
method Smooth probabilistic reformulation of ℓ0 regularized regression. result Empirically outperforms existing compressive sensing algorithms across various settings.
In the analysis of machine learning models, it is often convenient to assume that the parameters are IID. This assumption is not satisfied when the parameters are updated through training processes such as SGD. A relaxation of the IID condition is a probabilistic symmetry known as exchangeability. We show the sense in …
PRAE identifies outliers and reconstructs inliers in autoencoders.
problem Accurately identifying anomalies in data.
method Probabilistic Robust AutoEncoder (PRAE) approach.
result PRAE effectively removes outliers and reconstructs inliers.
Sparse versions of principal component analysis (PCA) have imposed themselves as simple, yet powerful ways of selecting relevant features of high-dimensional data in an unsupervised manner. However, when several sparse principal components are computed, the interpretation of the selected variables is difficult since ea…
This review synthesizes uncertainty modeling in probabilistic image segmentation.
problem Relaxed Bayesian assumptions lead to missing uncertainty information in deep models.
method Standardizes theory, notation, and terminology for feature- and parameter-distribution modeling.
result Establishes a common framework for robust decision-making in segmentation tasks.