Extends denoising and score estimation to energy models via Tweedie's formula.
problem Linking denoising and score estimation for a wider range of distributions.
method Derives a fundamental identity connecting energy score derivatives and scores.
result Establishes a new identity for energy scores analogous to Tweedie's formula.
Improves score estimation for noised targets using known clean scores.
problem Poor score estimation at low noise levels in Denoising Score Matching.
method Introduces Target Score Identity and Target Score Matching loss.
result Score estimates are more accurate at low noise levels.
Efficiently distills pretrained text-to-image models without real data, improving FID and CLIP scores.
problem Slow iterative refinement process of diffusion-based text-to-image models.
method Guided Score identity Distillation with Long and Short Classifier-Free Guidance.
result Achieves state-of-the-art FID performance with competitive CLIP score.
It is often stated in papers tackling the task of inferring Bayesian network structures from data that there are these two distinct approaches: (i) Apply conditional independence tests when testing for the presence or otherwise of edges; (ii) Search the model space using a scoring metric. Here I argue that for complete…
Improved sampling efficiency for inverse problems using variance-reduced diffusion methods.
problem Efficiently estimating noisy scores in inverse problems.
method Developed a nonparametric self-normalized importance sampling estimator and a state-dependent blending rule.
result Improved sample quality for fixed simulation budgets in synthetic targets and PDE-governed inverse problems.
Identity-link IRT improves TVD-MI scores without curvature violations.
problem Preserving additivity in TVD-MI scores for efficient LLM evaluation.
method Derives clipped-linear model from Gini entropy maximization, using identity link.
result Identity-link yields lower curvature violations (median curl 0.080-0.150) compared to probit/logit.
SiD distills pretrained diffusion models into a fast one-step generator.
problem Efficiently distilling pretrained diffusion models into a fast generator.
method Reformulates forward diffusion processes as semi-implicit distributions and uses three score-related identities to create a loss mechanism.
result Achieves high FID performance and significantly reduces generation time.
A new filter reduces density fitting to a linear solve, improving performance on nonlinear systems.
problem Nonlinear Bayesian filtering challenges in representing belief distributions.
method Combines score matching with Stein's identity to avoid partition function evaluation.
result The Score Kalman Filter (SKF) outperforms existing methods on nonlinear systems.
User identity linkage is a task of recognizing the identities of the same user across different social networks (SN). Previous works tackle this problem via estimating the pairwise similarity between identities from different SN, predicting the label of identity pairs or selecting the most relevant identity pair based …
We consider estimating the parametric components of semi-parametric multiple index models in a high-dimensional and non-Gaussian setting. Such models form a rich class of non-linear models with applications to signal processing, machine learning and statistics. Our estimators leverage the score function based first and…
The paper decomposes probabilistic scores into reliability, uncertainty, and information loss.
problem Understanding the reliability and uncertainty of probabilistic predictions.
method Developed decomposition identities for proper losses, quantifying reliability, residual uncertainty, and information gain.
result A three-term identity for classification scores, revealing miscalibration, grouping term, and feature-level uncertainty.
Estimates score function from data with optimal rate in high dimensions.
problem Estimating the score function of an unknown probability distribution from data.
method Empirical Bayes smoothing with a Gaussian kernel.
result Optimal rate of estimation ildeΘ(n−d+42) for d dimensions. Improves interpretability of anomaly scores in GBRBM-based detection.
problem Difficulty in setting a proper threshold for anomaly scores.
method Proposes a measure based on cumulative distribution and uses simulated annealing for evaluation.
result Established a guideline for setting the threshold using the interpretable measure.
Few-step distillation improves T2I models without real images or CFG trade-offs.
problem Challenges in accelerating T2I models with high-resolution and CFG.
method Score identity distillation (SiD) for few-step generation, with adversarial loss and new guidance strategies.
result State-of-the-art performance on SDXL at 1024x1024 resolution, robust to real images absence.
New method improves SBI efficiency and scalability.
problem Scalability issues in SBI methods for large datasets.
method Langevin dynamics with score matching, exploiting likelihood structure.
result Structured score network enhances statistical efficiency and scalability.
As data sets grow in size, the ability of learning methods to find structure in them is increasingly hampered by the time needed to search the large spaces of possibilities and generate a score for each that takes all of the observed data into account. For instance, Bayesian networks, the model chosen in this paper, ha…
A method makes particle filters differentiable without altering their forward pass.
problem Compatibility issues between particle filters and automatic differentiation.
method Introduces a correction to particle weights using the stop-gradient operator.
result Automatic differentiation produces good estimators for gradients and second-order derivatives.
Trimming helps in conformal prediction when it separates anomaly scores.
problem Effectiveness of trimming in conformal prediction under contamination.
method Analyse fixed-threshold trimming as a replacement of the contaminated calibration law with a retained law.
result Trimming helps when it separates anomaly scores, reducing clean-target coverage to a one-dimensional score-CDF transfer problem.
Exact posterior score estimation for solving linear inverse problems
problem Solving linear inverse problems
method Derive the exact posterior score and use it as a denoising training objective
result EPS outperforms training-free and training-based baselines on various metrics
New algorithm speeds up causal discovery for network data.
problem Scalability issues in score-matching for temporal network data.
method Developed a new parent-finding subroutine for DAGs, improving score matching efficiency.
result Efficiency-lifted score matching for both i.i.d. and temporal data on networks.
New geometric analysis shows L2 score error is flawed for diffusion models.
problem Score matching errors in diffusion models do not fully capture distributional quality.
method Decomposed score errors into gradient and solenoidal components, focusing on gradient's role in Fokker-Planck dynamics.
result Only gradient component affects marginal distributional quality; solenoidal component is structurally invisible.
ST-BCP narrows the coverage gap in BCP by transforming nonconformity scores.
problem The looseness in BCP's coverage guarantee due to Markov's inequality.
method Introduces a data-dependent transformation of nonconformity scores.
result Reduces the average coverage gap from 4.20% to 1.12% on benchmarks.
Score matching errors are not sufficient for measuring diffusion model quality.
problem The L2 score matching error is not a reliable measure of diffusion model performance. method Decomposed score errors into gradient and solenoidal components and analyzed their geometric properties.
result Only the gradient component of the score error affects the marginal distributional quality.
Unified score and distance-based GoF tests for model adequacy.
problem Difficulty in extending score-based GoF tests to nonparametric alternatives.
method Introducing semiparametric kernelized Stein discrepancy (SKSD) test.
result SKSD test is computationally efficient and universally consistent.
The article compares neural networks and logistic regression for credit scoring and introduces a new probability calibration technique.
problem Improving credit scoring accuracy using machine learning techniques.
method Comparison of logistic regression and neural networks, feature importance assessment, temporal feature inclusion, and SURE probability calibration.
result Neural networks can slightly improve credit scoring performance, and SURE calibration technique enhances probability calibration.
A new method improves density ratio estimation with fewer function evaluations.
problem Stable and accurate estimation of density ratios with high variance issues.
method Diffusion Secant Alignment for Score-Based Density Ratio Estimation (ISA-DRE)
result ISA-DRE achieves comparable or superior results with fewer function evaluations.
HED Score improves temporal evaluation of detection accuracy.
problem Temporal agnosticism in existing evaluation frameworks for non-stationary processes.
method Measure-theoretic HED Score integrating exponentially decaying kernel over posterior probability stream.
result HED Score achieves 388.8% improvement over ROC/AUC on NSL-KDD benchmark.
New method reduces variance in Bayesian inverse problems.
problem High variance in Monte Carlo estimates for inverse problems.
method Conditional neural control variates based on Stein's identity.
result Substantial variance reduction across different inverse problems.
Derives a general derivative identity for conditional mean in Gaussian noise.
problem Understanding conditional mean in Gaussian noise channels.
method Derives a general derivative identity for the conditional mean of X given Y=y in a Markov chain U↔X↔Y. result Provides a unifying view of conditional mean identities and derives new ones.
A new method uses a product of experts with Dirichlet variables to approximate complex distributions.
problem Approximating complex distributions with tractable models.
method A product of experts with auxiliary Dirichlet variables, using a Feynman identity to sample and optimize.
result The method efficiently approximates complex distributions using a product of experts and Dirichlet variables.
DSM on manifolds removes singularities and computes small-noise expansions.
problem DSM on manifolds with singular noise.
method Rao-Blackwellized score matching, nearest-point projection, intrinsic Riemannian score.
result Canonical target equals intrinsic Riemannian score up to a small correction.
The paper examines how slightly biasing towards under-represented groups in sequential selection processes can lead to long-term fairness.
problem Designing fair sequential decision-making processes for long-term social fairness.
method Proposes Multi-agent Fair-Greedy policy to balance score maximization and fairness.
result Proves convergence to long-term fairness target set by agents when score distributions are identical.
The study reveals flaws in pruning criteria and proposes a new assumption for better filter selection.
problem Flaws in existing pruning criteria for CNNs.
method Empirical experiments and Convolutional Weight Distribution Assumption.
result The Convolutional Weight Distribution Assumption improves filter selection in pruning.
Turtle Score analyzes developer similarity to match high-performing candidates.
problem Finding suitable candidates for IT companies based on cultural fit.
method Examines employee performance data, applies machine learning for similarity analysis.
result Develops a model for recruiters to identify high-performing candidates.
Unified framework for training diffusion and flow models to sample from target distributions.
problem Training diffusion and flow models to sample from target distributions defined by exponential tilting.
method Unified framework combining stochastic optimal control and non-equilibrium thermodynamics perspectives.
result Unified bias-variance decompositions and theoretical support for adjoint-based methods.
Proposes MSS to identify causal structure from heterogeneous environments.
problem Distribution shifts between environments violate i.i.d. data assumption.
method Sparse mechanism shift hypothesis, score-based approach.
result Identifies entire causal structure with high probability.
A new diffusion method approximates Schrödinger bridge with improved convergence.
problem Approximating Schrödinger bridge with Langevin diffusion.
method Leveraging Langevin diffusion to approximate Schrödinger bridge.
result The difference between the two approximations is proportional to the score function.
Outlier detection amounts to finding data points that differ significantly from the norm. Classic outlier detection methods are largely designed for single data type such as continuous or discrete. However, real world data is increasingly heterogeneous, where a data point can have both discrete and continuous attribute…
A new neural network initialization method is proposed for faster and more accurate training.
problem Efficient initialization for training multi-layer feedforward neural networks.
method Initialization based on Stein's identity, using eigenvectors of cross-moment matrix.
result The SteinGLM method is faster and more accurate than other initialization methods.
Fairness-aware learning involves designing algorithms that do not discriminate with respect to some sensitive feature (e.g., race or gender). Existing work on the problem operates under the assumption that the sensitive feature available in one's training sample is perfectly reliable. This assumption may be violated in…
The question of aggregating pair-wise comparisons to obtain a global ranking over a collection of objects has been of interest for a very long time: be it ranking of online gamers (e.g. MSR's TrueSkill system) and chess players, aggregating social opinions, or deciding which product to sell based on transactions. In mo…
In a Bayesian learning setting, the posterior distribution of a predictive model arises from a trade-off between its prior distribution and the conditional likelihood of observed data. Such distribution functions usually rely on additional hyperparameters which need to be tuned in order to achieve optimum predictive pe…
MIAEAD detects anomalies in mixed data types.
problem Challenges of heterogeneity in feature subsets for anomaly detection.
method Multiple-Input Variational Auto-Encoder (MIVAE) for simultaneous feature subset anomaly scoring.
result MIVAE outperforms conventional methods and state-of-the-art unsupervised models in AUC score.
Unintended bias in Machine Learning can manifest as systemic differences in performance for different demographic groups, potentially compounding existing challenges to fairness in society at large. In this paper, we introduce a suite of threshold-agnostic metrics that provide a nuanced view of this unintended bias, by…
Spatially-aware metrics improve uncertainty evaluation in segmentation.
problem Uncertainty evaluation metrics treat voxels independently, ignoring spatial context.
method Proposed three spatially aware metrics incorporating structural and boundary information.
result Improved alignment with clinically important factors and better discrimination between uncertainty patterns.
Few-step protein backbone generators reduce sampling time by over 20x.
problem Computational bottleneck in diffusion-based protein generation models.
method Score distillation adapted for protein backbone generation, combined with inference time noise modulation.
result Significant reduction in sampling time (20+ fold) while maintaining comparable performance.
The difficulty of multi-class classification generally increases with the number of classes. Using data from a subset of the classes, can we predict how well a classifier will scale with an increased number of classes? Under the assumptions that the classes are sampled identically and independently from a population, a…
Developing state-of-the-art approaches for specific tasks is a major driving force in our research community. Depending on the prestige of the task, publishing it can come along with a lot of visibility. The question arises how reliable are our evaluation methodologies to compare approaches? One common methodology to i…