Estimates score function from data with optimal rate in high dimensions.
problem Estimating the score function of an unknown probability distribution from data.
method Empirical Bayes smoothing with a Gaussian kernel.
result Optimal rate of estimation ildeΘ(n−d+42) for d dimensions. Probabilistic forecasts in the form of probability distributions over future events have become popular in several fields of statistical science. The dissimilarity between a probability forecast and an outcome is measured by a loss function (scoring rule). Popular example of scoring rule for continuous outcomes is the …
New scoring rules for multivariate distributions and level sets.
problem Evaluating forecast accuracy for multivariate distributions and level sets.
method Theoretical framework for scoring rules, decomposition of multivariate scoring functions, numerical algorithm for computation.
result New scoring functions for multivariate distributions and level sets, including density and cumulative distribution level sets.
We formulate a supervised learning problem, referred to as continuous ranking, where a continuous real-valued label Y is assigned to an observable r.v. X taking its values in a feature space X and the goal is to order all possible observations x in X by means of a scoring function $s:\mathcal{X}…
DLM-One speeds up language generation by 500x with continuous models.
problem Efficiently generating text sequences in natural language processing.
method Score-distillation of continuous diffusion language models.
result Achieves up to 500x speedup in inference time with competitive performance.
FDPs generalize diffusion models to function spaces, enabling efficient image generation.
problem Efficiently generating images from continuous data.
method Introducing FDPs with new mathematical frameworks and training objectives.
result FDPs achieve high-quality image generation with fewer parameters.
Bayesian approach models match and non-match score distributions over continuous covariates.
problem Complex evaluation of model performance over continuous covariates in biometric verification.
method Generative model of score distributions, mixture models, local basis functions, Bayesian inference.
result Accurate and effective method for studying model performance over continuous covariates.
Proposes MVG-CRPS for robust multivariate forecasting.
problem Outliers in multivariate forecasting lead to significant errors.
method Integrates CRPS for MVG distributions, optimizing with MVG-CRPS.
result Improves robustness, accuracy, and uncertainty quantification.
Study finds simple model-agreement scores perform well in various error estimation scenarios.
problem Evaluating model performance on unseen distributions using disparate scoring functions.
method Rigorously studied popular scoring functions (confidence, local manifold smoothness, model agreement) independently of mechanism choice.
result Simple model-agreement scores outperform confidence- and smoothness-based scores in realistic settings with compromised training data.
Paper establishes a density formula for diffusion models, linking target density to score function.
problem Lack of theoretical foundation for optimizing DDPMs using ELBO.
method Developed a density formula for continuous-time diffusion processes, revealing the connection between target density and score function.
result The minimizer of the ELBO objective for DDPMs nearly coincides with the true objective, providing a theoretical foundation.
Standardized fairness measures for continuous risk scores using Wasserstein distance.
problem Quantifying and interpreting group disparities in continuous risk scores.
method Proposes standardized fairness measures based on Wasserstein distance for continuous scores.
result Proposed measures outperform ROC-based fairness measures by being more explicit and quantifying significant biases.
Survival MDN uses invertible functions to speed up survival analysis models.
problem Training neural ODEs for survival analysis is computationally expensive.
method Survival MDN applies an invertible positive function to MDN outputs.
result Survival MDN outperforms or matches other models on concordance, Brier score, and log-likelihood.
SciRE-Solver accelerates DMs sampling by recursively calculating the score function derivative.
problem Slow iterative process of diffusion models due to estimating the score function derivative.
method Recursive Difference (RD) method combined with truncated Taylor expansion of score-integrand.
result SciRE-Solver achieves state-of-the-art FIDs with significantly fewer score function evaluations.
The output scores of a neural network classifier are converted to probabilities via normalizing over the scores of all competing categories. Computing this partition function, Z, is then linear in the number of categories, which is problematic as real-world problem sets continue to grow in categorical types, such as …
Study compares different scoring rules for machine-learned weather forecasts, finding scale-awareness improves forecast realism.
problem Improving the accuracy of machine-learned probabilistic weather forecasts.
method Comparison of scoring rules (CRPS, fair global energy score, graph energy score) and analysis of their impact on forecast field spectra.
result Scale-awareness improves forecast realism, particularly in the tropics.
A new RL framework for risk-sensitive decision-making using convex scoring functions.
problem Time-inconsistent risk measures in reinforcement learning.
method Convex scoring functions, augmented state space, auxiliary variable, customized Actor-Critic algorithm.
result Theoretical guarantees for approximation and convergence under certain conditions.
Training-free model learns SDE dynamics without training, accelerating parameter studies.
problem High computational cost of simulating parameter-dependent SDEs.
method Training-free conditional diffusion model with joint kernel-weighted Monte Carlo estimator.
result Accurate approximation of conditional distributions across varying parameter values.
Paper presents a new policy gradient theorem using weak derivatives for reinforcement learning.
problem Continuous state-action reinforcement learning problems.
method Introduced an alternative policy gradient theorem using weak derivatives.
result The new approach yields algorithms that converge almost surely to stationary points of the value function.
Paper discusses extending Gini score for tied rankings and case weights.
problem Extending Gini score for tied rankings and case weights.
method Discuss and adapt Gini score for ties and case weights.
result Gini score can be used for tied rankings and case weights.
bnlearn is an R package which includes several algorithms for learning the structure of Bayesian networks with either discrete or continuous variables. Both constraint-based and score-based algorithms are implemented, and can use the functionality provided by the snow package to improve their performance via parallel c…
We extend diffusion models to function spaces and introduce a new method for sampling from posterior distributions.
problem Sampling from posterior distributions in infinite-dimensional function spaces using diffusion models.
method Infinite-dimensional extension of Doob's h-transform, Supervised Guidance Training for efficient sampling. result We prove that diffusion models can be conditioned to sample from posterior distributions and introduce a simulation-free score matching objective.
This paper analyzes discrete diffusion models, deriving convergence bounds for their generated samples.
problem Theoretical guarantees for discrete-state diffusion models remain under-explored.
method Continuous Time Markov Chain (CTMC) framework and discrete-time sampling algorithm.
result Convergence bounds for KL divergence and TV distance are derived, showing linear dependence on dimension.
The consultative papers for the Basel II Accord require rating systems to provide a ranking of obligors in the sense that the rating categories indicate the creditworthiness in terms of default probabilities. As a consequence, the default probabilities ought to present a monotonous function of the ordered rating catego…
This paper proposes a continuous timing strategy for growth vs. defensive style allocation.
problem Dynamic allocation of growth and defensive ETF baskets using macro-market timing signals.
method Continuous smooth score combining multiple factors, mapped to G/D weights, smoothed with EWMA.
result Continuous style timing strategy outperforms static benchmarks in risk-adjusted returns.
Measuring dependence between two random variables is very important, and critical in many applied areas such as variable selection, brain network analysis. However, we do not know what kind of functional relationship is between two covariates, which requires the dependence measure to be equitable. That is, it gives sim…
Proposes CDTD, a diffusion model for mixed-type tabular data.
problem Adapting diffusion models to mixed-type tabular data.
method Score matching and score interpolation for continuous features, adaptive noise schedules for categorical features.
result Consistently outperforms state-of-the-art models in mixed-type tabular data.
Maximum likelihood training improves the performance of score-based diffusion models.
problem Training score-based diffusion models with maximum likelihood.
method Trained by minimizing a weighted combination of score matching losses, with a specific weighting scheme that bounds negative log-likelihood.
result Maximum likelihood training improves the log-likelihood of score-based diffusion models across multiple datasets.
New bounds for score matching in polynomial exponential families.
problem Understanding the sample complexity of score matching for polynomial exponential families.
method Non-asymptotic sample complexity analysis for score matching.
result First finite sample bounds for score matching in polynomial exponential families.
We describe algorithms for learning Bayesian networks from a combination of user knowledge and statistical data. The algorithms have two components: a scoring metric and a search procedure. The scoring metric takes a network structure, statistical data, and a user's prior knowledge, and returns a score proportional to …
A common challenge in estimating parameters of probability density functions is the intractability of the normalizing constant. While in such cases maximum likelihood estimation may be implemented using numerical integration, the approach becomes computationally intensive. The score matching method of Hyvärinen [2005] …
A new method calibrates forecasts without sacrificing expertise.
problem Forecasters' calibration scores can be manipulated to appear expert.
method Deterministic and stochastic online procedures to calibrate forecasts.
result Calibration can be achieved without losing expertise.
New score-based methods identify causal structures with latent variables.
problem Identifying causal structures involving latent variables.
method Score-based methods with identifiability guarantees.
result Score equivalence and consistency for latent variable causal models.
This work extends score-based methods to binary data on the Boolean hypercube.
problem Learning and sampling binary data on the Boolean hypercube.
method Adopting Bernoulli noise as a smoothing device, deriving a TMF-like expression for the optimal denoiser, and using a Langevin-like sampler.
result The method successfully samples noisy binary data and reduces effective noise through multiple measurements.
Proposes a method to estimate causal effects of continuous treatments using instrumental variables.
problem Estimating causal effects of continuous treatments in the presence of unmeasured confounders.
method Introduces a novel framework using instrumental variables and a uniform regular weighting function to identify and estimate average dose-response functions.
result Establishes the asymptotic properties of the proposed methods for estimating average dose-response functions.
New method improves sampling from score-based models by correcting bias.
problem Bias in sampling from score-based diffusion models.
method Metropolis-Hastings or Barker's accept-reject steps to correct bias, using the score function.
result Improves sample quality on synthetic and image datasets, yielding consistent gains in FID.
Improved continuous-time consistency models for large-scale image generation.
problem Training instability and discretization errors in existing diffusion models.
method Unified theoretical framework, improved diffusion process, and network architecture.
result Trained continuous-time CMs at 1.5B parameters, achieving state-of-the-art FID scores.
In this paper, we propose an effective THresholding method based on ORder Statistic, called THORS, to convert an arbitrary scoring-type classifier, which can induce a continuous cumulative distribution function of the score, into a cost-sensitive one. The procedure, uses order statistic to find an optimal threshold for…
ScoreGrad predicts multivariate time series with energy-based models, achieving state-of-the-art results.
problem Predicting multivariate time series with generative models while considering noise and distribution.
method ScoreGrad uses continuous energy-based generative models with a feature extraction and score matching module.
result ScoreGrad achieves state-of-the-art results on six real-world datasets.
A new method improves density ratio estimation efficiency and accuracy.
problem Density ratio estimation trade-off between quality and efficiency.
method One-step Score-based Density Ratio Estimation (OS-DRE) combining analytic and solver-free approach.
result OS-DRE offers a favorable balance between estimation quality and inference efficiency.
Paper analyzes neural network models for sub-Gaussian distributions, proving approximation and generalization abilities.
problem Estimating unknown distributions from i.i.d. observations using neural network models.
method Score-based neural network generative models (SGMs) with specific network architectures and stopping strategies.
result SGMs can approximate scores with high accuracy and achieve nearly optimal convergence rates under mild assumptions.
The study provides guarantees for diffusion-based models under log-concave data, offering best-known convergence rates.
problem Theoretical guarantees for convergence of diffusion-based generative models under log-concave data distributions.
method Assumption of strongly log-concave data distributions, Lipschitz continuous functions for score estimation, and novel auxiliary process.
result Best known upper bounds for Wasserstein-2 distance between Gaussian distribution and sampling algorithm.
Paper improves variance control in importance weighted variational bounds.
problem Improving the variance of gradient estimators for IWAE.
method Develops a novel control variate that grows SNR as √K for large K.
result Empirically, the method yields superior variance reduction for generative models.
Many scientific questions require estimating the effects of continuous treatments. Outcome modeling and weighted regression based on the generalized propensity score are the most commonly used methods to evaluate continuous effects. However, these techniques may be sensitive to model misspecification, extreme weights o…
SJDs unify masked, continuous, and hybrid diffusion models.
problem Unified modeling of diffusion processes.
method Continuous-time Markov processes with token embeddings and hazard rates.
result Unified model recovers masked, continuous, and hybrid diffusion as limits.
Bayesian method improves approximate model posteriors.
problem Poor uncertainty quantification in approximate Bayesian inference.
method Optimizing a transformation of the approximate posterior to maximize a scoring rule.
result Significant reduction in bias and improvement in posterior coverage properties.
Generative Fractional Diffusion Models improve image diversity and quality.
problem Diffusion models struggle with diversity, mode-collapse, and slow convergence.
method Replaces light-tailed BM with fractional Brownian motion (fBM) and its Markov approximation (MA-fBM).
result GFDM achieves greater diversity and quality in image generation.
SGMs are robust to practical errors via uncertainty quantification.
problem Robustness of SGMs to practical implementation errors.
method Wasserstein uncertainty propagation (WUP) theorem and Bernstein estimates.
result SGMs are provably robust to multiple sources of error.
A framework for sensitivity measures using scoring functions.
problem Constructing sensitivity measures for any elicitable functional.
method Score-based sensitivities constructed via consistent scoring functions.
result Demonstrated intuitive and desirable properties of score-based sensitivities.