New algorithms make softmax optimization unbiased and scalable.
problem Efficiently computing softmax distributions with large categories.
method Proposed unbiased algorithms for maximizing softmax likelihood.
result Comprehensive outperformance on seven real-world datasets.
Unbiased methods for alpha-divergence minimization struggle in high dimensions.
problem The difficulty of unbiased alpha-divergence minimization in high dimensions.
method Signal-to-Noise Ratio (SNR) analysis of gradient estimators.
result The SNR of the gradient estimator worsens exponentially with dimensionality.
New unbiased methods for generating stochastic bridges with given extrema.
problem Generating unbiased stochastic bridges with a specified extremum.
method Comparison and generalization of two algorithms for Brownian bridges to other diffusions, and application to Ornstein-Uhlenbeck and unconstrained processes.
result Generalization of unbiased generation methods to other diffusions and application to various processes.
New algorithm finds unbiased subnetworks in biased datasets.
problem Finding unbiased subnetworks in biased neural networks.
method Debiased Contrastive Weight Pruning (DCWP) algorithm.
result DCWP significantly outperforms state-of-the-art debiasing methods.
A new algorithm resamples Bernoulli race particle filters using true weights.
problem Handling intractable weights in particle filters.
method Proposes a novel resampling method using true weights with an unbiased estimator.
result Demonstrates lower variance in filtering estimates compared to standard methods.
New framework for unbiased sampling of temporal networks.
problem Challenges in analyzing and modeling large, continuous temporal networks.
method General framework for unbiased temporal network sampling with online, single-pass algorithms and unbiased estimators.
result Effective algorithms for fast, accurate, and memory-efficient statistical estimation of temporal network patterns and properties.
Paper proposes an unbiased optimization method for Bayesian experimental design.
problem Maximizing expected information gain in Bayesian experimental design.
method Randomized multilevel Monte Carlo (MLMC) method combined with stochastic gradient descent.
result An unbiased estimator for the gradient of expected information gain.
Computing partition functions, the normalizing constants of probability distributions, is often hard. Variants of importance sampling give unbiased estimates of a normalizer Z, however, unbiased estimates of the reciprocal 1/Z are harder to obtain. Unbiased estimates of 1/Z allow Markov chain Monte Carlo sampling of "d…
Proposes an adversarial algorithm to learn unbiased representations via HGR coefficient.
problem Learning fair representations without sensitive attribute information.
method Adversarial algorithm using Hirschfeld-Gebelein-Renyi (HGR) maximal correlation coefficient.
result Significant improvements in bias mitigation compared to existing methods.
Paper develops unbiased gradient estimator for continuous-time models.
problem Estimating unbiased gradient of log-likelihood for continuous-time models.
method Doubly randomized scheme with coupled conditional particle filter (CCPF).
result Unbiased gradient estimate facilitates gradient-based algorithms.
Enhanced framework selects features for unbiased causal inference.
problem Unbiased estimation of causal quantities in causal inference.
method Three-stage computational framework balancing treatment and non-treatment variables.
result Significantly reduces bias and variance in estimating causal quantities.
Paper introduces a new gradient statistic to improve deep learning convergence.
problem Fluctuation effect of gradient updates between iterations.
method Introduces an unbiased stratified statistic \(\bar{G}_{mst}\) and a new algorithm MSSG.
result MSSG algorithm outperforms other sgd-like algorithms in training deep models.
Optimal Gaussian noise mechanisms achieve nearly optimal error in unbiased mean estimation.
problem Efficiently estimating the mean of high-dimensional data while preserving privacy.
method Differential privacy mechanisms with Gaussian noise, focusing on optimal covariance.
result Gaussian noise mechanisms achieve nearly optimal error among all private unbiased mean estimation mechanisms.
UORO reduces gradient variance in online RNN learning.
problem Improving gradient estimates in online RNN learning.
method Analyzes and proposes variance reduction techniques for UORO.
result Reduces gradient variance both theoretically and practically.
Paper presents IPVI for unbiased DGP inference with efficiency.
problem Intractable exact inference in deep Gaussian processes.
method Implicit posterior variational inference framework.
result IPVI achieves unbiased posterior belief with efficiency.
Unbiased method for Bayesian posterior means using kinetic Langevin dynamics.
problem Estimating Bayesian posterior means efficiently and accurately.
method Combines advanced splitting methods with enhanced gradient approximations in a multilevel Monte Carlo approach.
result The method achieves unbiased estimates with finite variance and central limit theorem properties.
ENSURE framework trains deep image recon algorithms without clean data.
problem Lack of clean, fully sampled ground-truth data for deep learning image reconstruction.
method Introduces ENSURE framework, a generalization of SURE and GSURE to random sampling patterns.
result ENSURE loss function is an unbiased estimate for true mean-square error.
Develops unbiased averaging methods for second order optimization in distributed systems.
problem Computing the Hessian is challenging and communication is a bottleneck in distributed optimization.
method Unbiased parameter averaging methods using sampling and sketching of the Hessian.
result Provably minimizes bias for sketched Newton directions.
A new framework for bilevel optimization tackles stochastic and global variance reduction.
problem Bilevel optimization challenges in large-scale empirical risk minimization.
method Introducing a novel framework where inner and main variables evolve simultaneously, leading to unbiased estimates and global variance reduction algorithms.
result SABA algorithm achieves $O( rac{1}{T})$ convergence rate and linear convergence under Polyak-Lojasciewicz assumption.
Synthetic construction of 3D complex bases.
problem Creating a complete set of unbiased bases in 3D complex space.
method Synthetic construction using complex projective trigonometry.
result Synthetic construction of mutually unbiased bases in C^3.
Online boosting for multiclass classification with limited feedback.
problem Online multiclass classification with bandit feedback.
method Proposed unbiased loss estimate and extended full information boosting algorithms to bandit setting.
result Asymptotic error bounds match full information counterparts, with larger sample complexity due to limited feedback.
In applications of Gaussian processes where quantification of uncertainty is of primary interest, it is necessary to accurately characterize the posterior distribution over covariance parameters. This paper proposes an adaptation of the Stochastic Gradient Langevin Dynamics algorithm to draw samples from the posterior …
Unbiased wealth exchanges always lead to inequality.
problem Understanding wealth distribution in unbiased binary exchange systems.
method Analytical demonstration of unbiased binary exchanges leading to perfect inequality.
result Any system driven by unbiased binary exchanges will reach perfect inequality and zero mobility.
The paper presents unbiased estimators for random design regression.
problem Bias in least squares solutions for random design regression.
method Volume-rescaled sampling of input points to produce unbiased estimators.
result An unbiased estimator can be constructed with a sample size of O(dlogd + d/ε).
CausalSim corrects bias in trace-driven simulations for more accurate results.
problem Bias in trace-driven simulations due to system conditions during trace collection.
method CausalSim learns a causal model of system dynamics and latent factors from an RCT to remove bias from trace data.
result CausalSim reduces simulation errors by 53% and 61% compared to baselines, providing more accurate insights.
Improved MUSE boosts performance and reduces error in Bayesian inference.
problem Hierarchical Bayesian inference problems
method Implicit differentiation applied to MUSE algorithm
result Significant speedup and improved accuracy compared to Hamiltonian Monte Carlo
This work improves texture segmentation by automatically tuning hyperparameters for Total-Variation.
problem The challenge is to automatically select hyperparameters for Total-Variation texture segmentation.
method The approach involves extending Stein's unbiased gradient estimator to handle correlated Gaussian noise, leading to an automatic tuning method.
result The method provides an automatic way to select hyperparameters for Total-Variation texture segmentation.
SUCAG accelerates convergence in distributed optimization with unbiased curvature information.
problem Efficiently solving finite sum optimization problems in distributed settings.
method Stochastic Unbiased Curvature-aided Gradient (SUCAG) method using Hessian information.
result Establishes linear convergence for strongly convex problems under asynchronous model.
Equalized coverage method ensures fair prediction intervals across protected groups.
problem Ensuring fair use of data-driven recommendation systems by communicating their uncertainty.
method Operational methodology that forces unbiased prediction intervals across all protected groups, offering rigorous distribution-free coverage guarantees.
result Equalized coverage constructs unbiased prediction intervals, unlike competitive methods.
Improved method for unbiased causal discovery in presence of unobserved confounding.
problem Unbiased data synthesis for causal discovery algorithms in the presence of unobserved confounding.
method Explicit block-hierarchical ancestral sampling to address limitations of implicit parameterization.
result Our approach fully covers the space of causal models, including those generated by implicit parameterization.
Procedure for unbiasedly estimating value of optimized policies.
problem Unbiased estimation of value of optimized policies in A/B testing.
method Bagging process with inverse-propensity-weighting and per-sample value estimates.
result Unbiased estimator of the value of deploying an optimized policy.
The paper introduces tools for designing responsible scoring mechanisms using unbiased function sampling.
problem Ensuring fairness and responsibility in scoring mechanisms used by data-driven systems.
method Unbiased function sampling and perturbation in the function space for designing responsible scoring mechanisms.
result A novel algorithm for approximating the construction of hyperplane arrangements, which is linear in the number of samples and hyperplanes.
We revisit resampling procedures for error estimation in binary classification in terms of U-statistics. In particular, we exploit the fact that the error rate estimator involving all learning-testing splits is a U-statistic. Thus, it has minimal variance among all unbiased estimators and is asymptotically normally dis…
MUSE provides unbiased stopping estimates for optimal problems.
problem Estimating the utility of optimal stopping problems.
method Backward recursive construction of the Multilevel Unbiased Stopping Estimator (MUSE).
result MUSE achieves ε-accuracy with O(1/ε^2) computational cost.
A new method improves communication efficiency in distributed learning.
problem Reducing communication overhead in distributed machine learning.
method Transforming contractive compressors into induced unbiased compressors.
result Significant improvements in memory requirements and communication complexity.
U-Clip clips gradients to reduce bias without discarding them.
problem Reduces bias in stochastic gradient descent.
method Maintains a buffer of clipped gradients to add to next iteration.
result U-Clip updates are unbiased on average.
Contextual bandit algorithms have become popular for online recommendation systems such as Digg, Yahoo! Buzz, and news recommendation in general. \emph{Offline} evaluation of the effectiveness of new algorithms in these applications is critical for protecting online user experiences but very challenging due to their "p…
Federated learning improves by unbiased gradient aggregation and controllable meta updating.
problem Gradient biases and inconsistency between target and optimization objectives in federated averaging.
method Unbiased gradient aggregation with keep-trace gradient descent and gradient evaluation strategy, controllable meta updating with small data samples.
result Faster convergence and higher accuracy with different network architectures in various FL settings.
In this paper we address the problem of pool based active learning, and provide an algorithm, called UPAL, that works by minimizing the unbiased estimator of the risk of a hypothesis in a given hypothesis space. For the space of linear classifiers and the squared loss we show that UPAL is equivalent to an exponentially…
Stochastic EM with biased MCMC improves inference stability.
problem Intractable E-step in EM algorithm.
method Stochastic approximation with biased MCMC.
result ULA is more stable and sometimes faster than MALA.
Proposes a sample selection algorithm for fair and robust AI training.
problem Balancing fairness and robustness in AI models, especially with corrupted data.
method Formulates and solves a combinatorial optimization problem for unbiased sample selection, proposing a greedy algorithm.
result Improves fairness and robustness compared to state-of-the-art techniques, both synthetically and on real datasets.
Proposes unbiased estimators for training mixture of experts models.
problem Efficiently training large-scale mixture of experts models on modern hardware.
method Two unbiased estimators based on principled stochastic assignment procedures.
result Both estimators are more effective and robust than biased alternatives.
New method for inferring network topology from partial data.
problem Inferring network topology from limited node data.
method Vector autoregressive model and Gaussian mixture algorithm.
result The proposed method converges to the network combination matrix in probability.
A key quantity of interest in Bayesian inference are expectations of functions with respect to a posterior distribution. Markov Chain Monte Carlo is a fundamental tool to consistently compute these expectations via averaging samples drawn from an approximate posterior. However, its feasibility is being challenged in th…
SUMO provides unbiased log marginal likelihood estimation for latent variable models.
problem Biased estimates of log marginal likelihood in latent variable models.
method Randomized truncation of infinite series for unbiased estimation.
result Models trained with SUMO give better test-set likelihoods than standard methods.
Unbiased smoothing method improves accuracy in noisy measurements.
problem Approximating latent Markov process expectations with noisy measurements.
method Coupling Particle Independent Metropolis-Hastings (PIMH) chains.
result Unbiased smoothing estimators for parallel computing and confidence intervals.
Unified framework improves fair classification by selecting representative data points.
problem Improving fair classification outcomes in the presence of unintentional biases.
method Develops a unified framework to jointly optimize accuracy and fairness, recasting as mixed-integer convex programs.
result The framework can be used to enhance classification fairness by selecting more representative data points.
Unbiased gradient estimation for Markov chains
problem Estimating gradients of stationary means in Markov chains
method Propose new unbiased estimators
result Improves efficiency for slow mixing Markov chains