QC-ST and CoCo methods correct batch effects in metabolomics data.
problem Batch effects in metabolomics data obscure biological variations.
method QC-ST for simultaneous detection of QC samples' mean vectors and covariance matrices, CoCo for covariance correction.
result QC-ST and CoCo improve batch effect correction in metabolomics datasets.
GANs correct batch effects in biological data.
problem Batch effects confound biological data analysis.
method Generative Adversarial Networks (GANs) framework.
result GANs framework corrects batch effects without kernel selection.
Framework aligns datasets using harmonic expansion of intrinsic geometry.
problem Combining datasets from different modalities or correcting batch effects.
method Alignment through harmonic expansion of diffusion coordinates.
result Unified diffusion geometry for fused or corrected data.
A new BN method corrects size mismatch in WLF for imbalanced data.
problem Learning from imbalanced datasets in neural networks.
method Proposes weighted batch normalization (WBN) to correct size mismatch between BN and WLF.
result WBN corrects size mismatch and improves classification performance in imbalanced datasets.
A new method improves data representation for diverse tasks.
problem Learning meaningful representations for tasks like batch correction and counterfactual inference.
method Contrastive Mixture of Posteriors (CoMP) method using misalignment penalties.
result CoMP achieves state-of-the-art performance on challenging tasks.
Improved BN for better performance in imbalanced data.
problem Improving classification accuracy in imbalanced datasets.
method Combining weighted loss function with modified batch normalization.
result Modified batch normalization improves classification accuracy in imbalanced datasets.
Improved TD learning reduces batch sampling error.
problem TD learning can be inaccurate in batch settings due to action weighting.
method Introduced PSEC-TD(0) to correct action weighting using importance sampling.
result PSEC-TD(0) produces more accurate value function estimates.
Anytime-valid confirmation of label-shift corrections
problem Small-batch scientific deployments with scarce labeled outcomes
method Conditional e-value and martingale-based rule
result Nonnegative martingale and anytime-valid confirmation rule
Optimal δ-correct algorithm for heavy-tailed distributions with minimal samples.
problem Identifying the best arm from a set of unknown distributions with heavy tails.
method Developed a δ-correct algorithm that matches lower bounds under mild restrictions.
result The proposed algorithm achieves optimal sample complexity for heavy-tailed distributions.
Bias correction improves language model training performance.
problem Stochastic update bias in preconditioned optimizers.
method Cross-fitted preconditioning and variance-corrected inversion.
result Reduces held-out pretraining loss by 0.15 nats.
DAIS improves AIS for differentiable marginal likelihood estimation.
problem Differentiable marginal likelihood estimation for complex models.
method Proposes Differentiable Annealed Importance Sampling (DAIS) to make AIS differentiable.
result DAIS achieves convergence and consistency in Bayesian linear regression.
Recently, a lot of effort has been paid to the efficient computation of Kriging predictors when observations are assimilated sequentially. In particular, Kriging update formulae enabling significant computational savings were derived in Barnes and Watson (1992), Gao et al. (1996), and Emery (2009). Taking advantage of …
Proposes batch-weight method to correct mode imbalance in domain adaptation.
problem Mode imbalance between source and target distributions affects unsupervised domain transfer.
method Proposes batch-weight method to re-weight training samples.
result Effective in several image-to-image translation tasks.
A new method for batch prediction sets in classification problems.
problem Constructing reliable prediction sets for multiple unlabeled examples.
method Proposes a uniformly more powerful approach to batch prediction sets using specific combinations of conformal p-values.
result The proposed method provides narrower prediction sets compared to the Bonferroni correction.
The study analyzes batched methods for early stopping in stochastic multi-armed bandits.
problem Early stopping in stochastic multi-armed bandits with fixed confidence.
method Instance-dependent lower bounds and a general batched algorithm with upper bounds.
result Upper and lower bounds on the number of batches and sample complexity.
This study investigates the impact of importance weighting in deep learning models.
problem Understanding the effect of importance weighting in deep neural networks.
method The study uses theoretical and empirical approaches to analyze the behavior of importance weighting in deep learning models.
result Importance weighting impacts models early in training but diminishes over successive epochs in deep neural networks.
Proposes DC-S3GD for efficient large-scale decentralized neural network training.
problem Training large-scale decentralized neural networks efficiently.
method Decentralized stale-synchronous version of DC-ASGD with gradient correction.
result Achieves state-of-the-art results in training Convolutional Neural Networks.
Self-paced learning and hard example mining re-weight training instances to improve learning accuracy. This paper presents two improved alternatives based on lightweight estimates of sample uncertainty in stochastic gradient descent (SGD): the variance in predicted probability of the correct class across iterations of …
AMAGOLD improves stochastic gradient MCMC by infrequent Metropolis-Hastings corrections.
problem Bias in stochastic gradient Hamiltonian Monte Carlo (SGHMC).
method AMAGOLD infrequently uses Metropolis-Hastings corrections to remove bias, with a fixed step size schedule.
result AMAGOLD converges to the target distribution with a fixed, rather than a diminishing, step size, and at most a constant factor slower convergence rate.
We make policy optimization algorithms batch size-invariant by decoupling proximal and behavior policies.
problem Some policy optimization algorithms do not have batch size-invariance, leading to inefficiencies.
method We decouple the proximal policy from the behavior policy to achieve batch size-invariance.
result Our approach makes policy optimization algorithms more efficient and allows them to use stale data more effectively.
Geometric approach finds correspondences between different conditions.
problem Integrating multiple biological datasets.
method Fibered latent space with pull-back metric, diffeomorphism flows.
result Minimizing energy functional yields diffeomorphism flows.
Two new algorithms improve Q* approximation in batch RL with linear error propagation.
problem Improving Q* approximation in batch reinforcement learning.
method Two novel algorithms that estimate Bellman error directly, without quadratic dependence.
result Linear-in-horizon error propagation for batch RL algorithms.
Adaptive Langevin dynamics reduces bias in Bayesian inference with mini-batching.
problem Bias in posterior sampling due to mini-batching in Bayesian inference.
method Adaptive Langevin dynamics with dynamical friction to correct noise.
result Quantified bias in posterior distribution due to mini-batching.
Large batch sizes improve model performance but have limits; we found a simple predictor.
problem Understanding the limits of large batch sizes across different domains.
method Empirical model using gradient noise scale to predict optimal batch size.
result Gradient noise scale predicts the largest useful batch size across various domains.
New method for constructing confidence intervals for time series data.
problem Constructing confidence intervals for statistical functionals from time series data.
method Proposes a general purpose confidence interval procedure based on overlapping batches of time series data.
result Large overlapping batches yield confidence intervals of higher quality than generic methods.
Batch augmentation improves deep learning training by reducing batch size requirements.
problem Training deep neural networks with large batches can lead to overfitting.
method Replicate samples within a batch with different data augmentations.
result Batch augmentation reduces the number of necessary SGD updates for achieving the same accuracy.
Paper proposes an efficient method for bounding box annotation in object detection.
problem Manual annotation of bounding boxes is tedious and resource-intensive.
method Iterative training of object detector on small batches of labeled images, with human annotator correcting errors.
result Significant reduction in human annotation effort, up to 75%.
Large batch sizes reduce gradient variance in DP-SGD, improving privacy.
problem Understanding why large batch sizes work in DP-SGD.
method Decomposed total gradient variance into subsampling and noise-induced variances, proving batch size independence in the limit.
result Large batch sizes reduce effective total gradient variance, improving privacy in DP-SGD.
Paper explains contrastive learning using cosine similarity and proposes mitigations for batch size effects.
problem Understanding and improving contrastive learning through batch size effects.
method Unified framework of cosine similarity, theoretical insights, and auxiliary loss.
result Performance improvement in small-batch settings through proposed auxiliary loss.
Paper improves anomaly detection using tree-based ensembles with active learning.
problem Configuring anomaly detectors with true labels to minimize false positives.
method Develops batch and streaming active learning algorithms for tree-based ensembles.
result Significantly more anomalies discovered with active learning compared to baselines.
Introduce a variance-weighted batch distribution for diverse sampling in diffusion models.
problem Independent sampling in diffusion models.
method Introduce a variance-weighted batch distribution.
result Sampler with a transparent probabilistic target.
New framework optimizes deep learning training by deferring large batch sizes to late stages.
problem Optimizing batch size scheduling for deep learning training efficiency.
method Introduced the functional scaling law (FSL) framework to analyze and optimize batch size scheduling.
result Large batch sizes can be deferred to late training stages without sacrificing performance.
CBN improves batch normalization for small mini-batch sizes.
problem Reduced effectiveness of Batch Normalization in small mini-batch sizes.
method CBN uses statistics from multiple recent iterations, compensating for network weight changes via Taylor polynomials.
result CBN outperforms original batch normalization and direct iteration statistics in object detection and image classification.
Bayesian priors offer a compact yet general means of incorporating domain knowledge into many learning tasks. The correctness of the Bayesian analysis and inference, however, largely depends on accuracy and correctness of these priors. PAC-Bayesian methods overcome this problem by providing bounds that hold regardless …
Four improvements to Batch Normalization improve deep learning performance.
problem Improving Batch Normalization for better deep learning performance.
method Proposed improvements include reasoning about current examples, Ghost Batch Normalization, weight decay regularization, and a new normalization algorithm for small batch sizes.
result Performance gains across all batch sizes with no additional computation during training.
Sources of variability in experimentally derived data include measurement error in addition to the physical phenomena of interest. This measurement error is a combination of systematic components, originating from the measuring instrument, and random measurement errors. Several novel biological technologies, such as ma…
This paper investigates the impact of normalization on deep neural networks for click-through rate prediction.
problem The effect of normalization on deep neural network models for CTR estimation.
method Systematic study of various normalization approaches applied to feature embedding and MLP part of DNN models.
result Correct normalization significantly enhances model performance, as demonstrated by extensive experiments on real-world datasets.
Method detects batch heterogeneity in genomic data.
problem Batch effects confound genomic diagnostics.
method Bayesian model evidence clustering.
result Detects batch effects without known labels.
Neural networks learn the support of the target function through SGD's implicit regularization effect.
problem Learning the support of the target function in neural networks.
method Investigation of mini-batch SGD's ability to learn the support in the first layer of a neural network.
result Mini-batch SGD effectively learns the support in the first layer by shrinking irrelevant weights, while vanilla GD requires an explicit regularization term.
New SGD algorithm finds critical points faster with second-order corrections.
problem Finding critical points in non-convex optimization efficiently.
method Uses Hessian-vector products to correct momentum bias in SGD.
result Finds ε-critical points in O(ε−3) time. New algorithm SAGA++ improves on SAGA for faster convergence in stochastic batch size methods.
problem Optimizing convergence rate in stochastic variance reduction methods with batch size.
method Proposes SAGA++ algorithm with optimal average batch size considering cache/disk IO effects.
result SAGA++ outperforms SAGA and other solvers on real datasets.
BaSE policy optimizes multi-armed bandits with batched data.
problem Optimizing multi-armed bandits with batched data.
method BaSE (batched successive elimination) policy for batched multi-armed bandits.
result Achieves rate-optimal regrets with adaptive batch sizes.
New method reduces labeling costs in semi-supervised learning.
problem Efficiently label data with limited labeled samples.
method Formulates batch acquisition as bilevel optimization.
result Shows effectiveness in keyword detection tasks.
WEINCE improves contrastive learning by correcting softmax biases.
problem Softmax in InfoNCE can lead to misaligned statistical assumptions in contrastive learning.
method WEINCE uses anchor-wise online batch statistics to blend softmax logits with an endpoint shortfall correction.
result WEINCE yields consistent improvements in frozen-feature evaluation across five vision benchmarks.
Adam optimizer's bias is influenced by mini-batch size and momentum hyperparameters.
problem Understanding how Adam's implicit bias is affected by mini-batch size and momentum parameters.
method Theoretical framework to analyze mini-batch noise's impact on Adam's memory and bias.
result The magnitude of anti-regularization by memory depends on batch size and momentum hyperparameters.
Empirical study on SGD hyperparameters and adversarial robustness.
problem Effect of SGD hyperparameters on adversarial robustness and generalization.
method Empirical observation of learning rate, batch size, and momentum effects on adversarial robustness and generalization.
result Constant learning rate to batch size ratio leads to good generalization and almost constant adversarial robustness.
Batch Active Learning uses derivative information for Gaussian Process regression.
problem Efficiently selecting data batches in Gaussian Process regression models.
method Proposes using the predictive covariance matrix to select data batches, exploiting full correlation.
result Demonstrates the effectiveness of incorporating derivative information across diverse applications.
PSI-LinUCB improves scalability for large recommender systems.
problem Efficiently training and inferring for large action spaces in recommender systems.
method Represent inverse design matrix as diagonal + low-rank correction, derive stable rank-1 and batched updates, use projector-splitting integrator.
result Demonstrated effectiveness on recommender system datasets, achieving scalable training and inference.