Framework improves gradient estimation for faster training convergence.
problem Efficiently estimating noisy gradients in stochastic optimization.
method Dynamic adaptive importance sampling combining multiple distributions.
result Adaptively weighted multiple importance sampling yields superior gradient estimates.
The clustering algorithms that view each object data as a single sample drawn from a certain distribution, Gaussian distribution, for example, has been a hot topic for decades. Many clustering algorithms: such as k-means and spectral clustering are proposed based on the single sample assumption. However, in real life, …
Paper finds efficient OPE estimator for multiple logging policies with minimum variance.
problem Finding optimal importance sampling weights for multiple logging policies with varying variances.
method Established efficiency bound under stratified sampling and proposed an estimator achieving this bound.
result Proposed estimator achieves minimum variance for any instance.
MT-SGD samples from multiple target distributions using gradient descent.
problem Sampling from multiple unnormalized target distributions.
method Proposes MT-SGD, a flow of intermediate distributions to sample from multiple target distributions.
result Asymptotic analysis shows MT-SGD reduces to multiple-gradient descent for multi-objective optimization.
Matrix multiplication is a fundamental building block for large scale computations arising in various applications, including machine learning. There has been significant recent interest in using coding to speed up distributed matrix multiplication, that are robust to stragglers (i.e., machines that may perform slower …
New sampling-based approach for filtering problems using multiplicative Gaussian functions.
problem Approximate inference in filtering problems.
method Approximates distribution with a weighted sum of continuous functions using sampling for multiplications.
result Preliminary experiments show potential of the new method compared to particle filters.
New method combines multiple data sources for optimal decision-making with limited outcomes.
problem Optimal decision-making with limited outcome data from multiple heterogeneous sources.
method Calibrated optimal decision-making method leveraging common intermediate outcomes.
result Proposed estimator of conditional mean outcome is asymptotically normal and more efficient.
We determine the sample complexity of pure exploration bandit problems with multiple good answers. We derive a lower bound using a new game equilibrium argument. We show how continuity and convexity properties of single-answer problems ensures that the Track-and-Stop algorithm has asymptotically optimal sample complexi…
This paper solves the multiple reference model problem in RLHF with exact solutions and sample complexity guarantees.
problem Limitations of single reference models in aligning LLMs with human feedback.
method Integrates multiple reference models into RLHF frameworks, addressing theoretical challenges with exact solutions and sample complexity guarantees.
result First exact solution to the multiple reference model problem in reverse KL-regularized RLHF.
We propose a patch sampling strategy based on a sequential Monte-Carlo method for high resolution image classification in the context of Multiple Instance Learning. When compared with grid sampling and uniform sampling techniques, it achieves higher generalization performance. We validate the strategy on two artificial…
Improved locally private sparse estimation with multiple samples per user.
problem Challenges in high-dimensional locally private sparse estimation.
method Proposes a framework for user-level locally private sparse linear regression with multiple samples per user.
result Eliminates the dependency of dimensionality on error bounds, achieving tighter error bounds.
We analyze the sample complexity of learning from multiple experiments where the experimenter has a total budget for obtaining samples. In this problem, the learner should choose a hypothesis that performs well with respect to multiple experiments, and their related data distributions. Each collected sample is associat…
Proposes using Wasserstein barycenters for robust optimization with multiple data sources.
problem Distributionally robust optimization with multiple heterogeneous data sources.
method Construct nominal distribution through Wasserstein barycenter of multiple data samples, reformulates as a finite convex program.
result Proposed scheme outperforms other estimators in sparse inverse covariance matrix estimation.
Improved SVMs learn from few samples with composition and multiple scales.
problem Learning with small sample sizes.
method Transformation-invariant SVMs with composition and locality at multiple scales.
result Kernels based on maximum similarity are positive definite and yield superior accuracy.
Paper adapts multiplicative weights method to Gaussian graphical models.
problem Graphical model selection in Gaussian random fields.
method Adapted multiplicative weights method from Ising model to Gaussian model.
result Achieves sample complexity bound similar to existing methods.
SMTM improves MCMC sampling in high dimensions with multiple proposals and stereographic integration.
problem Improving MCMC performance in high-dimensional sampling.
method Integrating multiple-try Metropolis with stereographic MCMC framework.
result SMTM outperforms classical MTM and other methods in high-dimensional sampling.
Study reduces NAS search cost by generating multiple complex architectures in one shot.
problem Finding multiple neural architectures with varying complexities efficiently.
method Uses importance sampling to generate and update multiple distributions of architectures.
result Reduces search cost by finding multiple architectures with different complexities in a single search.
Shielded LMC samples from non-convex spaces with repulsive drift.
problem Sampling from non-convex spaces with convex holes.
method Combining adaptive temperature and repulsive drift.
result Advantages over unconstrained sampling in constrained spaces.
Introduces Rashomon Capacity to measure predictive multiplicity in probabilistic classifiers.
problem Predictive multiplicity in classification models leading to unjustified decisions.
method Introduces Rashomon Capacity, a metric for probabilistic classifiers, and provides a rigorous derivation.
result Rashomon Capacity captures nuanced score variations and provides strategies for disclosing conflicting models.
We present a TTS neural network that is able to produce speech in multiple languages. The proposed network is able to transfer a voice, which was presented as a sample in a source language, into one of several target languages. Training is done without using matching or parallel data, i.e., without samples of the same …
Enhances anomaly detection using multiple reference datasets.
problem Lack of performance with multiple reference datasets.
method Generalizes existing techniques CWoLa and SALAD for multiple reference datasets.
result Improved performance across various settings.
Enhanced feedback model improves sample-efficiency in POMDPs.
problem Exponential hardness of learning in POMDPs.
method Multiple observations in hindsight feedback model.
result Sample-efficient learning possible for new subclasses of POMDPs.
We discuss a multiple-play multi-armed bandit (MAB) problem in which several arms are selected at each round. Recently, Thompson sampling (TS), a randomized algorithm with a Bayesian spirit, has attracted much attention for its empirically excellent performance, and it is revealed to have an optimal regret bound in the…
Localized sketching improves matrix multiplication and ridge regression complexity.
problem Efficiently approximate matrix multiplication and ridge regression with limited data availability.
method Localized sketching matrices for block diagonal structure, reducing sample complexity.
result Localized sketching achieves sample complexity matching global sketching methods.
Paper settles sample complexity for learning from multiple distributions.
problem Learning from multiple data distributions with a hypothesis class of bounded VC dimension.
method Introduced an algorithm with sample complexity of O((d+k)ε^-2)·(k/ε)^o(1).
result Algorithm matches lower bound up to sub-polynomial factor.
WiGS improves active learning for regression by dynamically selecting informative samples.
problem Reducing labeling costs in regression tasks.
method Formulated as a reinforcement learning problem, WiGS adapts the exploration-investigation balance.
result WiGS outperforms static methods in accuracy and labeling efficiency, especially in irregular data density.
Given samples from a distribution, how many new elements should we expect to find if we continue sampling this distribution? This is an important and actively studied problem, with many applications ranging from unseen species estimation to genomics. We generalize this extrapolation and related unseen estimation proble…
Bayesian methods improve tracking multiple objects through dynamic dependencies.
problem Tracking multiple objects with time-varying cardinality and unordered measurements.
method Employing Bayesian nonparametric models, specifically dependent Dirichlet and Pitman-Yor processes, for state estimation and Monte Carlo sampling for trajectory learning.
result The proposed methods outperform existing algorithms in estimating the time-varying number of objects and identifying object associations.
Unified framework for simulation-based inference learns a single model for multiple tasks.
problem Simulation-based inference for multiple tasks with limited model retraining.
method Unified flow-matching generative model with query-aware masking distribution.
result Competitive performance on various inference tasks and real-world problems.
We describe the first sub-quadratic sampling algorithm for the Multiplicative Attribute Graph Model (MAGM) of Kim and Leskovec (2010). We exploit the close connection between MAGM and the Kronecker Product Graph Model (KPGM) of Leskovec et al. (2010), and show that to sample a graph from a MAGM it suffices to sample sm…
Advances in molecular "omics'" technologies have motivated new methodology for the integration of multiple sources of high-content biomedical data. However, most statistical methods for integrating multiple data matrices only consider data shared vertically (one cohort on multiple platforms) or horizontally (different …
A new bandit algorithm for web page item display.
problem Optimizing item display at specific positions on a web page.
method Position-based multiple-play bandits with Thompson sampling.
result Delivers better recommendations with less prior information.
Researchers study fairness-accuracy tradeoffs in predictive models for multiple groups.
problem Understanding the tradeoff between fairness and accuracy in models serving multiple demographic groups.
method Characterizing the fairness-accuracy (FA) Pareto frontier, approximating it from limited data, and bounding the worst-case gap.
result Derivation of worst-case-optimal estimators and uniform finite-sample bounds for the entire FA frontier.
A weakly-supervised learning framework named as complementary-label learning has been proposed recently, where each sample is equipped with a single complementary label that denotes one of the classes the sample does not belong to. However, the existing complementary-label learning methods cannot learn from the easily …
LLM-as-a-service prices vary arbitrarily due to tokenization multiplicity.
problem Arbitrary price variation in LLM-as-a-service due to multiple tokenizations of the same output.
method Introduce canonical generation to restrict LLMs to unique tokenizations and develop an efficient sampling algorithm.
result Our sampling algorithm for canonical generation solves tokenization multiplicity and maintains comparable performance and runtime to standard sampling.
Neural Architecture Search (NAS) has shown great potentials in finding better neural network designs. Sample-based NAS is the most reliable approach which aims at exploring the search space and evaluating the most promising architectures. However, it is computationally very costly. As a remedy, the one-shot approach ha…
Measures consistency of tabular LLM predictions under fine-tuning multiplicity.
problem Conflicting predictions from fine-tuned tabular LLMs.
method Local stability measure in embedding space.
result Probabilistic guarantees on prediction consistency under multiplicity.
We propose a Genetic Programming architecture for the generation of foreign exchange trading strategies. The system's principal features are the evolution of free-form strategies which do not rely on any prior models and the utilization of price series from multiple instruments as input data. This latter feature consti…
Introduces MLM dataset for multitask learning across multiple languages and modalities.
problem Training and evaluating multitask systems on diverse data.
method Develops a new benchmark dataset with semantic data and modalities.
result Demonstrates challenges in generalizing on diverse data.
This paper studies the sample complexity of searching over multiple populations. We consider a large number of populations, each corresponding to either distribution P0 or P1. The goal of the search problem studied here is to find one population corresponding to distribution P1 with as few samples as possible. The main…
Proposes a robust FMR model for handling sample heterogeneity.
problem Handling sample heterogeneity with a single regression model.
method Clusters samples and jointly models multiple incomplete mixed-type targets.
result Achieves state-of-the-art performance on synthetic and real-world data.
Algorithm identifies Pareto front using multiple context directions and reuses exploration samples.
problem Identifying a set of arms with undominated mean reward vectors in linear bandits.
method Proposes a new estimator that updates estimates along multiple context directions and reuses exploration samples.
result Optimal sample complexity and logarithmic regret compared to optimal algorithms.
We study binary classification in the setting where the learner is presented with multiple corrupted training samples, with possibly different sample sizes and degrees of corruption, and introduce an approach based on minimizing a weighted combination of corruption-corrected empirical risks. We establish a generalizati…
In the mixture models problem it is assumed that there are K distributions θ1,…,θK and one gets to observe a sample from a mixture of these distributions with unknown coefficients. The goal is to associate instances with their generating distributions, or to identify the parameters of the hidden distribu…
We learn linear models from nonlinear systems using multiple trajectories and regularization.
problem Identifying linear models from data when the underlying dynamics are nonlinear.
method Multiple trajectories data acquisition followed by regularized least squares.
result Learn linearized dynamics with arbitrarily small error given enough samples.
Study tackles RLHF with diverse human feedback, showing limitations and proposing a meta-learning approach.
problem Traditional RLHF fails to balance diverse human preferences.
method Integrates meta-learning and multiple social welfare functions to optimize diverse preferences.
result Establishes sample complexity bounds for optimizing diverse social welfare functions.
The paper develops methods to identify stable associations across multiple studies.
problem Identifying stable associations across multiple studies with possible distributional shifts.
method Modeling heterogeneous multi-source data with multiple high-dimensional regressions and devising a novel sampling method for valid confidence intervals of maximin effects.
result Significant maximin effects indicate stable associations that can be generalized to target populations.
PBO framework optimizes latent preferences over multiple objectives.
problem Optimizing latent preferences with multiple conflicting objectives.
method Proposes DSTS, a multi-objective generalization of dueling Thompson sampling.
result DSTS outperforms benchmarks and provides asymptotic consistency.