Paper proposes a 1-bit mean estimation method with near-optimal sample complexity.
problem Distributed mean estimation with 1-bit communication constraints.
method Randomized and sequentially-chosen interval queries to estimate mean.
result Sample complexity bound matches minimax lower bound with logarithmic factors.
FAQ efficiently evaluates LLMs with statistical guarantees using adaptive query selection.
problem Efficiently evaluating many LLMs on a large suite of benchmarks is expensive.
method FAQ uses Bayesian factor models, adaptive sampling, and proactive active inference to select queries.
result FAQ delivers up to 5x effective sample size gains over baselines, matching CI width with fewer queries.
New insights on active sequential prediction for mean estimation.
problem Active sequential prediction-powered mean estimation problem.
method Combining uncertainty-based suggestion with a constant probability, analyzing non-asymptotic bounds, and using no-regret learning.
result The optimal query probability is close to the constraint when using no-regret learning.
New method reduces regret in budgeted learning problems.
problem Decision-making with limited reward queries.
method Confidence-Budget Matching (CBM) principle.
result CBM-based algorithms perform well in adversarial settings.
Having a regression model, we are interested in finding two-sided intervals that are guaranteed to contain at least a desired proportion of the conditional distribution of the response variable given a specific combination of predictors. We name such intervals predictive intervals. This work presents a new method to fi…
New protocols show 1-bit mean estimation can be order-optimal without interaction.
problem Can 1-bit mean estimation be optimal without interaction?
method Adaptive and non-adaptive threshold and interval queries, with one adaptive transition.
result Arbitrary non-adaptive quantizers can match the adaptive rate, suggesting interaction is not necessary.
We study the query complexity of Bayesian Private Learning: a learner wishes to locate a random target within an interval by submitting queries, in the presence of an adversary who observes all of her queries but not the responses. How many queries are necessary and sufficient in order for the learner to accurately est…
Adaptive truncation improves privacy in online Bayesian estimation.
problem Ensuring privacy in online Bayesian estimation of a static parameter.
method Sequential Monte Carlo, adaptive truncation, Thompson sampling.
result Adaptive truncation reduces privacy-preserving noise, enabling more accurate estimation.
Algorithm identifies nearest mode in noisy data.
problem Identifying the point with the minimum k-th nearest neighbor distance in unknown multivariate probability density.
method Sequential learning algorithm using noisy oracle queries to adaptively decide which points to query.
result Upper bounds on query complexity show significant improvement over baselines.
We design a general framework for answering adaptive statistical queries that focuses on providing explicit confidence intervals along with point estimates. Prior work in this area has either focused on providing tight confidence intervals for specific analyses, or providing general worst-case bounds for point estimate…
The paper develops methods to estimate frequencies in large discrete data sets with improved coverage and robustness.
problem Estimating frequencies in large, discrete data sets with valid coverage and robustness.
method Conformal inference methods using discrete sketches, marginal coverage for queries, and novel conformal calibration.
result Improved empirical performance compared to existing methods in simulations and real data.
New algorithms achieve optimal regret in sliding window model with limited memory.
problem Experts problem in the sliding window model with limited information.
method 2 queries, polylog(nT) memory, exponential improvement on memory.
result Achieve optimal regret of sqrt(nW)polylog(nT) with 2 queries and polylog(nT) memory.
TQA improves prediction intervals for time series data by adjusting quantiles for both cross-sectional and longitudinal coverage.
problem Constructing reliable prediction intervals for cross-sectional time series data.
method Temporal Quantile Adjustment (TQA) method that adjusts the quantile in Conformal Prediction to account for both cross-sectional and longitudinal coverage.
result TQA improves longitudinal coverage while preserving cross-sectional coverage, as validated through extensive experimentation.
The paper tackles efficient change point detection with limited samples.
problem Identifying multiple change points with minimal queries in noisy environments.
method Adaptive algorithm that first detects likely change points and refines their locations.
result The sample complexity is jointly governed by jump magnitudes and change point positions.
Paper develops online statistical inference methods for stochastic optimization using Kiefer-Wolfowitz algorithms.
problem Online statistical inference of model parameters in stochastic optimization problems.
method Kiefer-Wolfowitz algorithm with random search directions, asymptotic distribution analysis.
result Developed valid confidence intervals for online statistical inference.
Flexible method for estimating frequencies in large datasets using sketching.
problem Estimating frequencies in very large datasets efficiently and accurately.
method Data-adaptive conformal inference method based on a smaller sketch of data.
result Proves validity of frequentist confidence intervals under data exchangeability.
Probabilistic representations, such as Bayesian and Markov networks, are fundamental to much of statistical machine learning. Thus, learning probabilistic representations directly from data is a deep challenge, the main computational bottleneck being inference that is intractable. Tractable learning is a powerful new p…
The paper introduces CPICFs for better counterfactual explanations in high-dimensional spaces.
problem Creating useful counterfactual explanations for complex machine learning models.
method Modeling individual knowledge and using conformal prediction intervals to identify informative counterfactuals.
result CPICFs provide more informative counterfactuals by considering individual knowledge and prediction uncertainty.
Active local learning uses fewer labels to predict near-optimal functions.
problem Efficiently predicting near-optimal functions with fewer labels.
method Active local learning algorithm for estimating functions with fewer labels.
result Algorithm makes significantly fewer label queries than traditional methods.
Paper develops an efficient mean estimator for 1-bit communication constraints.
problem Mean estimation under 1-bit communication constraints.
method Adaptive mean estimator based on randomized threshold queries.
result Order-optimal sample complexity in various tail regimes.
Proposes a new model to identify unknown counterfactual outcomes for continuous variables.
problem Counterfactual inference for continuous outcomes with strong assumptions.
method Curvature Sensitivity Model to relax assumptions and provide informative bounds.
result Demonstrates effectiveness of the Curvature Sensitivity Model in identifying counterfactual outcomes.
A new method combines synthetic data analysis and DP generation to produce accurate uncertainty estimates.
problem Invalid inferences from DP synthetic data analysis.
method Combining synthetic data analysis techniques from MI and NA Bayesian modeling with a novel noise-aware synthetic data generation algorithm.
result Accurate confidence intervals from DP synthetic data are produced, wider with tighter privacy.
Online minimization of an unknown convex function over the interval [0,1] is considered under first-order stochastic bandit feedback, which returns a random realization of the gradient of the function at each query point. Without knowing the distribution of the random gradients, a learning algorithm sequentially choo…
Improved DP KDE with better privacy and efficiency.
problem Kernel density estimation with differential privacy.
method Refined search tree construction for DP KDE.
result Improved query time and approximation ratio.
Paper develops PAC verification for hypothesis classes and statistical algorithms.
problem Verifying machine learning models interactively.
method Develops interactive proof for PAC verification, proves lower bounds, and introduces a generalization.
result Improved protocol for verifying unions of intervals and statistical query algorithms.
Decision tree classifiers are a widely used tool in data stream mining. The use of confidence intervals to estimate the gain associated with each split leads to very effective methods, like the popular Hoeffding tree algorithm. From a statistical viewpoint, the analysis of decision tree classifiers in a streaming setti…
In this work, we give the first algorithms for tolerant testing of nontrivial classes in the active model: estimating the distance of a target function to a hypothesis class C with respect to some arbitrary distribution D, using only a small number of label queries to a polynomial-sized pool of unlabeled examples drawn…
Study on tradeoffs between mistakes and ERM oracle calls in online and transductive learning.
problem Analyzing online and transductive learning with limited ERM and weak consistency oracle access.
method Proves lower bounds and upper bounds on mistakes and oracle calls, considering realizable and agnostic cases.
result Achieves optimal mistake bounds with weak consistency queries for certain concept classes.
Generative AI reduces IR evaluation costs but introduces errors; this work provides reliable CIs.
problem Generating relevance annotations using AI introduces errors that affect IR evaluation metrics.
method Proposes two methods: prediction-powered inference and conformal risk control to place reliable CIs around IR metrics.
result Proposed methods accurately capture both variance and bias in evaluation based on AI-generated annotations.
End-to-end framework learns LLM routing from observational data.
problem Compounding errors in decoupled approaches and reliance on full-feedback data.
method Causal end-to-end framework minimizing decision-making regret from observational data.
result Method outperforms existing baselines across different embedding models.
Solves TOD systems' query annotation problem without explicit annotations.
problem Training TOD systems without explicit KB query annotation.
method Reinforcement learning (RL) and pipelined approach for query prediction and system training.
result Improved RL agent with modifications for TOD tasks.
Study exact community recovery in noisy SBM with limited queries.
problem Community recovery in noisy stochastic block models with limited queries.
method Balanced uniform querying, two-stage adaptive strategy, sublinear queries, subsampled graph.
result Adaptive querying can improve exact recovery limits in noisy SBM.
DP-SPRT improves privacy in sequential tests with near-optimal error rates.
problem Privacy constraints in sequential probability ratio tests.
method A wrapper for SPRT that uses a private mechanism to determine when to stop based on predefined intervals.
result DP-SPRT achieves near-optimal error rates and privacy guarantees.
Proposes a new query autocompletion method that maximizes retrieval performance.
problem Users often select suboptimal queries due to unknown best retrieval performance.
method Formulates query autocompletion as ranking item rankings, uses counterfactual learning.
result Empirical results show improved query suggestions for better retrieval performance.
A new method for private query release using Johnson-Lindenstrauss projection.
problem Private release of query answers with minimal privacy loss.
method Random projection of query answers to a lower dimension, followed by noise addition.
result Optimal worst-case sample complexity for answering a workload of k queries.
LAZO reduces query complexity and variance in ZO methods.
problem High query complexity and variance in zeroth-order optimization.
method LAZO uses adaptive lazy queries to reduce variance and save queries.
result LAZO achieves lower regret and query complexity compared to existing methods.
We consider a query-based data acquisition problem for binary classification of unknown labels, which has diverse applications in communications, crowdsourcing, recommender systems and active learning. To ensure reliable recovery of unknown labels with as few number of queries as possible, we consider an effective quer…
We consider the problem of adaptively PAC-learning a probability distribution P's mode by querying an oracle for information about a sequence of i.i.d. samples X1,X2,… generated from P. We consider two different query models: (a) each query is an index i for which the oracle reveals…
Estimates heavy hitters in data streams with queries, balancing accuracy and efficiency.
problem Identifying elements with high probability in i.i.d. samples.
method Sequential estimation algorithms for two query models: index and pair queries.
result Upper and lower bounds on query complexity for different distributions and noise models.
Survey of statistical queries and their applications.
problem Understanding statistical queries and their applications.
method Exploration of statistical queries model, definitions, and connections to learnability.
result Connections to learnability and applications in optimization, evolvability, and differential privacy.
We study the query complexity of a learner-private sequential learning problem, motivated by the privacy and security concerns due to eavesdropping that arise in practical applications such as pricing and Federated Learning. A learner tries to estimate an unknown scalar value, by sequentially querying an external datab…
Source coding is the canonical problem of data compression in information theory. In a locally encodable source coding, each compressed bit depends on only few bits of the input. In this paper, we show that a recently popular model of semi-supervised clustering is equivalent to locally encodable source coding. In this …
A new query embedding method improves KB performance on complex queries.
problem QE systems disagree with deductive reasoning on some queries.
method A novel QE method more faithful to deductive reasoning.
result Better performance on complex queries to incomplete KBs.
Answering complex logical queries on large-scale incomplete knowledge graphs (KGs) is a fundamental yet challenging task. Recently, a promising approach to this problem has been to embed KG entities as well as the query into a vector space such that entities that answer the query are embedded close to the query. Howeve…
In this paper we study the adaptive learnability of decision trees of depth at most d from membership queries. This has many applications in automated scientific discovery such as drugs development and software update problem. Feldman solves the problem in a randomized polynomial time algorithm that asks $\tilde O(2^…
We study black-box attacks on machine learning classifiers where each query to the model incurs some cost or risk of detection to the adversary. We focus explicitly on minimizing the number of queries as a major objective. Specifically, we consider the problem of attacking machine learning classifiers subject to a budg…
Fairly allocate items with noisy queries, reducing envy.
problem Fairly allocate indivisible goods with unknown valuations.
method Use Gaussian noisy queries to find an envy-free allocation.
result The optimal number of queries scales as \( \frac{m^{2.5}}{Δ^2} \) for large negative-envy.
This paper models the crowdsourced labeling/classification problem as a sparsely encoded source coding problem, where each query answer, regarded as a code bit, is the XOR of a small number of labels, as source information bits. In this paper we leverage the connections between this problem and well-studied codes with …