Designers of AI agents often iterate on the reward function in a trial-and-error process until they get the desired behavior, but this only guarantees good behavior in the training environment. We propose structuring this process as a series of queries asking the user to compare between different reward functions. Thus…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “query design”
Efficient Preference-Based Reinforcement Learning: Randomized Exploration Meets Experimental Designcs.LG
Meta-algorithm for efficient reinforcement learning from human preferences.
problem Learning from human preference comparisons in Markov decision processes.
method Randomized exploration and experimental design for batch comparison queries.
result Meta-algorithm achieves both regret and last-iterate guarantees with minimal preference queries.
Adaptive querying learns user psychometrics with AI personas.
problem Learning user psychometrics within query budgets.
method Persona-induced latent variable model with AI personas and large language model response distributions.
result Persona-based posteriors deliver accurate probabilistic predictions.
CUBE explains models by balanced experiments and contrasts.
problem Post-hoc explanation of trained predictive models.
method Design-based framework using balanced low-high probes.
result Reveals dominant learned effect structure and clarifies query efficiency.
Optimizes reward learning design for complex tasks using nonparametric methods.
problem Challenges in specifying reward functions for complex tasks.
method Models rewards and policies as nonparametric functions in RKHSs, derives risk bounds, and optimizes query design.
result Derives non-asymptotic excess risk bounds and finite sample statistical rates for reward learning.
Efficient algorithms recover two sparse models from a mix of linear queries.
problem Recovering two sparse models from a mix of linear queries.
method Efficient algorithms for query complexity problem.
result Improved query complexity for model recovery.
Optimal DP mechanisms for vector queries are found to be staircase distributions.
problem Designing optimal additive mechanisms for vector-valued queries under differential privacy.
method Reduction to radially symmetric distributions and convex rearrangement theory.
result Staircase mechanisms are optimal for any norm and cost function.
In this paper, we initiate a rigorous theoretical study of clustering with noisy queries (or a faulty oracle). Given a set of elements, our goal is to recover the true clustering by asking minimum number of pairwise queries to an oracle. Oracle can answer queries of the form : "do elements and belong to the…
We propose a hypergraph-based active learning scheme which we term , generalizes the previously reported algorithm originally proposed for graph-based active learning with pointwise queries [Dasarathy et al., COLT 2015]. Our method can accommodate hypergraph structures and allows one to ask bo…
The paper tests properties of trees in graphical models using covariance queries.
problem Testing properties of trees in graphical models.
method Covariance queries model, randomized tests for tree properties.
result Efficient testing of global tree properties using sub-quadratic number of queries.
Differential privacy mechanism design has traditionally been tailored for a scalar-valued query function. Although many mechanisms such as the Laplace and Gaussian mechanisms can be extended to a matrix-valued query function by adding i.i.d. noise to each element of the matrix, this method is often suboptimal as it for…
This work tackles model-based RL by optimizing state-action queries to learn policies with minimal data.
problem Expensive state transitions in practical RL problems limit the use of standard RL algorithms.
method Bayesian optimal experimental design to guide selection of state-action queries.
result Data-efficient RL approach that learns optimal policies with up to 1,000x less data.
The paper tackles machine unlearning by designing efficient algorithms for adaptive query classes.
problem Designing efficient unlearning algorithms for machine learning models.
method Formalizes the problem and gives efficient unlearning algorithms for linear and prefix-sum query classes.
result Improved guarantees for stochastic convex optimization with reduced unlearning query complexity.
Study on how many queries an adversary needs to make to match the best attack in white-box model.
problem Query complexity of adversarial attacks in machine learning models.
method Investigates the number of queries required for an adversary to match the best possible attack in the white-box model, using entropy of decision boundaries as a measure.
result Lower bound on the number of queries required for an adversary to match the best possible attack in the white-box model.
Estimates heavy hitters in data streams with queries, balancing accuracy and efficiency.
problem Identifying elements with high probability in i.i.d. samples.
method Sequential estimation algorithms for two query models: index and pair queries.
result Upper and lower bounds on query complexity for different distributions and noise models.
We study the query complexity of a learner-private sequential learning problem, motivated by the privacy and security concerns due to eavesdropping that arise in practical applications such as pricing and Federated Learning. A learner tries to estimate an unknown scalar value, by sequentially querying an external datab…
New attacks reduce bad queries in black-box classifiers, improving effectiveness.
problem Evasion attacks often issue many bad queries, which are costly.
method Design new attacks that minimize bad queries while increasing non-bad queries.
result Reduces the number of bad queries by 1.5-7.3 times, improving attack effectiveness.
Platform uses queries to elicit investor preferences for portfolio trades, improving allocation efficiency.
problem Hidden-information problem in institutional crossing markets where investors value trades as portfolios but liquidity discovery is organized by individual securities.
method Modeling portfolio crossing as preference elicitation, using price-directed demand queries and value queries to verify selected packages.
result Hybrid procedure using demand and value queries recovers 88-95% of full-information welfare with a limited query budget.
New mechanisms improve differential privacy for scalar queries.
problem Improving differential privacy for scalar, real-valued query functions.
method Mixing multiple Gaussian distributions to satisfy differential privacy.
result Mechanisms yield lower noise amplitudes and variances compared to the analytic Gaussian mechanism.
Algorithm identifies nearest mode in noisy data.
problem Identifying the point with the minimum k-th nearest neighbor distance in unknown multivariate probability density.
method Sequential learning algorithm using noisy oracle queries to adaptively decide which points to query.
result Upper bounds on query complexity show significant improvement over baselines.
The paper bridges theory and practice in query-driven selectivity learning.
problem Insufficient theoretical understanding of query-driven selectivity learning.
method Demonstrates learnability of selectivity predictors and establishes favorable OOD generalization error bounds.
result Theoretical advances improve OOD generalization of query-driven selectivity models.
In many high-throughput experimental design settings, such as those common in biochemical engineering, batched queries are more cost effective than one-by-one sequential queries. Furthermore, it is often not possible to directly choose items to query. Instead, the experimenter specifies a set of constraints that genera…
Crowdsourced labeling recovers task types with minimal queries.
problem Labeling tasks accurately with minimal queries.
method Worker clustering, skill estimation, weighted majority voting.
result Achieves any targeted recovery accuracy with minimum queries.
Unified query framework for active metric learning and classification.
problem Combining representation learning and task-specific goals in machine learning.
method Adaptive selection of nearest neighbor queries using information theoretic criterion.
result Actively selected nearest neighbor queries outperform recent methods in deep metric learning and classification.
New algorithms recover clusters with minimal queries, connecting margins to recoverability.
problem Active cluster recovery with oracle queries for minimal cost.
method Introducing margin-based clustering, designing algorithms for various spaces.
result Achieve queries for general pseudometric spaces and convex clusters.
Study efficient sequential evaluation of large language models using historical data.
problem Sequentially evaluate a new large language model (LLM) on a fixed question set.
method Construct a confidence sequence (CS) and design active querying rules to shrink CS width.
result Simple uniform sampling can sometimes outperform adaptive querying rules.
ALINE optimizes Bayesian inference and data acquisition by strategically querying informative data.
problem Strategic acquisition of informative data for Bayesian inference in challenging tasks.
method Unified framework combining amortized Bayesian inference and active data acquisition using a transformer architecture trained via reinforcement learning.
result ALINE delivers both instant and accurate inference along with efficient selection of informative points.
Marich: A Query-efficient Distributionally Equivalent Model Extraction Attack using Public Datacs.LG
Marich extracts high-fidelity models from public data with minimal queries.
problem Creating an accurate replica of a target ML model using few queries.
method Sequentially selects informative queries to maximize entropy and reduce model mismatch.
result Extracted models achieve 60-95% of target model's accuracy with 1,000-8,500 queries.
A new attack for probabilistic classifiers adapts to noise levels.
problem Adversarial examples can mislead probabilistic classifiers.
method Adapts HopSkipJump attack for probabilistic classifiers, adjusting queries based on noise levels.
result Decision-based attacks are effective against probabilistic classifiers, even with noise.
Paper proposes efficient RLHF methods for LLMs using active queries.
problem Efficiently aligning LLMs with human preferences using RLHF.
method Formalizes RLHF as a dueling bandit problem, introduces APPO and ADPO algorithms.
result ADPO achieves similar performance to state-of-the-art methods with fewer queries.
New method tackles geodesically convex optimization with polynomial convergence.
problem Designing an efficient algorithm for geodesically convex optimization.
method Ellipsoid-like algorithm with polynomial query and per-query complexity.
result Achieves polynomial convergence for geodesically convex functions.
Algorithm learns diverse rankings for search engines.
problem Designing algorithms for search engines to rank diverse items.
method LDR (Learning Diverse Rankings) algorithm, efficient learning based on users' feedback.
result Algorithm achieves optimal ranking performance with regret.
Many real-world problems can be formalized as predicting links in a partially observed network. Examples include Facebook friendship suggestions, consumer-product recommendations, and the identification of hidden interactions between actors in a crime network. Several link prediction algorithms, notably those recently …
Designs an online selective sampling approach for choosing which model to use.
problem Active model selection for pre-trained classifiers in unlabeled data streams.
method Online selective sampling approach to query and label examples.
result High probability of outputting the best model with minimal label queries.
VACA models graph data for causal inference without hidden confounders.
problem Causal inference in observational data with hidden confounders.
method Variational graph autoencoders for structural causal models.
result Accurately approximates interventional and counterfactual distributions.
GO-CBED optimizes experiments for specific causal queries, improving efficiency.
problem Efficiently infer causal relationships with limited resources.
method Goal-oriented Bayesian framework that maximizes expected information gain on user-specified causal quantities.
result GO-CBED outperforms existing methods in various causal tasks, especially with limited budgets.
Extends V-IP framework to use LLMs for generating task-relevant concepts, improving interpretability and performance.
problem Limited applicability of V-IP to small-scale tasks due to manual data annotation.
method Integrates Foundational Models with Large Language and Multimodal Models to generate and annotate concepts.
result FM+V-IP achieves better test performance with fewer concepts/queries compared to other frameworks.
New method recovers clusters in non-convex finite metric spaces with oracle queries.
problem Exact recovery of clusters in non-convex finite metric spaces.
method Introducing -convexity and a deterministic algorithm using oracle queries.
result Clusters can be recovered using same-cluster queries.
New tool for parallel and private stochastic convex optimization reduces query complexity.
problem Parallel and private stochastic convex optimization with reduced query complexity.
method Reweighted Stochastic Query (ReSQue) estimator combined with ball oracle acceleration.
result Achieves state-of-the-art complexities for SCO in parallel and private settings.
We formalize the notion of nesting probabilistic programming queries and investigate the resulting statistical implications. We demonstrate that while query nesting allows the definition of models which could not otherwise be expressed, such as those involving agents reasoning about other agents, existing systems take …
MICO uses mutual information co-training to improve selective search efficiency.
problem Efficiently search and route unseen queries in large-scale search systems.
method Mutual Information Co-training framework for selective search with minimal supervision.
result Significantly improves performance on multiple metrics of selective search.
CwA optimizes search performance by jointly learning a balanced database partition and a neural probing function.
problem Suboptimal search performance due to mismatched database and query distributions.
method CwA jointly learns a balanced database partition and a neural probing function to optimize search performance directly for the query distribution.
result CwA achieves up to 4.7x throughput over state-of-the-art methods at equal recall.
This paper explores the adaptive (active) PAC (probably approximately correct) top- ranking (i.e., top- item selection) and total ranking problems from -wise () comparisons under the multinomial logit (MNL) model. By adaptively choosing sets to query and observing the noisy output of the most favored …
Exact cluster recovery with same-cluster queries for arbitrary ellipsoidal clusters.
problem Recovering clusters from same-cluster queries in arbitrary ellipsoidal clusters.
method Relaxing spherical -means assumption to arbitrary ellipsoidal clusters, designing an algorithm with logarithmic query complexity.
result Exact recovery of clusters using queries and time.
Despite the great achievements of the modern deep neural networks (DNNs), the vulnerability/robustness of state-of-the-art DNNs raises security concerns in many application domains requiring high reliability. Various adversarial attacks are proposed to sabotage the learning performance of DNN models. Among those, the b…
Bayesian framework learns latent preference archetypes for many-objective optimization.
problem Expanding space of trade-offs and context-dependent human values.
method Dirichlet-process mixture model for latent preference archetypes, hybrid queries for efficient information.
result Mixture-aware Bayesian optimization outperforms standard methods on synthetic and real-world benchmarks.
AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networkscs.CV
Recent studies have shown that adversarial examples in state-of-the-art image classifiers trained by deep neural networks (DNN) can be easily generated when the target model is transparent to an attacker, known as the white-box setting. However, when attacking a deployed machine learning service, one can only acquire t…
We revisit the problem of differentially private release of classification queries. In this problem, the goal is to design an algorithm that can accurately answer a sequence of classification queries based on a private training set while ensuring differential privacy. We formally study this problem in the agnostic PAC …