LAZO reduces query complexity and variance in ZO methods.
problem High query complexity and variance in zeroth-order optimization.
method LAZO uses adaptive lazy queries to reduce variance and save queries.
result LAZO achieves lower regret and query complexity compared to existing methods.
Lower bounds on queries needed for finding stationary points in non-convex optimization.
problem Finding ε-stationary points in non-convex stochastic optimization. method Proving lower bounds on the number of queries required by stochastic first-order methods.
result Lower bounds on the number of queries required to find ε-stationary points are tight and optimal. In standard passive imitation learning, the goal is to learn a target policy by passively observing full execution trajectories of it. Unfortunately, generating such trajectories can require substantial expert effort and be impractical in some cases. In this paper, we consider active imitation learning with the goal of…
Two novel search strategies reduce complexity for target localization with size-dependent noise.
problem Target localization with varying measurement noise based on query region size.
method Proposes dyaPM and hiePM strategies with low complexity and connected query geometry. result Unified analysis shows dyaPM asymptotically optimal in search time, hiePM near-optimal in rate. Norm-range partition improves MIPS search efficiency by reducing query complexity.
problem Efficiently searching for maximum inner product in large datasets.
method Norm-range partition technique that divides datasets into sub-datasets with similar norms and builds independent hash indexes.
result Significantly reduces the number of probed buckets for LSH-based MIPS algorithms.
Clustered attention improves transformer efficiency for large sequences.
problem Quadratic complexity of transformer attention matrix for large sequences.
method Group queries into clusters, compute attention only for centroids, and use centroids to approximate key/query dot products.
result Linear complexity with respect to sequence length for a fixed number of clusters.
Causal discovery from empirical data is a fundamental problem in many scientific domains. Observational data allows for identifiability only up to Markov equivalence class. In this paper we first propose a polynomial time algorithm for learning the exact correctly-oriented structure of the transitive reduction of any c…
The paper provides an almost optimal learning and testing algorithm for sparse polynomials.
problem Learning and testing sparse multivariate polynomials efficiently.
method The paper presents an algorithm with sublinear query complexity in 1/ε and almost linear in s for learning and testing s-sparse polynomials. result The algorithm achieves almost optimal query complexity, making it the first of its kind.
SNG-DBSCAN clusters data faster with subsampled similarity queries.
problem Efficiently clustering large datasets with DBSCAN's high complexity.
method Subsampled ε-neighborhood graph for similarity queries. result Subsampling 0.1% of the graph leads to 200x speedup and 250x RAM reduction. New bounds show complex neural networks need many queries to learn.
problem Learning non-polynomial activation functions with Gaussian marginals.
method Gradient boosting procedure to amplify lower bounds on SQ dimension of neural networks.
result Statistical-query lower bounds for ReLU regression with 2ncε queries. RadGrad selects expert queries based on agent error and risk to reduce costs.
problem Mitigating covariate shift in sequential decision prediction problems.
method Selective querying based on prediction of agent error and risk.
result Improves upon existing safety-aware algorithms and matches DAgger's performance.
New lower bounds for sampling from log-concave distributions in higher dimensions.
problem Proving lower bounds for sampling from log-concave distributions in higher dimensions.
method Multiscale construction inspired by geometric measure theory and reduction to block Krylov algorithms.
result Query lower bounds for sampling from log-concave distributions in higher dimensions are established.
We prove a \emph{query complexity} lower bound on rank-one principal component analysis (PCA). We consider an oracle model where, given a symmetric matrix M∈Rd×d, an algorithm is allowed to make T \emph{exact} queries of the form w(i)=Mv(i) for i∈{1,…,T}, where v(i)…
New method reduces version space for CNNs, improving active learning performance.
problem Sampling bias in active learning hinders optimal hypothesis finding in neural networks.
method Version space reduction through prior mass reduction and diameter reduction, proposing a new Gibbs-vote disagreement method.
result Diameter-based querying method reduces version space more effectively than prior mass reduction and other methods.
We present a novel active learning algorithm for community detection on networks. Our proposed algorithm uses a Maximal Expected Model Change (MEMC) criterion for querying network nodes label assignments. MEMC detects nodes that maximally change the community assignment likelihood model following a query. Our method is…
Active sampling algorithm for linear regression with various norms and improved query complexity.
problem Efficiently querying a few entries of a target vector for near optimal minimizers of linear regression.
method Lewis weight sampling and active sampling algorithms for different p norms. result Optimal query complexity for p∈(0,1), 1<p<2, and 2<p<∞. Lower bounds on queries for Bayesian private learning.
problem Estimating a target within error ε while keeping it hidden from an adversary.
method Lower bound proof using Fano's inequality and proportional-sampling estimators.
result Query complexity is on the order of Llog(1/ε) for accurate estimation. New methods for sketching non-PSD matrices improve regression and optimization tasks.
problem Efficiently handling non-PSD matrices in computations.
method Developed novel matrix sketching techniques for non-PSD and complex matrices.
result Improved performance in convex and non-convex optimization, regression, and vector-matrix-vector queries.
Study optimizes query strategy for private learning in eavesdropping scenarios.
problem Private sequential learning in the presence of eavesdropping.
method Developed new querying strategies and analytical techniques.
result Proved tight upper and lower bounds on optimal query complexity.
Improved query complexity for adaptive learning of decision trees.
problem Learning decision trees of depth at most d from membership queries.
method Randomized and deterministic polynomial time algorithms with improved query complexity.
result Queries reduced for both randomized and deterministic algorithms.
Derivative-free optimization has become an important technique used in machine learning for optimizing black-box models. To conduct updates without explicitly computing gradient, most current approaches iteratively sample a random search direction from Gaussian distribution and compute the estimated gradient along that…
New algorithms sample structured logconcave families with improved efficiency.
problem Sampling structured logconcave families to high accuracy.
method Reduction framework inspired by proximal point methods, combined with restricted Gaussian oracles.
result Improved bounds for sampling structured distributions, matching or surpassing state-of-the-art results.
A new method for reducing model complexity using neural active manifolds.
problem Uncertainty quantification in computationally expensive models.
method Autoencoders and surrogate models to discover a neural active manifold.
result Neural active manifolds reduce model variance in multifidelity sampling.
A new hypergraph-based active learning scheme reduces query complexity.
problem Efficiently querying and learning from complex hypergraph structures.
method Developed a novel hypergraph-based active learning scheme HS2 that can handle both pointwise and pairwise queries. result Demonstrated that HS2 requires significantly fewer queries than a previously used graph-based method S2. The paper tackles sequential mode estimation with oracle queries.
problem Adaptively PAC-learning a probability distribution's mode.
method Two query models: index queries and pair queries. Sequential algorithms for mode estimation.
result Lower bounds on optimal query complexity for both models.
Estimates heavy hitters in data streams with queries, balancing accuracy and efficiency.
problem Identifying elements with high probability in i.i.d. samples.
method Sequential estimation algorithms for two query models: index and pair queries.
result Upper and lower bounds on query complexity for different distributions and noise models.
In this paper, we consider the convex and non-convex composition problem with the structure n1∑i=1nFi(G(x)), where G(x)=n1∑j=1nGj(x) is the inner function, and Fi(⋅) is the outer function. We explore the variance reduction based met…
Efficient method reduces black-box adversarial queries for neural networks.
problem Solving for adversarial examples in black-box settings with limited query access.
method Efficient combinatorial optimization surrogate for gradient estimation.
result State-of-the-art black-box attack performance with reduced query count.
Query access significantly speeds up learning Multi-Index Models under Gaussian distribution.
problem Agnostically learning Multi-Index Models (MIMs) under Gaussian distribution.
method Query access for MIMs with complexity O(k)poly(1/ε)poly(d) under standard regularity assumptions. result Query access gives significant runtime improvements over random examples for agnostically learning MIMs.
Paper tackles gradient-free minimax optimization with variance reduction for faster convergence.
problem Gradient-free minimax optimization problems in machine learning.
method Variance reduction technique to design a novel zeroth-order gradient descent ascent algorithm.
result Achieves the best known query complexity of O(κ(d₁ + d₂)ε⁻³), outperforming previous methods.
The paper tackles learning smooth distance functions using query-based methods.
problem Learning smooth distance functions under query constraints.
method Global and local approaches using Mahalanobis distance functions.
result Quadratic query complexity for both additive and multiplicative approximations.
Lower bounds on query complexity for reconstructing private learner's training data.
problem Query complexity of reconstructing private learner's training data.
method Minimax analysis, Rényi DP, Metric DP framework.
result First known lower bounds on adversary's query complexity for various DP learners.
Optimal DP mechanisms for vector queries are found to be staircase distributions.
problem Designing optimal additive mechanisms for vector-valued queries under differential privacy.
method Reduction to radially symmetric distributions and convex rearrangement theory.
result Staircase mechanisms are optimal for any norm and cost function.
We formulate a private learning model to study an intrinsic tradeoff between privacy and query complexity in sequential learning. Our model involves a learner who aims to determine a scalar value, v∗, by sequentially querying an external database and receiving binary responses. In the meantime, an adversary observes…
Paper investigates neural query graph ranking for complex question answering over knowledge graphs.
problem Improving neural models for complex question answering over knowledge graphs.
method Experimented with six ranking models, proposed a self-attention based slot matching model.
result Proposed model outperforms other models on DBpedia QA datasets.
Improved online gradient descent with fewer queries for dynamic environments.
problem Efficiently tracking changes in functions with varying gradients and smoothness.
method Developed a new theoretical framework to analyze online gradient descent in dynamic settings, reducing query complexity.
result Achieved state-of-the-art dynamic regret with significantly fewer gradient queries, independent of condition number.
Query2box embeds complex queries as boxes to handle logical operations in large KGs.
problem Handling complex logical queries on large-scale incomplete knowledge graphs.
method Embed KG entities and queries into a vector space as boxes, handling conjunctions as intersections and disjunctions through Disjunctive Normal Form.
result Query2box achieves up to 25% relative improvement over state-of-the-art methods.
New insights into query complexity for Nash equilibrium learning.
problem Characterizing the number of queries needed to learn approximate Nash equilibria in matrix games.
method Introduced a new technique to prove lower bounds on query complexity, improving previous techniques.
result Lower bounds of order Ω(log(Kε1)) for any ε≤1/(cK4), where c is a constant. A new method for private query release using Johnson-Lindenstrauss projection.
problem Private release of query answers with minimal privacy loss.
method Random projection of query answers to a lower dimension, followed by noise addition.
result Optimal worst-case sample complexity for answering a workload of k queries.
Study private query release with public data, reducing sample sizes.
problem Answering a wide range of statistical queries while maintaining privacy.
method Combines public and private samples to answer queries with differential privacy.
result Private and public sample complexities for different query classes.
This paper tackles the computational complexity of finding approximate stationary points in non-convex optimization.
problem Finding approximate stationary points in non-convex optimization problems.
method PLS-completeness, zero-order algorithms, and gradient queries.
result The query complexity of finding approximate stationary points is Θ(1/ε) for d=2.
A new perceptual adjustment query for metric learning reduces complexity in high-dimensional data.
problem Metric learning in high-dimensional data with limited human feedback.
method Inverted measurement scheme and two-stage estimator for PAQs.
result Sample complexity guarantees for the two-stage estimator of metric learning from PAQs.
A new query embedding method improves KB performance on complex queries.
problem QE systems disagree with deductive reasoning on some queries.
method A novel QE method more faithful to deductive reasoning.
result Better performance on complex queries to incomplete KBs.
Meta-ensemble scheme allocates queries to EC nodes for reduced latency.
problem Efficiently allocating queries to EC nodes to minimize latency.
method Combining ensemble models to decide query allocation based on node and query characteristics.
result Meta-ensemble scheme outperforms traditional allocation methods in reducing query processing latency.
Introduces a continuous version of LWE problem.
problem Hardness of learning mixtures of Gaussians.
method Polynomial-time quantum reduction from CLWE to lattice problems.
result CLWE shares hardness with LWE.
New bounds on query learning complexity for various concept classes.
problem Learning complexity in query learning models.
method Introducing new combinatorial quantities and proving lower and upper bounds.
result New and shorter proofs of efficient learnability for prominent examples.
New memory-query tradeoffs for convex optimization algorithms.
problem Optimizing memory usage for convex optimization algorithms.
method Analyzing randomized first-order algorithms for minimizing convex functions.
result Cutting plane methods are optimal in terms of memory and query complexity.
StreamEnsemble dynamically selects ML models for ST data streams to improve predictive accuracy.
problem Predictive queries over spatiotemporal data streams are challenging due to varying distributions and patterns.
method Dynamic selection and allocation of ML models based on time series distributions and characteristics.
result Significantly outperforms traditional ensemble and single model approaches, reducing prediction error by over 10 times.