Query2box embeds complex queries as boxes to handle logical operations in large KGs.
problem Handling complex logical queries on large-scale incomplete knowledge graphs.
method Embed KG entities and queries into a vector space as boxes, handling conjunctions as intersections and disjunctions through Disjunctive Normal Form.
result Query2box achieves up to 25% relative improvement over state-of-the-art methods.
Study on recovering sparse linear classifiers from mixed binary responses.
problem Learning a mixture of sparse linear classifiers from binary responses.
method Query-based approach to identify all sparse vectors from a set.
result Upper bounds on the number of queries required for recovery.
Optimal DP mechanisms for vector queries are found to be staircase distributions.
problem Designing optimal additive mechanisms for vector-valued queries under differential privacy.
method Reduction to radially symmetric distributions and convex rearrangement theory.
result Staircase mechanisms are optimal for any norm and cost function.
Noisy Max and Sparse Vector are selection algorithms for differential privacy and serve as building blocks for more complex algorithms. In this paper we show that both algorithms can release additional information for free (i.e., at no additional privacy cost). Noisy Max is used to return the approximate maximizer amon…
CwA optimizes search performance by jointly learning a balanced database partition and a neural probing function.
problem Suboptimal search performance due to mismatched database and query distributions.
method CwA jointly learns a balanced database partition and a neural probing function to optimize search performance directly for the query distribution.
result CwA achieves up to 4.7x throughput over state-of-the-art methods at equal recall.
Paper proposes a new pipeline for few-shot classification using forget-update module and channel vector sequence.
problem Few-shot classification with limited support samples.
method Channel vector sequence construction module and forget-update module.
result Pipeline achieves state-of-the-art results on various datasets.
Attention improves edge prediction in e-commerce graphs.
problem Predicting edges in graphs from node attributes in e-commerce.
method Used attention mechanism in simple feedforward networks, derived analytically tractable model AttEST.
result Attention network outperforms LSTM architectures by over 20% on F-1 score.
SOLAR improves search efficiency and accuracy with sparse, orthogonal embeddings.
problem Bottleneck of indexing large dense vectors and NNS for query efficiency and accuracy.
method Proposes SOLAR embeddings: sparse, orthogonal, learned, and random vectors across multiple GPUs.
result Successfully trains 500K dimensional SOLAR embeddings for 1.6M books and multi-label classification.
Personalizes query completion in digital commerce with image-based vectors.
problem High bounce rate and low recurring users in query completion.
method Injects dense vectors from product images into a noisy channel model.
result Image-based personalization improves query completion and user experience.
Path queries on a knowledge graph can be used to answer compositional questions such as "What languages are spoken by people living in Lisbon?". However, knowledge graphs often have missing facts (edges) which disrupts path queries. Recent models for knowledge base completion impute missing facts by embedding knowledge…
This paper improves coreset size via smoothed analysis.
problem Efficiently computing small subsets that approximate query errors.
method Smoothed analysis for approximate average error over queries.
result Deterministic and randomized algorithms for smaller coresets.
ICQ improves high-dimensional similarity search without sacrificing precision.
problem High-dimensional similarity search is computationally expensive.
method Interleaved Composite Quantization (ICQ) reduces code length and quantization error.
result ICQ achieves fast similarity search without using shorter codes.
A new method for private query release using Johnson-Lindenstrauss projection.
problem Private release of query answers with minimal privacy loss.
method Random projection of query answers to a lower dimension, followed by noise addition.
result Optimal worst-case sample complexity for answering a workload of k queries.
This paper addresses the nearest neighbor search problem under inner product similarity and introduces a compact code-based approach. The idea is to approximate a vector using the composition of several elements selected from a source dictionary and to represent this vector by a short code composed of the indices of th…
We propose a quantization based approach for fast approximate Maximum Inner Product Search (MIPS). Each database vector is quantized in multiple subspaces via a set of codebooks, learned directly by minimizing the inner product quantization error. Then, the inner product of a query to a database vector is approximated …
Statistical Query lower bound shows difficulty in list-decodable linear regression.
problem List-decodable linear regression with adversarial corruption.
method Statistical Query (SQ) lower bound analysis.
result Lower bound of dpoly(1/α) for list-decodable linear regression. Improved privacy for heavy-hitters detection.
problem Private detection of heavy-hitters in dynamic data.
method New algorithm for privately identifying heavy-hitters.
result Improved error guarantees over existing techniques.
Gradient flow on softmax attention minimizes nuclear norm of weight matrices.
problem Classification with separate key and query weight matrices.
method Gradient flow on exponential loss, separability assumption, reparameterization, approximate KKT conditions.
result Gradient flow implicitly minimizes nuclear norm of weight matrices, contrasting with Frobenius norm minimization.
This paper investigates and evaluates support vector machine active learning algorithms for use with imbalanced datasets, which commonly arise in many applications such as information extraction applications. Algorithms based on closest-to-hyperplane selection and query-by-committee selection are combined with methods …
New lower bounds for sampling from log-concave distributions in higher dimensions.
problem Proving lower bounds for sampling from log-concave distributions in higher dimensions.
method Multiscale construction inspired by geometric measure theory and reduction to block Krylov algorithms.
result Query lower bounds for sampling from log-concave distributions in higher dimensions are established.
Amortizes MIPS by training neural networks to predict optimal keys.
problem Efficiently solving Maximum Inner Product Search for repeated queries.
method Regression-based SupportNet and KeyNet models trained on support functions.
result Significant improvement in IVF match rates for document embeddings.
Study on recovering supports of multiple sparse vectors from mixed linear measurements.
problem Recovering supports of multiple sparse vectors from a mixture of linear measurements.
method Developed algorithms to identify the support of all component vectors using polynomial and quasi-polynomial number of measurements.
result Polynomial and quasi-polynomial number of measurements sufficient for recovering the supports of all component vectors.
Generative model connects physical properties to latent vectors for solar magnetic patches.
problem Disconnection between generative latent vectors and scientifically relevant quantities.
method Integrating GAN, SVM, and SSL to generate and retrieve physically interpretable solar magnetic patches.
result GAN-SVM combination enables smooth changes in physical parameters with generated patches.
Active learning aims to obtain a classifier of high accuracy by using fewer label requests in comparison to passive learning by selecting effective queries. Many active learning methods have been developed in the past two decades, which sample queries based on informativeness or representativeness of unlabeled data poi…
This study examines a single attention layer's capabilities using random features.
problem Understanding the learning and generalization of a single multi-head attention layer.
method Random feature setting with large number of heads, frozen query and key matrices, and trainable value matrices.
result Random-feature attention layer can express a broad class of permutation-invariant target functions.
Paper tackles binary feedbacks in contextual search learning.
problem Learning underlying mean value function in context with binary feedbacks.
method Tri-section search combined with margin-based active learning.
result Algorithm achieves O(1/ε2) queries for ε-estimation accuracy. Differential privacy provides strong privacy guarantees simultaneously enabling useful insights from sensitive datasets. However, it provides the same level of protection for all elements (individuals and attributes) in the data. There are practical scenarios where some data attributes need more/less protection than ot…
Efficient method for choosing data points in machine learning models.
problem Choosing optimal data points for semi-supervised learning.
method Computing a 'regularity tangent' vector to measure model complexity and influence.
result The method efficiently calculates a measure of model complexity and data point influence.
Real-world networks such as social and communication networks are too large to be observed entirely. Such networks are often partially observed such that network size, network topology, and nodes of the original network are unknown. In this paper we formalize the Adaptive Graph Exploring problem. We assume that we are …
Collaborative filtering (CF) allows the preferences of multiple users to be pooled to make recommendations regarding unseen products. We consider in this paper the problem of online and interactive CF: given the current ratings associated with a user, what queries (new ratings) would most improve the quality of the rec…
We propose an intriguingly simple method for the construction of adversarial images in the black-box setting. In constrast to the white-box scenario, constructing black-box adversarial images has the additional constraint on query budget, and efficient attacks remain an open problem to date. With only the mild assumpti…
Deep Neural Networks, and specifically fully-connected convolutional neural networks are achieving remarkable results across a wide variety of domains. They have been trained to achieve state-of-the-art performance when applied to problems such as speech recognition, image classification, natural language processing an…
Studying a softmax-attention model, we show that the learned query converges to the latent signal subspace spanned by the informative direction.
problem Understanding the theoretical principles of attention mechanisms in large-scale token collections.
method Deriving a population objective and analyzing the limiting ordinary differential equation of the learning dynamics.
result The learned query asymptotically recovers the latent signal up to the intrinsic sign ambiguity.
Active sampling algorithm for linear regression with various norms and improved query complexity.
problem Efficiently querying a few entries of a target vector for near optimal minimizers of linear regression.
method Lewis weight sampling and active sampling algorithms for different p norms. result Optimal query complexity for p∈(0,1), 1<p<2, and 2<p<∞. We prove a \emph{query complexity} lower bound on rank-one principal component analysis (PCA). We consider an oracle model where, given a symmetric matrix M∈Rd×d, an algorithm is allowed to make T \emph{exact} queries of the form w(i)=Mv(i) for i∈{1,…,T}, where v(i)…
Recent state-of-the-art natural language understanding models, such as BERT and XLNet, score a pair of sentences (A and B) using multiple cross-attention operations - a process in which each word in sentence A attends to all words in sentence B and vice versa. As a result, computing the similarity between a query sente…
In recent years, deep metric learning has achieved promising results in learning high dimensional semantic feature embeddings where the spatial relationships of the feature vectors match the visual similarities of the images. Similarity search for images is performed by determining the vectors with the smallest distanc…
Adversarial MoE learns category-specific models for product search.
problem Variations in product features and importance across categories.
method Mixture of Experts with adversarial regularization and soft gating constraints.
result Improved clustering of gate output vectors and shared experts among similar categories.
Improved sampling from high-dimensional Gaussians using smoothed scores.
problem Sampling from high-dimensional Gaussian distributions with gradient information.
method Using smoothed scores, which are gradients of the logarithms of Gaussian-convolved densities, to overcome approximation barriers.
result Improved sampling efficiency with a complexity of \(O\left(\left(\logκ+\log(e\sqrt d/δ_{
m TV})
ight)\log(e\sqrt d/δ_{
m TV})
ight)\) smoothed-score queries.
Solves TOD systems' query annotation problem without explicit annotations.
problem Training TOD systems without explicit KB query annotation.
method Reinforcement learning (RL) and pipelined approach for query prediction and system training.
result Improved RL agent with modifications for TOD tasks.
Algorithm learns latent simplex from perturbed points in input-sparsity time.
problem Learning a latent k-vertex simplex from noisy data. method Input-sparsity time algorithm using low-rank approximation and adaptive selection.
result Algorithm achieves O(extrmnnz(A)) time complexity, avoiding k⋅extrmnnz(A). Meta-learned confidence improves few-shot learning accuracy.
problem Improving accuracy in few-shot learning with unreliable model confidence.
method Meta-learning confidence weights for query samples to improve transductive inference performance.
result Meta-learned confidence leads to new state-of-the-art results on benchmark datasets.
Improves retrieval accuracy for hierarchical documents, especially for distant matches.
problem Limited expressive power of dual encoder models in hierarchical retrieval.
method Proves feasibility of DEs for HR, introduces pretrain-finetune recipe to improve long-distance retrieval.
result Pretrain-finetune boosts recall on long-distance pairs from 19% to 76%.
Study exact community recovery in noisy SBM with limited queries.
problem Community recovery in noisy stochastic block models with limited queries.
method Balanced uniform querying, two-stage adaptive strategy, sublinear queries, subsampled graph.
result Adaptive querying can improve exact recovery limits in noisy SBM.
Efficiently classifies binary labels with XOR queries, even under noisy conditions.
problem Binary classification with unknown labels using XOR queries.
method Effective query type and an efficient inference algorithm for noisy conditions.
result Achieves information-theoretic limit on optimal number of queries.
Neural autoregressive models are explicit density estimators that achieve state-of-the-art likelihoods for generative modeling. The D-dimensional data distribution is factorized into an autoregressive product of one-dimensional conditional distributions according to the chain rule. Data completion is a more involved ta…
Proposes a new query autocompletion method that maximizes retrieval performance.
problem Users often select suboptimal queries due to unknown best retrieval performance.
method Formulates query autocompletion as ranking item rankings, uses counterfactual learning.
result Empirical results show improved query suggestions for better retrieval performance.
Adapts large transformer model for search query intent understanding.
problem Understanding and predicting user intents from search queries.
method Adapts BERT-like architecture for search queries, accounts for query noisiness and sparseness.
result Builds a shareable deep learning model for query intent identification.