Extractive compression is a challenging natural language processing problem. This work contributes by formulating neural extractive compression as a parse tree transduction problem, rather than a sequence transduction task. Motivated by this, we introduce a deep neural model for learning structure-to-substructure tree …
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,878 papers · 148 categories
Trend · papers per month
48 results for “neural transduction”
Many machine learning tasks can be expressed as the transformation---or \emph{transduction}---of input sequences into output sequences: speech recognition, machine translation, protein secondary structure prediction and text-to-speech to name but a few. One of the key challenges in sequence transduction is learning to …
Graph neural networks generalize well under certain conditions, explained by learning theory.
problem Understanding why graph neural networks generalize well in transductive inference.
method Analysis of transductive Rademacher complexity to explain generalization properties of graph convolutional networks.
result Transductive Rademacher complexity can explain the generalization of graph convolutional networks for node classification in stochastic block models.
Proposes a framework for compositional generalization in language models.
problem Lack of compositional generalization in neural networks compared to humans.
method Introduces Generalized Grammar Rules (GGRs) for transduction tasks, formalizing symmetry-based constraints.
result Framework enables models to generalize compositionally, similar to human learning.
Proposes a neural framework to select subsets efficiently across different models.
problem Lack of generalizability in subset selection methods for unseen architectures.
method Introduces a trainable subset selection framework, SubSelNet, that uses attention-based neural gadgets and subset samplers.
result SubSelNet generalizes across architectures and outperforms existing methods.
NPGNN improves graph link prediction by adapting to new graphs.
problem Inductive link prediction in graphs with limited training data.
method Meta-learning with graph neural networks (NPGNN).
result NPGNN outperforms state-of-the-art models in real-world graphs.
New method uses graph-based interpolation for few-shot classification.
problem Learning models with limited labeled examples.
method Graph-based feature vector interpolation for transductive learning.
result Significant gains in few-shot classification compared to other methods.
Node Masking improves GNNs' scalability and generalization.
problem Improving GNNs' ability to handle arbitrary graphs.
method Introducing Node Masking to enhance GNNs' performance.
result Node Masking enables GNNs to generalize and scale better.
Enhances few-shot image classification using unlabelled examples.
problem Few-shot image classification with limited labeled data.
method Transductive meta-learning combining soft k-means clustering and neural feature extractor.
result State-of-the-art performance on Meta-Dataset, mini-ImageNet, and tiered-ImageNet benchmarks.
Transductive Generalization via Optimal Transport and Its Application to Graph Node Classificationcs.LG
New bounds improve graph node classification using optimal transport.
problem Improving transductive generalization bounds for graph node classification.
method Representation-based generalization bounds via optimal transport, expressed in terms of Wasserstein distances.
result Strong correlation between derived bounds and empirical generalization in graph node classification.
Training-free GNNs use labels as features to improve node classification.
problem Improving graph neural networks for transductive node classification.
method Advocates labels as features, designs training-free GNNs based on this.
result Training-free GNNs outperform traditional GNNs in node classification.
Boosting theory explains why multi-scale GNNs work.
problem Over-smoothing in graph neural networks.
method Gradient boosting and transductive learning analysis.
result Test error bound decreases with more node aggregations.
GEN tackles few-shot out-of-graph link prediction in evolving multi-relational graphs.
problem Predicting links between unseen nodes in evolving multi-relational graphs with few edges per node.
method Transductive meta-learning framework (GEN) for inductive and transductive inference.
result GEN significantly outperforms relevant baselines for out-of-graph link prediction tasks.
We develop a technique for deriving data-dependent error bounds for transductive learning algorithms based on transductive Rademacher complexity. Our technique is based on a novel general error bound for transduction in terms of transductive Rademacher complexity, together with a novel bounding technique for Rademacher…
The paper establishes bounds for transductive learning using information theory.
problem Transductive learning generalization gap control.
method Information theory, PAC-Bayes, mutual information, conditional mutual information, different information measures.
result Established transductive information-theoretic and PAC-Bayesian bounds.
Local regularization fails in transductive learning for some multiclass problems.
problem Whether local regularization can learn all transductive multiclass problems.
method Provided a negative answer by exhibiting a specific multiclass problem.
result Local regularization cannot learn all transductive multiclass problems.
GCL-LRR improves node classification in noisy graphs.
problem Noise in real-world graph data impairs GNNs' effectiveness.
method Two-stage transductive learning with low-rank regularization and attention.
result Improved node classification performance in noisy graphs.
A new graph neural network tackles oversmoothing and generalization issues.
problem Oversmoothing and poor generalization for unseen graphs in graph neural networks.
method Graph Entities with Step Mixture via random walk (GESM) that considers both edge-based and node-based features.
result GESM achieves state-of-the-art or comparable performances on benchmark datasets.
Improved mistake bounds for transductive online learning.
problem Quantifying the power of unlabeled data in online learning.
method Proving lower and upper bounds on transductive mistake bounds.
result Exponential improvement in mistake bounds for transductive learning.
New method TLC improves transductive learning bounds.
problem Sharp generalization bounds for transductive learning.
method Transductive Local Complexity (TLC) framework.
result Nearly sharp bounds consistent with inductive results.
Transductive learning considers situations when a learner observes labelled training points and unlabelled test points with the final goal of giving correct answers for the test points. This paper introduces a new complexity measure for transductive learning called Permutational Rademacher Complexity (PRC) and …
The paper finds a fundamental trade-off between confidence and efficiency in transductive conformal prediction.
problem The challenge is to balance confidence and efficiency in predicting multiple data points.
method The authors derive a strict finite-sample bound and introduce a practical algorithm to approach this bound.
result Any non-trivial confidence level leads to exponential growth in prediction set size, with a linear scaling in the number of samples.
Transductive learning considers a training set of labeled samples and a test set of unlabeled samples, with the goal of best labeling that particular test set. Conversely, inductive learning considers a training set of labeled samples drawn iid from , with the goal of best labeling any future sample…
Adversarial robust learning improved for transductive setting.
problem Adversarial robust learning in transductive setting.
method Simple transductive learner for bounded VC dimension classes.
result Robust error rate linear in VC dimension, adaptive to perturbation complexity.
DGM learns graph structure for better graph neural network performance.
problem Graphs are often unknown or noisy, limiting graph neural network performance.
method DGM learns graph structure from data, improving performance in transductive and inductive settings.
result DGM achieves state-of-the-art results across various domains.
Unified theory linking node embeddings and graph representations.
problem Clarifying the relationship between node embeddings and graph representations.
method Using invariant theory, the paper establishes a theoretical framework bridging node embeddings and structural graph representations.
result Proves equivalence between node embeddings and structural graph representations, showing they are interchangeable for various tasks.
Graph Attention Networksstat.ML
We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior methods based on graph convolutions or their approximations. By stacking layers in which nodes are able to attend over thei…
Multi-group learners suffer a penalty in transductive learning.
problem The penalty on multi-group learners in transductive learning.
method Analyzing the relationship between the number of groups and the error rate.
result The penalty can increase linearly with the number of groups, up to the square-root of the sample size.
Study shows transductive learning is equivalent to PAC learning for most natural loss functions.
problem Understanding the relationship between transductive and PAC learning models.
method Extending existing results and developing new techniques to analyze the equivalence of the two models.
result Transductive learning is essentially equivalent to PAC learning for realizable learning with most natural loss functions.
New method improves prediction accuracy by using future test points.
problem Improving prediction accuracy by leveraging future test points.
method Transductive inference techniques from semi-parametric inference.
result Non-asymptotic upper bounds on transductive prediction errors.
The paper surveys recent extensions of the Long-Short Term Memory networks to handle tree structures from the perspective of learning non-trivial forms of isomorph structured transductions. It provides a discussion of modern TreeLSTM models, showing the effect of the bias induced by the direction of tree processing. An…
Proposes a transductive matrix completion method with calibration for multi-task learning.
problem Improving multi-task learning with multiple related data sources.
method Transductive matrix completion with calibration constraint.
result The proposed algorithm recovers incomplete feature and target matrices with improved results.
New adversarial training methods generate multiplicative perturbations for robust DNN training.
problem Training Deep Neural Networks with adversarial examples to improve robustness.
method Proposes xAT and xVAT, generating multiplicative perturbations for robust training.
result xAT and xVAT match or outperform state-of-the-art classification accuracies and are faster.
Most traditional online learning algorithms are based on variants of mirror descent or follow-the-leader. In this paper, we present an online algorithm based on a completely different approach, tailored for transductive settings, which combines "random playout" and randomized rounding of loss subgradients. As an applic…
Geom-GCN improves graph neural networks by preserving structural information and capturing long-range dependencies.
problem Weaknesses in MPNNs' aggregators: loss of structural information and lack of long-range dependencies.
method Proposes a geometric aggregation scheme with three modules: node embedding, structural neighborhood, and bi-level aggregation.
result Achieved state-of-the-art performance on various graph datasets.
PAC learning simplified as bipartite matching.
problem Efficiently solving PAC learning problems.
method Transductive learning and one-inclusion graphs.
result PAC learning can be reduced to bipartite matching.
We present transductive Boltzmann machines (TBMs), which firstly achieve transductive learning of the Gibbs distribution. While exact learning of the Gibbs distribution is impossible by the family of existing Boltzmann machines due to combinatorial explosion of the sample space, TBMs overcome the problem by adaptively …
We show two novel concentration inequalities for suprema of empirical processes when sampling without replacement, which both take the variance of the functions into account. While these inequalities may potentially have broad applications in learning theory in general, we exemplify their significance by studying the t…
Develops bounds for deep learning risk via Hilbert coresets.
problem Risk estimation for complex deep learning models.
method Hilbert coreset approach for transductive risk bounds.
result Effective and meaningful bounds for deep neural networks.
TIM maximizes mutual information for few-shot learning, outperforming state-of-the-art methods.
problem Few-shot learning with limited labeled data.
method Transductive Information Maximization (TIM) with alternating-direction solver.
result Significant improvement in accuracy across various datasets and networks.
Supervised learning with large scale labeled datasets and deep layered models has made a paradigm shift in diverse areas in learning and recognition. However, this approach still suffers generalization issues under the presence of a domain shift between the training and the test data distribution. In this regard, unsup…
PARMESAN learns from memory without parameters for fast, efficient continual learning.
problem Inflexibility in deep learning methods for continual learning.
method Transductive reasoning and memory search for parameter-free learning.
result 3-4 orders of magnitude faster than baselines, comparable performance.
GraIL predicts relations by reasoning over subgraphs, outperforming embeddings.
problem Relation prediction in knowledge graphs using latent representations is limited.
method Graph neural network with inductive bias to learn entity-independent relational semantics.
result GraIL outperforms existing rule-induction baselines in the inductive setting.
Improved few-shot learning with LSSVM and transductive modules.
problem Few-shot learning with limited data and samples.
method Introducing LSSVM as a base learner and transductive modules to enhance classification accuracy.
result FSLSTM achieves state-of-the-art performance on miniImageNet and CIFAR-FS benchmarks.
The study finds conditions for compressing the hidden dimension of Graph Transformers for transductive learning.
problem The challenge of efficiently analyzing and training Graph Transformers for transductive learning.
method Theoretical bounds on hidden dimension compression for Graph Transformers, considering both sparse and dense variants.
result Theoretical findings on how and under what conditions the hidden dimension of Graph Transformers can be compressed.
We propose a dynamic neighborhood aggregation (DNA) procedure guided by (multi-head) attention for representation learning on graphs. In contrast to current graph neural networks which follow a simple neighborhood aggregation scheme, our DNA procedure allows for a selective and node-adaptive aggregation of neighboring …
We propose an inductive matrix completion model without using side information. By factorizing the (rating) matrix into the product of low-dimensional latent embeddings of rows (users) and columns (items), a majority of existing matrix completion methods are transductive, since the learned embeddings cannot generalize …
Improved bounds for unbounded losses using transductive priors.
problem Sequential regression and classification with unbounded losses.
method Exponential weights algorithm with transductive priors.
result Statistical bounds independent of design vectors and optimal solution norm.