New method improves consistency in preference learning for neural networks.
problem Inconsistent surrogate losses in preference learning for neural networks.
method Formulated a margin-shifted ranking framework and introduced Structure-Aware H-consistency. result Proved superior consistency guarantees for capacity-bounded models using heavy-tailed surrogates.
Transformer learns graph structure better with subgraph info.
problem Transformer struggles with structural similarity in graph learning.
method Structure-Aware Transformer with subgraph attention.
result Improves graph prediction benchmarks significantly.
Graph Cascades rewire graphs to improve structure-aware learning.
problem Improving graph neural networks and transformers for structure-aware learning.
method Graph Cascades uses contagion-based diffusion processes to construct an auxiliary graph with reinforced edges.
result Graph Cascades improves node-classification benchmarks across various graph types.
dGAP learns feature dependencies and predicts targets simultaneously.
problem Learning task-agnostic statistical dependencies and missing explicit feature dependencies.
method Jointly optimizes a neural dependency graph and target prediction loss.
result dGAP can recover correct feature dependencies and improve prediction accuracy.
AlphaSAGE mines diverse alphas via GFlowNets, overcoming RL issues.
problem Reward sparsity, inadequate sequential representations, and single optimal mode issues in RL for alphas.
method Structure-aware encoder (RGCN), GFlowNets, dense reward structure.
result Empirically outperforms existing baselines in mining diverse alphas.
SANS uses graph structure to find meaningful negatives for entity and relation embeddings.
problem Finding hard negatives for entity and relation embeddings in knowledge graphs.
method Structure Aware Negative Sampling (SANS) that selects negatives from a node's k-hop neighborhood.
result SANS finds semantically meaningful negatives and is competitive with state-of-the-art approaches.
Graph neural networks benefit from a new initialization method that improves node learning.
problem Poor initialization in GNNs leads to slower convergence and increased training instability.
method Integrates a statistically grounded one-hot graph encoder embedding (GEE) into standard GNNs.
result GG framework provides consistent and substantial performance gains in node classification.
TableQnA answers web queries about lists and superlatives from HTML tables.
problem Answer web queries about lists and superlatives from HTML tables.
method Extract intent from queries, use structure-aware matching, and train models with automatic data generation.
result Significantly higher precision and coverage for list and superlative queries.
Develops structured noise for more accurate graph classifier robustness certificates.
problem Isotropic noise limits robustness certificates for graph classifiers.
method Randomized smoothing with anisotropic noise distribution.
result Structured-aware robustness certificates provide more accurate predictions.
ROTS improves sentence similarity by incorporating structural information.
problem Measuring sentence similarity with theoretical insights and structural awareness.
method Recursive Optimal Transport (ROT) framework to incorporate structural information.
result ROTS outperforms weakly supervised approaches in sentence similarity tasks.
CASPER improves DAG structure learning by integrating graph structure into score function.
problem Discovering suboptimal DAGs and model vulnerabilities in causal discovery.
method CASPER integrates graph structure into the score function as a new measure in the causal space, enhancing DAG structure learning via adaptive attention to DAG-ness.
result CASPER outperforms state-of-the-art methods in terms of accuracy and robustness.
In this paper, we propose a theory which unifies kernel learning and symbolic algebraic methods. We show that both worlds are inherently dual to each other, and we use this duality to combine the structure-awareness of algebraic methods with the efficiency and generality of kernels. The main idea lies in relating polyn…
GraphSAIL updates GNN-based recommender models incrementally to reduce computation time and improve frequent updates.
problem Incremental updates in GNN-based recommender systems are computationally expensive and prone to forgetting.
method GraphSAIL uses a graph structure preservation strategy to update GNN models incrementally, preserving long-term preferences and properties.
result GraphSAIL reduces computation time and improves frequent updates compared to other incremental learning techniques.
A new hybrid GNN framework tackles oversmoothing in graph data.
problem Oversmoothing in graph convolutional networks limits their expressive power and generalization.
method Combines traditional GCN filters with band-pass filters defined via geometric scattering and introduces an attention framework.
result Improves expressive power and generalization of graph convolutional networks.
ASAP improves graph pooling for hierarchical graph representations.
problem Pooling in graphs fails to effectively capture substructure or scale to large graphs.
method ASAP uses self-attention and modified GNN to capture node importance and learn sparse soft cluster assignments.
result Combining ASAP with GNN architectures leads to state-of-the-art results on graph classification benchmarks.
Clustering is fundamental for gaining insights from complex networks, and spectral clustering (SC) is a popular approach. Conventional SC focuses on second-order structures (e.g., edges connecting two nodes) without direct consideration of higher-order structures (e.g., triangles and cliques). This has motivated SC ext…
PolarGrad optimizes deep learning models by considering matrix structure, outperforming Adam and Muon.
problem Efficient optimization of large-scale neural networks and language models.
method A unifying framework for analyzing matrix-aware preconditioned methods, including PolarGrad.
result PolarGrad outperforms Adam and Muon in various tasks.
Quantum state preparation framework speeds up basket option pricing.
problem Limited practical benefit of quantum amplitude estimation due to state-preparation depth.
method Structure-aware tensor-train rank-based variational state preparation.
result State-preparation depth scaling replaced with linear scaling, maintaining low basket-pricing errors.
Optimized variable orderings improve autoregressive model performance.
problem Challenges in variable ordering affect autoregressive model efficiency.
method Learn graphical model structure to inform optimal variable orderings.
result Graph-informed orderings yield higher-fidelity samples.
Flexible Kernels for Protein Property Prediction
problem Predicting protein properties from sparse experimental data
method Sequence kernels using evolutionary substitution matrices and local linearity
result Data-efficient models of protein property landscapes
A RL algorithm learns optimal multi-threshold policies for MDPs.
problem Overcoming the curse of dimensionality in MDPs.
method Structure-aware RL algorithm exploiting multi-threshold optimal policies.
result The algorithm converges to the optimal policy asymptotically.
Recent deep learning approaches for representation learning on graphs follow a neighborhood aggregation procedure. We analyze some important properties of these models, and propose a strategy to overcome those. In particular, the range of "neighboring" nodes that a node's representation draws from strongly depends on t…
Preconditioned gradient methods are among the most general and powerful tools in optimization. However, preconditioning requires storing and manipulating prohibitively large matrices. We describe and analyze a new structure-aware preconditioning algorithm, called Shampoo, for stochastic optimization over tensor spaces.…
To overcome the curse of dimensionality and curse of modeling in Dynamic Programming (DP) methods for solving classical Markov Decision Process (MDP) problems, Reinforcement Learning (RL) algorithms are popular. In this paper, we consider an infinite-horizon average reward MDP problem and prove the optimality of the th…
Dual-Channel Tensor Neural Network (DC-TNN) decomposes tensor data into low-rank and sparse components for better estimation and inference.
problem Tensor-valued data with multilinear dependencies are challenging to process due to loss of multiway geometry under vectorization.
method DC-TNN decomposes tensors into a low-rank core and a sparse refinement, processing them through coupled neural channels.
result Established non-asymptotic risk bounds and developed structure-aware conformal ROC and AUC confidence bands.
EdgeFool generates adversarial images to mislead classifiers.
problem Misleading classifiers with adversarial images.
method Trains a fully convolutional neural network to generate perturbations that enhance image details and mislead classifiers.
result EdgeFool outperforms other adversarial methods on various classifiers and datasets.
HCL learns shared and modality-specific latent representations for multimodal data.
problem Binary shared-private decomposition inadequately represents shared information across subsets of modalities.
method Hierarchical Contrastive Learning framework combining latent-variable formulation, structural sparsity, and contrastive objective.
result HCL accurately recovers hierarchical structure and improves predictive performance on multimodal data.
Study uses deep learning to predict stock trends with superior performance.
problem Predicting short-term equity trends with high accuracy.
method Dual-task multilayer perceptron (MLP) integrating technical signals and deep learning.
result Deep learning model outperforms linear baselines in multi-factor stock selection.
DBGAN learns graph node representations by balancing distribution consistency.
problem Graph representation learning overfits due to ignoring data distribution.
method DBGAN uses a structure-aware prior distribution and bidirectional adversarial learning.
result DBGAN achieves better trade-off between robustness and dimensionality.
Proposes a graph pooling method leveraging node proximity for hierarchical graph representation learning.
problem Efficiently exploiting the geometry of graph data for hierarchical representation learning.
method Combines node proximity with kernel representation of topology and node features for adaptive node signal similarities evaluation.
result Achieves state-of-the-art performance on graph classification benchmark datasets.
Training large machine learning (ML) models with many variables or parameters can take a long time if one employs sequential procedures even with stochastic updates. A natural solution is to turn to distributed computing on a cluster; however, naive, unstructured parallelization of ML algorithms does not usually lead t…
Generative model for hypergraphs captures complex interactions without pairwise reductions.
problem Challenges in generating realistic hypergraphs with pairwise reductions.
method Structured stochastic diffusion on relaxed incidence matrices.
result Generative model preserves structure-aware noising and yields explicit Gaussian law.
Survey explores cohomology's roles in applied math and sciences.
problem Understanding cohomology's role in solving differential equations.
method Examining differential complexes and structure-preserving discretizations.
result Various fundamental concepts in mechanics are formulated using differential complexes.
CP-DRL improves curriculum reinforcement learning by leveraging causal relationships.
problem Designing effective task sequences for reinforcement learning.
method Causal-Paced Deep Reinforcement Learning (CP-DRL) that approximates SCM differences based on interaction data.
result CP-DRL outperforms existing methods on benchmarks, achieving faster convergence and higher returns.
The goal of graph representation learning is to embed each vertex in a graph into a low-dimensional vector space. Existing graph representation learning methods can be classified into two categories: generative models that learn the underlying connectivity distribution in the graph, and discriminative models that predi…
It is important to learn various types of classifiers given training data with noisy labels. Noisy labels, in the most popular noise model hitherto, are corrupted from ground-truth labels by an unknown noise transition matrix. Thus, by estimating this matrix, classifiers can escape from overfitting those noisy labels. …
NeuroMAS treats multi-agent systems as neural networks for scalable, trainable coordination.
problem Designing multi-agent systems as hand-designed workflows is inefficient and inflexible.
method NeuroMAS treats multi-agent systems as a neural network architecture with reinforcement learning for scalable coordination.
result NeuroMAS improves significantly over multi-agent baselines and can be scaled progressively.
New method sparsifies hybrid neural ODEs for better performance and stability.
problem Excessive latent states and interactions from mechanistic models lead to training inefficiency and over-fitting.
method Automatic state selection and structure optimization combining domain-informed graph modifications with data-driven regularization.
result Improved predictive performance and robustness with desired sparsity.
Paper proposes a new method for joint feature selection and graph learning.
problem Previous methods suffer from neglecting joint formulation and lack of graph learning.
method Formulates multi-view feature selection with orthogonal decomposition, incorporates cross-space locality preservation, and uses a unified objective function for simultaneous learning.
result Demonstrates superior performance in multi-view feature selection and graph learning tasks.
Unified PAC-Bayesian framework for deep learning generalization.
problem Limitations of existing PAC-Bayesian norm-based bounds for deep neural networks.
method Unified framework using anisotropic Gaussian posteriors and sensitivity matrix.
result Comparable or tighter generalization bounds compared to state-of-the-art approaches.
AniDS improves molecular force field modeling by learning anisotropic noise.
problem Molecular force field modeling suffers from oversimplified assumptions about atomic motions.
method AniDS introduces anisotropic noise generation for better modeling of directional and structural variability.
result AniDS outperforms existing methods on benchmarks, achieving significant improvements in force prediction accuracy.
Unified framework for clustering and learning causal graphs across subjects.
problem Bias and obscured subpopulation-specific dependencies in multivariate systems.
method Directed Acyclic Graph-based Dependency Clustering via Alternating Direction Method of Multipliers (DAG-DC-ADMM) integrated with Structural Equation Modeling (SEM).
result Unified framework recovers cluster-specific causal dependency structures with high true positive rate and low false discovery rate.
SCNode improves node embeddings for GNNs in both homophilic and heterophilic graphs.
problem Challenges in node representation quality and generalization in GNNs, especially in heterophilic graphs.
method SCNode integrates spatial and contextual information to create more discriminative and structurally aware node embeddings.
result SCNode achieves superior performance over conventional GNN models on benchmark datasets.
A method to generate long-range human actions by leveraging graph convolutional networks and self-attention.
problem Generating long-range skeleton-based human actions is challenging due to small frame deviations.
method Proposes a variant of GCNs with self-attention to adaptively sparsify action graphs and capture structure information.
result Extensive experiments show superior performance compared to existing methods on human action datasets.
Enhances LDL by integrating distance and directional information for more robust label feature representation.
problem Lack of robust label feature representation in LDL tasks, especially with label ambiguity.
method Introduces Structural Anchor Points (SAPs) to capture inter-cluster interactions and a novel LSFs construction strategy, LIFT-SAP.
result Improves LDL performance by 15% on average across 15 real-world datasets.
Scattering GCN improves graph neural networks by filtering oversmoothing.
problem Oversmoothing in GCNs limits their ability to distinguish graph nodes.
method Augmenting GCNs with geometric scattering transforms and residual convolutions.
result Scattering GCN outperforms GAT in semi-supervised node classification.
Gradient descent converges geometrically to optimal self-attention parameters.
problem Training softmax self-attention layers for linear regression.
method Structure-aware gradient descent with preconditioner and regularizer.
result Gradient descent converges geometrically to global minima.
Solves parameter non-identifiability in Bayesian LTI system identification.
problem Parameter non-identifiability in standard Bayesian approaches for LTI system identification.
method Embedding canonical forms of LTI systems within the Bayesian framework.
result Unlocking the use of meaningful priors and robust uncertainty estimates.