Tangent automates derivatives in Python, improving expressiveness and performance.
problem Efficiently calculating derivatives for complex models in Python.
method Source-code transformation for dynamically typed array programming.
result Demonstrates improved expressiveness and performance in automatic differentiation.
Study quantifies how LLMs capture higher-order statistical structure using cumulant expansion.
problem Understanding how LLMs internalize statistical structure during next-token prediction.
method Cumulant-expansion framework treating softmax entropy as perturbation around center distribution.
result Cumulants reveal distinct signatures for mathematical vs. general text prompts, quantifying feature-learning dynamics.
We present a system for the automatic differentiation of a higher-order functional array-processing language. The core functional language underlying this system simultaneously supports both source-to-source automatic differentiation and global optimizations such as loop transformations. Thanks to this feature, we demo…
HyperBERT enhances BERT for node classification on text-attributed hypergraphs.
problem Challenges in capturing hypergraph structure and text attributes in node classification.
method Mixing hypergraph-aware layers with BERT for improved node classification.
result HyperBERT achieves state-of-the-art results on text-attributed hypergraph benchmarks.
Myia compiler optimizes ML models with efficient AD for array programming.
problem Efficient automatic differentiation for array programming in ML.
method Introduces a new graph-based IR that supports function calls, higher-order functions, and recursion.
result Myia compiler enables efficient AD using source transformation without a tape, supporting higher-order derivatives.
This work explores higher-order algebroids via vector bundle comorphisms.
problem Generalizing concepts of higher-order tangent bundles and Lie algebroids.
method Introduces a vector bundle comorphism approach to describe higher-order algebroids.
result Establishes a one-to-one correspondence between higher-order Lie algebroids and specific algebraic structures.
Graduate-level introduction to probabilistic programming.
problem Designing and building probabilistic programming systems.
method First-order and higher-order probabilistic programming languages, inference algorithms, and gradient-based maximum likelihood estimation.
result Efficient inference methods and neural network parameterization.
We describe Venture, an interactive virtual machine for probabilistic programming that aims to be sufficiently expressive, extensible, and efficient for general-purpose use. Like Church, probabilistic models and inference problems in Venture are specified via a Turing-complete, higher-order probabilistic language desce…
Transformer models can solve complex math problems with less data.
problem Solving complex symbolic mathematics problems with limited data.
method Pretrain transformer models on language translation tasks and fine-tune for symbolic math.
result Pretrained transformer models achieve comparable accuracy to state-of-the-art models with less data.
We develop a technique for generalising from data in which models are samplers represented as program text. We establish encouraging empirical results that suggest that Markov chain Monte Carlo probabilistic programming inference techniques coupled with higher-order probabilistic programming languages are now sufficien…
Graph Structured Prediction Energy Networks model correlations for joint inference.
problem Joint inference over multiple variables with high-order correlations.
method Energy Networks for modeling explicit local and implicit higher-order correlations.
result Tractable inference with explicit modeling of correlations.
StakeBench evaluates language understanding by linking comments to market commitments, improving model alignment with real-world outcomes.
problem Existing financial NLP benchmarks measure perceived language rather than market commitments.
method StakeBench uses observable market behavior to supervise models, testing their ability to detect commitments, identify sides, and project odds.
result Models partially recover position-side signals but struggle with later tasks, highlighting structural failures.
Infinitesimal variation of Action functional in classical (non-quantum) field theory with higher derivatives is presented in terms of well-defined intrinsic geometric objects independent of the particular field which varies. 'Integration by parts' procedure for this variation is then described in purely formal language…
Study reveals structural differences in financial networks near and far from crises using balance theory.
problem Understanding the complex behavior of stocks and their collective behavior in financial crises.
method Investigates financial networks by triplet interaction in the framework of balance theory, focusing on higher-order interactions.
result Formation of an ordered structure in crisis networks makes them resistant to disorder, with a critical temperature measuring crisis strength.
We study the geometry of jets of submanifolds with special interest in the relationship with the calculus of variations. We give a new proof of the fact that higher order jets of submanifolds are affine bundles; as a by-product we obtain a new expression for the associated vector bundles. We use Green-Vinogradov formul…
Develops a method to control model behavior with synthetic data.
problem Controlling language model behavior via synthetic training data.
method Dataset Policy Gradient (DPG) to optimize synthetic data generators.
result Can cause target models to embed specific patterns or rephrase inputs.
Graphs improve theorem proving in higher-order logic.
problem Challenges in converting higher-order logic formulas into graph-based representations.
method Used graph neural networks (GNNs) to represent and search higher-order logic.
result GNNs outperform state-of-the-art methods in higher-order theorem proving.
Paper introduces techniques to learn higher-order programs, improving predictive accuracy and reducing learning times.
problem Expressing and learning complex programs in ILP.
method Extending meta-interpretive learning to support higher-order definitions as background knowledge.
result Learning higher-order programs reduces hypothesis space and sample complexity, improving predictive accuracy and reducing learning times.
For certain classes of knots we define geometric invariants called higher-order genera. Each of these invariants is a refinement of the slice genus of a knot. We find lower bounds for the higher-order genera in terms of certain von Neumann ρ-invariants, which we call higher-order signatures. The higher-order genera o…
A fundamental property of complex networks is the tendency for edges to cluster. The extent of the clustering is typically quantified by the clustering coefficient, which is the probability that a length-2 path is closed, i.e., induces a triangle in the network. However, higher-order cliques beyond triangles are crucia…
Stability of capillary hypersurfaces with higher order mean curvature.
problem Stability of capillary hypersurfaces with constant higher order mean curvature.
method Generalization of classical stability theory for capillary hypersurfaces.
result Results on stability for capillary hypersurfaces with higher order mean curvature.
New methods for faster ranking and link prediction using higher-order motifs.
problem Real-time ranking and link prediction in applications like web search.
method Higher-order ranking and link prediction methods based on closing higher-order network motifs.
result The methods are faster and more efficient than existing methods based on closing triangles.
The paper improves CR Sobolev inequalities and classifies minimizers.
problem Higher-order CR Sobolev inequalities on the CR sphere.
method Improvement through vanishing higher order moments of the volume element.
result New direct proof of minimizers' classification and existence of minimizers in C2k(N). In this paper we develop a geometric approach to higher order mechanics on graded bundles in both, the Lagrangian and Hamiltonian formalism, via the recently discovered weighted algebroids. We present the corresponding Tulczyjew triple for this higher order situation and derive in this framework the phase equations fro…
The paper glosses different forms of an introducing of higher order tangent-like functors, especially functors derived from higher order nonholonomic tangent functors. A special attention is devoted to higher order osculating bundles: their identification with higher order tangent bundles is demonstrated as the main re…
A new method predicts higher-order interactions in evolving graphs using simplicial complexes.
problem Predicting higher-order interactions in dynamic graphs with theoretical guarantees.
method Capturing higher-order interactions as simplices, modeling neighborhoods with face-vectors, and developing a nonparametric kernel estimator.
result Our method outperforms existing higher-order prediction methods and is theoretically consistent.
Novel higher-order group synchronization for noisy local measurements on hypergraphs.
problem Synchronizing higher-order local measurements on hyperedges to global estimates on nodes.
method Message passing algorithm for global synchronization of higher-order measurements.
result Higher-order method outperforms standard pairwise synchronization methods in certain applications.
Higher-order tangent bundles have geometric structures compatible with their iterated bundle structure.
problem Connection towers and Sasaki metrics on higher-order tangent bundles
method Introduce the notion of a connection tower and study the geometric structures induced by such towers.
result Connection towers determine multiconnections, adapted splittings, and canonical vector bundle structures.
Bilinear MLPs offer a new way to interpret deep learning models without complex nonlinearities.
problem Lack of mechanistic understanding in how MLPs compute.
method Introduced bilinear MLPs without element-wise nonlinearities, analyzed their weights using tensor and eigendecomposition.
result Bilinear MLPs provide interpretable weight structures and enable adversarial attacks and overfitting analysis.
We use the Frölicher-Nijenhuis formalism to reformulate the inverse problem of the calculus of variations for a system of differential equations of order 2k in terms of a semi-basic 1-form of order k. Within this general context, we use the homogeneity proposed by Crampin and Saunders in [14] to formulate and discuss t…
We construct new examples of algebraic curvature tensors so that the Jordan normal form of the higher order Jacobi operator is constant on the Grassmannian of subspaces of type (r,s) in a vector space of signature (p,q). We then use these examples to establish some results concerning higher order Osserman and highe…
A new GCN model learns higher-order neighbors without explicit adjacency matrix computation.
problem GCN's performance drops for deeper structures due to limited neighborhood information.
method Assumes higher-order neighbors are similar to first-order neighbors, learns weights through Lasso to minimize feature loss.
result HWGCN achieves state-of-the-art results on various datasets.
Study on bias-variance trade-off in hierarchical models with higher-order interactions.
problem Understanding the bias-variance trade-off in hierarchical probabilistic models with higher-order interactions.
method Proposed an efficient inference algorithm using Gibbs sampling and annealed importance sampling for log-linear higher-order Boltzmann machine.
result Higher-order interactions produce less variance for smaller sample size and comparable error with hidden layers.
After defining cohomologically higher order BRST and anti-BRST operators for a compact simple algebra {\cal G}, the associated higher order Laplacians are introduced and the corresponding supersymmetry algebra Σ is analysed. These operators act on the states generated by a set of fermionic ghost fields transforming u…
HONEM learns embeddings for higher-order networks, improving performance in various tasks.
problem Existing methods fail to capture non-Markovian higher-order dependencies in networks.
method HONEM is a higher-order network embedding method designed for HON, capturing non-Markovian dependencies.
result HONEM outperforms other methods in node classification, network reconstruction, link prediction, and visualization.
State-of-the-art methods in convex and non-convex optimization employ higher-order derivative information, either implicitly or explicitly. We explore the limitations of higher-order optimization and prove that even for convex optimization, a polynomial dependence on the approximation guarantee and higher-order smoothn…
Introduce Collapsed Effective Operators for higher-order structures.
problem Existing spectral operators decompose topology into separate ranks, leaving practitioners to fuse information back to vertices.
method Introduce Collapsed Effective Operators via Schur complementation of a graded Laplacian.
result Preserves positive semi-definiteness, lowers system energy under higher-order connectivity.
Given a closed, oriented, connected 3-manifold, M, we define higher-order linking forms on the higher-order Alexander modules of M. These higher-order linking forms generalize similar linking forms for knots previously studied by the author, which were themselves generalizations of the classical Blanchfield linking for…
The paper studies higher order Dirichlet-to-Neumann maps on graphs and their eigenvalues.
problem Analyzing eigenvalues of higher order Dirichlet-to-Neumann maps on graphs.
method Introducing and studying higher order Dirichlet-to-Neumann maps on graphs, deriving estimates on eigenvalues.
result Raulot-Savo-type estimates on the eigenvalues of the DtN maps.
The paper connects higher order risk measures and stochastic dominance, showing their equivalence and integrating them with optimization.
problem Comparing and characterizing random outcomes in risk assessment.
method Exploring the equivalence between higher order risk measures and stochastic dominance, using stochastic optimization and expectiles as examples.
result Higher order risk measures and stochastic dominance are equivalent and can be used to characterize random outcomes.
New estimator stabilizes higher-order influence functions for stable statistical inference.
problem Numerical instability in estimating inverse population Gram matrix.
method Proposes a new stabilized higher-order estimator without sample splitting.
result Stabilized estimator exhibits more stable performance and similar statistical guarantees.
New estimator stabilizes higher-order influence functions for bilinear forms.
problem Stability issues in estimating bilinear forms using higher-order influence functions.
method Proposes a new stabilized higher-order estimator for a class of bilinear forms without sample splitting.
result New estimator exhibits more stable finite-sample performance compared to the empirical higher-order estimator.
Enhanced Adam uses higher-order moments for better performance.
problem Improving the performance of Adam optimization algorithm.
method Proposes HAdam, an extension of Adam using higher-order moments of the stochastic gradient.
result Higher-order moments of the stochastic gradient can lead to better performance than vanilla Adam.
We present a geometric approach to the field theory with higher order anisotropic interactions. The concepts of higher order space, or locally anisotropic, space (in brief, h-space, or la-space) are introduced as general ones for various types of higher order extensions of Lagrange and Finsler geometry and higher dimen…
Method provides formal guarantees for decomposing model uncertainty.
problem Decomposing model uncertainty into aleatoric and epistemic components.
method Higher-order calibration using k-snapshots.
result Formal guarantees for aleatoric uncertainty matching real-world distribution.
Develops higher-order Euler-Poincaré field equations for principal G-bundles.
problem Formulating field equations for higher-order jet bundles of principal G-bundles.
method Reduction theory applied to G-invariant Lagrangian field theories on jet bundles, transferring Hamilton's principle to reduced configuration bundles. result Higher-order Euler-Poincaré field equations are equivalent to conservation of Noether current.
This paper describes a general framework for learning Higher-Order Network Embeddings (HONE) from graph data based on network motifs. The HONE framework is highly expressive and flexible with many interchangeable components. The experimental results demonstrate the effectiveness of learning higher-order network represe…
We compare two ways of interpreting higher order connections. The geometric approach lies in the decomposition of higher order tangent space into the horizontal and vertical structures while the jet--like approach considers a higher order connection as the section of a jet prolongation of a fibered manifold. Particular…