Revisits orbital minimization for neural operator decomposition.
problem Training neural networks to approximate eigenfunctions of operators.
method Adapts orbital minimization method (OMM) for neural networks.
result Justifies broader applicability of OMM in modern learning pipelines.
A new algorithm speeds up matrix operations in Neural Networks.
problem Time-consuming matrix operations in Neural Networks.
method An algorithm that increases the degree of parallelism of matrix multiplication.
result The algorithm speeds up several matrix operations in Neural Networks.
Paper proposes a new optimization framework for learning eigenfunctions of operators.
problem Computing eigenvalue decomposition of high-dimensional operators.
method Operator SVD with Neural Networks via Nested Low-Rank Approximation.
result Proposed method efficiently learns top-L singular values and functions in the correct order.
RaNNDy uses randomized neural networks to learn transfer operators efficiently.
problem Efficiently learning transfer operators from data.
method Randomized neural network approach with randomly initialized hidden layers and trained output layer.
result Significant reduction in training time and resources with improved stability.
Paper proposes a new method for training diffusion models using Markov operators.
problem Training efficiency and accuracy in diffusion models.
method Operator-informed score matching using spectral decomposition of Markov operators.
result Improved score matching for both low and high-dimensional distributions.
Combining neural networks and multiscale decomposition for financial market analysis.
problem Financial markets' complexity and mainstream models' limitations in capturing non-linear structures.
method Neural networks for non-linear associations combined with multiscale decomposition.
result Improved understanding of financial market data substructures.
SKOLR uses linear RNNs to approximate Koopman operators for time-series forecasting.
problem Nonlinear dynamical system analysis and time-series forecasting with infinite-dimensional Koopman operators.
method Established a connection between Koopman operator approximation and linear RNNs, integrating learnable spectral decomposition and MLP.
result SKOLR delivers exceptional performance in various forecasting benchmarks and dynamical systems.
Novel approach using Lagrangian Decomposition for neural network verification.
problem Computing tight bounds on neural network outputs.
method Lagrangian Decomposition, efficient supergradient ascent, proximal algorithm.
result Proves tighter bounds than previous dual algorithms.
Extends geometric decompositions to arbitrary meshes and forms.
problem Constructing local bases for finite element spaces on arbitrary meshes.
method Generalizes extension operators to arbitrary meshes and forms, showing they yield geometric decompositions.
result Extension operators yield geometric decompositions for arbitrary meshes and forms.
The paper defines and proves stabilization for 3-manifold decompositions with multibranched surface intersections.
problem Decomposing 3-manifolds with more than 3 handlebodies and multibranched surface intersections.
method Definition and proof of stabilization operations for these decompositions.
result Stable equivalence of handlebody decompositions with multibranched surface intersections.
Distributed model training suffers from communication overheads due to frequent gradient updates transmitted between compute nodes. To mitigate these overheads, several studies propose the use of sparsified stochastic gradients. We argue that these are facets of a general sparsification method that can operate on any p…
Geometrically studies Moore-Penrose inverse and polar decomposition continuity.
problem Continuity of Moore-Penrose inverse for perturbations by operator ideals.
method Geometric construction using essential codimension and Banach-Lie group action.
result Moore-Penrose inverse is a real analytic map between manifolds.
HFNO enhances interpretability of turbulent flows through parallel wavenumber bin processing.
problem Opaque inner workings of Fourier Neural Operators (FNOs) hinder physical interpretability.
method Introduces HFNO, a novel FNO-based architecture that processes wavenumber bins in parallel, enhancing interpretability.
result HFNO decomposes turbulent flows across various scales, enabling increased interpretability and multiscale modeling.
We prove an adiabatic decomposition formula of the zeta-determinant of the Laplace type operator with respect to Dirichlet boundary condition. We allow the non-invertible tangential operator. As a result, our adiabatic decomposition formula involves the scattering matrix over the manifold with cylindrical end. We also …
Galerkin method outperforms graph-based methods in spectral decompositions.
problem Improving spectral decomposition methods in machine learning.
method Restricting study to a small set of test functions using the Galerkin method.
result Statistical and computational superiority of Galerkin method over graph-based approaches.
Study finds conditions for operator fields to be in strictly upper triangular form in small dimensions.
problem Jordan-Chevalley decomposition for operator fields in small dimensions.
method Tensorial conditions and proof of conjecture for higher order brackets.
result Proves Tempesta-Tondo conjecture for higher order brackets.
Extends a theorem for first-order elliptic operators on manifolds.
problem Proving the relative index theorem for general first-order elliptic operators.
method Using boundary value problems and graphical decomposition of elliptically regular boundary conditions.
result Proves the relative index theorem for general first-order elliptic operators.
Geometry arising from two diffusion operators (smooth semi-elliptic, second order differential operators) on different spaces but intertwined by a smooth map is described. Particular cases arise from Riemannian submersions when the operators are Laplace-Beltrami operators, from equivariant operators on the total space …
Kernel Dynamic Mode Decomposition reconstructs dynamical systems using Laplacian kernel.
problem Reconstructing spatial-temporal dynamics of complex systems.
method Kernel Dynamic Mode Decomposition with Laplacian kernel.
result Laplacian kernel allows for the closability of Koopman operators in RKHS, enabling reconstruction.
Study shows trained neural networks can overfit without bias or variance issues.
problem Understanding overfitting in trained two-layer ReLU networks.
method Analysis of gradient flow in the neural tangent kernel regime, decomposition of excess risk.
result Trained networks can overfit benignly without bias or variance issues.
Training neural networks is a challenging non-convex optimization problem, and backpropagation or gradient descent can get stuck in spurious local optima. We propose a novel algorithm based on tensor decomposition for guaranteed training of two-layer neural networks. We provide risk bounds for our proposed method, with…
The higher spin Laplace operator has been constructed recently as the generalization of the Laplacian in higher spin theory. This acts on functions taking values in arbitrary irreducible representations of the Spin group. In this paper, we first provide a decomposition of the higher spin Laplace operator in terms of Rr…
we discuss the decomposition of the zeta-determinant of the square of the Dirac operator into the contributions coming from the different parts of the manifold in the case of an invertible tangential operator.
New SPD metrics improve stability and efficiency in neural networks.
problem Designing stable and efficient Riemannian metrics on SPD manifolds.
method Cholesky decomposition to derive SPD metrics.
result Proposed metrics provide closed-form operators, computational efficiency, and improved numerical stability.
Reproducing kernel Hilbert spaces (RKHSs) play an important role in many statistics and machine learning applications ranging from support vector machines to Gaussian processes and kernel embeddings of distributions. Operators acting on such spaces are, for instance, required to embed conditional probability distributi…
The paper classifies decompositions of 3-sphere and lens spaces with handlebodies.
problem Classifying decompositions of 3-manifolds with handlebodies.
method Studied decompositions of 3-sphere and lens spaces with three handlebodies, using stabilizations.
result Determined whether decompositions are stabilized.
Noise-robust Koopman operator framework for control with improved stability and performance.
problem Developing a stable and noise-robust Koopman operator for control tasks.
method Proposes a learning framework using Hankel matrix and neural network approximations for system dynamics, ensuring long-term stability and noise robustness.
result Demonstrates improved model performance and noise robustness in control tasks compared to existing methods.
Biological and artificial neural systems are composed of many local processors, and their capabilities depend upon the transfer function that relates each local processor's outputs to its inputs. This paper uses a recent advance in the foundations of information theory to study the properties of local processors that u…
The paper uses tensor decompositions to improve neural network models for tree data.
problem Encoding structural knowledge from tree-structured data efficiently.
method Introduces new aggregation functions using Canonical and Tensor-Train decompositions.
result Proposed models outperform traditional methods on tree classification tasks.
We consider a general framework for reducing the number of trainable model parameters in deep learning networks by decomposing linear operators as a product of sums of simpler linear operators. Recently proposed deep learning architectures such as CNN, KFC, Dilated CNN, etc. are all subsumed in this framework and we il…
In the present paper we show properties of a little-known Laplacian operator acting on symmetric tensors. This operator is an analogue of the well known Hodge-de Rham Laplacian which acts on exterior differential forms. Moreover, this operator admits the Weitzenböck decomposition and we study it using the analytical me…
Neural-ANOVA breaks down neural networks into simpler models.
problem Understanding complex neural network decision-making processes.
method Formulates a learning problem to decompose neural networks into lower-order models using ANOVA.
result Demonstrates improved approximation properties compared to other regression methods.
Spectral decomposition of the Koopman operator is attracting attention as a tool for the analysis of nonlinear dynamical systems. Dynamic mode decomposition is a popular numerical algorithm for Koopman spectral analysis; however, we often need to prepare nonlinear observables manually according to the underlying dynami…
Researchers develop neural networks for manifold data with a convergence rate.
problem Analyzing high-dimensional data on non-Euclidean domains.
method Constructing manifold neural networks using spectral decomposition of the Laplace Beltrami operator.
result Established a rate of convergence for the neural network scheme that depends on intrinsic manifold dimension.
In this paper, we determine the solitonic decomposition of a Fano toric manifold by computing eigenfunctions of solitonic complex Laplacian operator.
Graph Neural Networks outperform the Weisfeiler-Lehman algorithm in representation power.
problem Limited representation power of Graph Neural Networks compared to the Weisfeiler-Lehman algorithm.
method Algebraic analysis using eigenvalue decomposition of graph operators.
result Graph Neural Networks produce more discriminative representations than the Weisfeiler-Lehman algorithm.
Study on Kähler manifolds proves weak decompositions and relates harmonic forms.
problem Analyzing harmonic forms on Kähler manifolds.
method Proves weak W1,2 Bott-Chern and Dolbeault decompositions. result Strict relation between W1,2 Bott-Chern harmonic forms and the W1,2 Bott-Chern decomposition. Core-Halo solves large-scale fixed-point problems by decentralizing updates.
problem Large-scale fixed-point equations with block dependencies.
method Core-Halo decomposition separates write ownership from read-only context, aligning with block-dependence structure.
result Core-Halo achieves near-centralized performance while retaining parallelism.
MARS automatically selects tensor decomposition ranks, improving performance in neural network tasks.
problem Determining optimal decomposition ranks in tensor decompositions.
method MARS uses binary masks to learn optimal tensor structure during training via relaxed MAP estimation.
result MARS achieves better results than previous methods in various tasks.
Transformer models improve arithmetic accuracy with number decomposition.
problem Transformer models struggle with arithmetic operations without decomposition.
method Fine-tuning models with a pipeline that decomposes numbers into units, tens, etc.
result Accuracy increased by 63% in five-digit addition tasks.
Tensor decomposition methods are widely used for model compression and fast inference in convolutional neural networks (CNNs). Although many decompositions are conceivable, only CP decomposition and a few others have been applied in practice, and no extensive comparisons have been made between available methods. Previo…
Handles decompositions reveal new open book structures.
problem Understanding and manipulating open book structures on manifolds.
method Build handle decompositions and describe handle slides.
result Stabilization of open books to trivial monodromy.
Neural signals are characterized by rich temporal and spatiotemporal dynamics that reflect the organization of cortical networks. Theoretical research has shown how neural networks can operate at different dynamic ranges that correspond to specific types of information processing. Here we present a data analysis framew…
There is a class of Laplacian like conformally invariant differential operators on differential forms Lkℓ which may be considered the generalisation to differential forms of the conformally invariant powers of the Laplacian known as the Paneitz and GJMS operators. On conformally Einstein manifolds we give explic…
We consider ill-posed inverse problems where the forward operator T is unknown, and instead we have access to training data consisting of functions fi and their noisy images Tfi. This is a practically relevant and challenging problem which current methods are able to solve only under strong assumptions on the t…
Graphical notation simplifies tensor operations and decompositions.
problem Complex tensor operations are difficult to understand and represent.
method Introduces graphical notation to represent tensor operations.
result Simplified representation of tensor operations and decompositions.
Proposes neural dynamic mode decomposition for end-to-end modeling of nonlinear dynamics.
problem Understanding and modeling nonlinear dynamical systems.
method Trains neural networks to minimize forecast error based on spectral decomposition in the lifted space.
result Demonstrates effectiveness in eigenvalue estimation and forecast performance.
New DDMs use neural networks for solving equations on manifold shapes.
problem Solving equations on complex, high-dimensional shapes.
method Physics-informed neural networks combined with domain decomposition methods.
result Validated methods work well on various shapes in high dimensions.