A new algorithm speeds up matrix operations in Neural Networks.
problem Time-consuming matrix operations in Neural Networks.
method An algorithm that increases the degree of parallelism of matrix multiplication.
result The algorithm speeds up several matrix operations in Neural Networks.
We recover matrix networks from incomplete observations using low-rank graph Fourier transform.
problem Recovering a partially observed matrix network from incomplete observations.
method We propose a convex optimization problem with a structural assumption of low-rank graph Fourier transform. We prove an exact recovery guarantee and provide an iterative imputation algorithm.
result We discover a new phase transition phenomenon and demonstrate the algorithm's effectiveness on large-scale matrix networks.
DeepTMR reorders matrices without prior knowledge of structural patterns.
problem Matrix reordering without prior structural knowledge.
method DeepTMR uses a neural network to automatically extract features and reorder matrices.
result Trained network produces denoised mean matrix for visualization.
NetSMF efficiently embeds large networks by sparse matrix factorization.
problem Learning latent representations for large-scale networks efficiently.
method NetSMF leverages spectral sparsification to efficiently sparsify and factorize a dense matrix.
result NetSMF achieves high efficiency and effectiveness on large-scale networks.
New neural networks for non-commutative data.
problem No existing neural networks suitable for non-commutative data.
method Developed compact matrix quantum group equivariant neural networks.
result Characterized weight matrices for easy compact matrix quantum groups.
Enhances network embedding with auxiliary info using matrix factorization.
problem Lack of flexible incorporation of auxiliary info (content and labels) in network embedding.
method Explicit matrix factorization incorporating structure, content, and label info.
result Unified framework for learning network embedding with structure, content, and label info.
We investigated the topological properties of stock networks through a comparison of the original stock network with the estimated stock network from the correlation matrix created by the random matrix theory (RMT). We used individual stocks traded on the market indices of Korea, Japan, Canada, the USA, Italy, and the …
We provide a proof of backpropagation algorithm in matrix notation.
problem The lack of a full induction proof of backpropagation algorithm in matrix notation.
method We provide a full induction proof of the BP algorithm in matrix notation, situating it in the framework of matrix differential calculus.
result We prove the validity of the backpropagation algorithm in inductive form.
Graph neural networks speed up nonnegative matrix factorization.
problem Efficiently factorize nonnegative matrices for various applications.
method Developed a graph neural network that combines bipartite self-attention with ADMM updates.
result Significant acceleration achieved in nonnegative matrix factorization.
Variational inference improves neural network matrix factorization for stochastic blockmodels.
problem Improving predictive performance of neural network matrix factorization for stochastic blockmodels.
method Construct Bayesian neural networks and fit with variational inference.
result Variational inference can achieve equivalent performance to neural networks on Movielens data.
The study characterizes the conditioning of the Gauss-Newton matrix in neural networks.
problem Understanding the conditioning of the Gauss-Newton matrix in neural networks.
method Theoretical analysis of the GN matrix in deep linear and ReLU networks, extending to residual connections and convolutional layers.
result Established tight bounds on the condition number of the GN matrix in neural networks.
Study on neural network initialization with shaped infinite depth-and-width networks.
problem Understanding the distribution of random covariance matrices in shaped infinite-depth-and-width networks.
method Introduced the Neural Covariance SDE to model the distribution of the random covariance matrix.
result Identified the precise scaling of the activation function necessary for a non-trivial limit.
Two tricks reduce LSTM complexity and speed up training.
problem Training large LSTM networks is computationally expensive.
method Matrix factorization and partitioning of LSTM components.
result Significantly faster training with fewer parameters.
Unified deep learning models for network embedding using matrix factorization.
problem Improving network representation learning for various applications.
method Unified matrix factorization approach for DeepWalk, LINE, PTE, and node2vec.
result Unified models reveal theoretical connections to graph Laplacian.
New neural model processes 2D data with long-range dependencies efficiently.
problem Limited receptive field of convolutions for complex 2D tasks.
method Proposes Matrix Shuffle-Exchange network with O ( log n ) \mathcal{O}( \log{n}) O ( log n ) layers and O ( n 2 log n ) \mathcal{O}( n^2 \log{n}) O ( n 2 log n ) complexity. result Exceeds convolutional and graph neural network baselines in long-range dependency modeling.
A method for community detection in multilayer networks using data matrices.
problem Community detection in multilayer networks with various node attributes.
method Data matrix representation and regular decomposition method extension for compression.
result Method identifies community structures well-aligned with real-world network hierarchies.
MFNs parameterize non-local interactions through matrix equivariant functions, improving graph neural network performance.
problem Challenges in modeling non-local interactions in graphs, such as oversmoothing and oversquashing.
method Matrix Function Neural Networks (MFNs) using resolvent expansions for non-local interactions.
result Achieves state-of-the-art performance in graph benchmarks and captures intricate non-local interactions.
Deep neural network model extends matrix completion to unseen data.
problem Limited extendability of neural-network-based matrix completion models.
method Two-branch neural network model that can predict and extend to unseen data.
result Model outperforms state-of-the-art in accuracy and extendability.
SNN architecture shows gradient descent converges to regularized solution in matrix sensing problems.
problem Understanding implicit regularization in neural networks for matrix sensing.
method Developed Spectral Neural Networks (SNN) for matrix learning problems, rigorously demonstrating implicit regularization.
result Gradient descent converges to the solution of a regularized learning problem in matrix sensing problems.
Data often comes in the form of an array or matrix. Matrix factorization techniques attempt to recover missing or corrupted entries by assuming that the matrix can be written as the product of two low-rank matrices. In other words, matrix factorization approximates the entries of the matrix by a simple, fixed function-…
Paper presents new matrix formats for deep neural networks that improve inference efficiency.
problem High computational cost of dot product operations in deep neural networks.
method Develops new matrix formats with bounded complexity by entropy of weight matrices.
result Up to x90 energy savings and x5 speed ups in dot product operations.
Automatically designs neural network structure using matrix conditioning.
problem Choosing an effective size and structure of neural networks for new datasets is time-consuming.
method Adjusts neuron proportions and scales network size using matrix conditioning.
result Small networks achieve high accuracy on various datasets.
New methods for community detection in multi-layer networks improve upon existing techniques.
problem Estimating a consensus community structure in multi-layer networks.
method Spectral clustering and matrix factorization methods for low-rank matrix optimization.
result Consistency properties of intermediate fusion techniques under multi-layer stochastic blockmodel.
Effective Gram matrix predicts deep network generalization.
problem Understanding and predicting deep network generalization.
method Derived a differential equation governing generalization gap, analyzed with effective Gram matrix.
result Effective Gram matrix accurately predicts test loss during training.
New method tightens spectral bounds for percolation in clustered networks.
problem Tight spectral bounds for percolation in sparse networks with clustering.
method Message passing algorithm on triangle-non-backtracking matrix.
result Method gives tighter lower-bound to percolation transition.
MPDCompress compresses deep neural networks by rearranging their matrices for better hardware compatibility.
problem Large deep neural networks are hard to deploy on edge devices due to size and computational complexity.
method Matrix permutation decomposition via random mask generation to transform irregular matrices into structured blocks.
result Achieved up to 8x network compression with less than 1% accuracy loss on various datasets.
Geometric matrix completion learns graph patterns and non-linear diffusion efficiently.
problem Efficiently learn graph patterns and non-linear diffusion from user/item graphs.
method Geometric deep learning on graphs with graph convolutional and recurrent neural networks.
result Outperforms state-of-the-art techniques on synthetic and real datasets.
Flat minima lead to better generalization in low-rank matrix recovery models.
problem Understanding why flat minima generalize well in overparameterized models.
method Analysis of overparameterized matrix and bilinear sensing, robust PCA, covariance matrix estimation, and neural networks with quadratic activation functions.
result Flat minima, measured by the trace of the Hessian, exactly recover the ground truth in low-rank matrix recovery models under standard statistical assumptions.
Paper proposes new costs for learning multiple centers in MDNs.
problem Learning multiple centers for density approximation in MDNs.
method Combines MDNs with contrastive costs using four types of kernelized matrix costs.
result New costs improve data density approximation in MDNs.
Method detects communities in networks using matrix factorization.
problem Community detection in complex networks.
method Orthogonal symmetric non-negative matrix tri-factorization of the normalized Laplacian matrix.
result Consistent for community detection in graphs from stochastic block models.
New algorithm for efficient prediction intervals in neural networks.
problem Challenges in estimating uncertainty in neural network predictions.
method Applies matrix sketching to approximate Jacobian matrix for efficient uncertainty estimation.
result Produces approximate prediction intervals with competitive performance.
Method selects number of communities in weighted networks.
problem Selecting the number of communities in weighted networks.
method Proposes a novel weighted DCSBM and uses a sequential testing framework with spectral clustering and matrix scaling.
result Method is consistent in estimating the true number of communities under mild conditions.
Random matrix theory analyzes learning dynamics in neural networks.
problem Understanding learning dynamics in neural networks.
method Random matrix approach to analyze a linear network.
result Insights into overfitting, early stopping, and initialization.
Paper builds neural networks on matrix manifolds using gyrovector spaces.
problem Lack of concepts in gyrovector spaces for matrix manifolds.
method Generalized gyrovector space concepts for SPD and Grassmann manifolds, proposing new neural network models.
result Demonstrated effectiveness in human action recognition and knowledge graph completion.
Improves deep neural network generalization with Jacobian matrix bounds.
problem Deep neural networks' generalization error with arbitrary depth and width.
method Analysis of Jacobian matrix spectral norm for arbitrary non-linearities and pooling layers.
result Bounded spectral norm of Jacobian matrix ensures good generalization.
Gradient descent implicitly regularizes over-parameterized matrix factorization and neural networks with quadratic activations.
problem Implicit regularization in over-parameterized models with quadratic activations.
method Gradient descent applied to parameterizing U U o p UU^ op U U o p with U ∈ R d i m e s d U\in \mathbb R^{d imes d} U ∈ R d im es d to recover a rank r r r positive semidefinite matrix X ⋆ X^{\star} X ⋆ . result Gradient descent recovers X ⋆ X^{\star} X ⋆ in i l d e O ( r ) ilde{O}(\sqrt{r}) i l d e O ( r ) iterations starting from a small initialization. Unified spectral clustering for sparse networks with heterogeneous degrees.
problem Efficiently detecting communities in sparse networks with varying degrees.
method Developed a parametrized regularized Laplacian matrix for spectral clustering.
result Improved parametrization accounts for network heterogeneity and community hardness.
Sharp asymptotics reveal how network width controls learnability in quadratic neural networks.
problem Understanding learnability in overparameterized quadratic neural networks.
method Mapping ERM to convex matrix sensing with nuclear norm penalization.
result Characterization of global minima and precise generalization thresholds.
The article proposes a new portfolio allocation method using network theory.
problem Portfolio allocation problem by improving network theory tools.
method Enhancing network theory tools to construct risk-based models and using two covariance matrix estimators.
result Network-based portfolios consistently outperform standard portfolios in terms of performance and risk.
Efficient approximations for AdaGrad reduce computation while maintaining performance.
problem Training deep neural networks efficiently in high dimensions.
method Ada-LR and RadaGrad use random projections to approximate full-matrix AdaGrad.
result Regret of Ada-LR is close to full-matrix AdaGrad, achieving similar performance with less computation.
LNMC improves link prediction on social networks by considering log-normal degree distributions.
problem Link prediction in social networks with log-normal degree distributions.
method Log-Normal Matrix Completion (LNMC) using Alternating Direction Method of Multipliers.
result Up to 5% AUC increase over non-structured sparsity based methods.
The paper uses NMF to detect political communities in Twitter networks.
problem Detecting pure political communities in Twitter networks.
method Developed three NMF frameworks to analyze user connectivity and content.
result User content and endorsement filtered connectivity are complementary.
Paper proposes neural networks for fundamental matrix estimation without key-point correspondences.
problem Estimating fundamental matrices from noisy and unreliable key-point correspondences.
method End-to-end neural network architectures preserving fundamental matrix properties.
result Neural networks achieve competitive performance on the KITTI dataset without correspondences.
A neural network learns data similarities and predicts pairwise relations.
problem Predicting and understanding pairwise relations between data points.
method Introducing a neural network architecture called Similarity Encoder (SimEc).
result SimEc can predict multiple pairwise relations and preserve non-metric similarities.
Analyzes Hessian spectrum for neural networks near optimal learning.
problem Understanding learning dynamics near optimal points in neural networks.
method Characterizes Hessian eigenspectrum for teacher-student problems, using analytical and numerical methods.
result The rank of the Hessian matrix determines effective number of parameters for non-linear networks.
AutoLL uses neural networks to automatically reorder graph nodes for linear layouts.
problem Finding optimal node order in adjacency matrices without predefined features.
method Developed AutoLL-D and AutoLL-U neural network models for one-mode reordering of directed and undirected graphs.
result Demonstrated effectiveness of AutoLL through qualitative and quantitative evaluations.
A method learns matrix factorization from diverse matrices and applies the knowledge to unseen matrices.
problem Matrix factorization without shared rows or columns.
method Neural network meta-learned to minimize expected imputation error using MAP estimation.
result The method can impute missing values from unseen matrices efficiently.
Two Fisher information matrix estimators are analyzed for neural networks, focusing on their variances and trade-offs.
problem Estimating the Fisher information matrix in neural networks due to its high computational cost.
method Examined two popular diagonal Fisher information matrix estimators and their variances in neural networks for regression and classification.
result The variances of the estimators depend on the non-linearity with respect to different parameter groups and should not be neglected.