Study of pseudo-Riemannian manifolds with M{ö}bius group actions.
problem Characterizing pseudo-Riemannian manifolds with specific group actions.
method Analyzing the action of the conformal group on compact manifolds.
result If the non-compact semi-simple part of the conformal group is M{ö}bius, the manifold is conformally flat.
Dynamic sparseness reduces neural network computation by selectively omitting parts of computations.
problem Reducing the computational and memory footprint of neural networks.
method Combining dynamic sparseness with block-wise matrix-vector multiplications to selectively omit parts of computations.
result The proposed method outperforms static sparseness and achieves similar perplexities at half the computational cost.
MuRP embeds multi-relational graphs in hyperbolic space for better hierarchical representation.
problem Current hyperbolic models struggle with multi-relational knowledge graphs that exhibit multiple hierarchies.
method MuRP embeds multi-relational graph data in the Poincaré ball model of hyperbolic space, learning relation-specific parameters for entity embeddings.
result MuRP embeddings outperform Euclidean counterparts and other methods on link prediction tasks, especially at lower dimensions.
Study analyzes Bayesian inference algorithms using dynamical functional approach.
problem Analysis of approximate inference algorithms for large Gaussian latent variable models.
method Dynamical functional approach to model nontrivial dependencies and obtain exact effective stochastic process.
result Closed-form expressions for the rate of convergence are derived and validated.
SKI accelerates GP inference with sparse grids to handle higher dimensions.
problem SKI scales poorly in high dimensions due to dense grid size.
method Sparse grids within SKI framework, novel matrix-vector multiplication algorithm.
result SKI can be scaled to higher dimensions while maintaining accuracy.
Paper speeds up Gaussian process inference using Matérn kernels.
problem Efficiently performing Gaussian process inference for large datasets.
method Exact Matérn kernel decomposition into empirical cumulative distribution functions, combined with divide-and-conquer approach.
result The proposed algorithm significantly speeds up Gaussian process inference for low-dimensional problems with hundreds of thousands of data points.
Gradient-based optimization improves variational empirical Bayes regression.
problem Sparse, large-scale multiple regression models.
method Gradient-based optimization (GradVI) for variational empirical Bayes (VEB) regression.
result GradVI produces similar predictive performance to CAVI but converges faster and is faster in certain settings.
The graph Laplacian is a standard tool in data science, machine learning, and image processing. The corresponding matrix inherits the complex structure of the underlying network and is in certain applications densely populated. This makes computations, in particular matrix-vector products, with the graph Laplacian a ha…
Efficiently computes matrix square roots and their inverses for large matrices.
problem Computing matrix square roots and inverses for large matrices efficiently.
method Combines Krylov subspace methods with rational approximation for quadratic-time computation.
result Achieves 4 decimal places of accuracy with fewer than 100 matrix-vector multiplications.
Fast linear transforms are ubiquitous in machine learning, including the discrete Fourier transform, discrete cosine transform, and other structured transformations such as convolutions. All of these transforms can be represented by dense matrix-vector multiplication, yet each has a specialized and highly efficient (su…
Deep ReLU networks can approximate matrix-vector products with error bounds.
problem Can deep ReLU networks accurately approximate matrix-vector products?
method Derived error bounds in Lebesgue and Sobolev norms for deep ReLU FNNs.
result Developed deep approximation theory with successful applications.
Flexible knot construction for low genus surfaces.
problem Finding knots with unexpectedly low genus surfaces.
method Flexible construction of knots in 3-sphere that bound surfaces of low genus in punctured open books.
result First examples of knots with differing genus in different homology balls.
New algorithms estimate matrix norms without matrix multiplication.
problem Estimating matrix norms efficiently in a matrix-free setting.
method Randomized algorithms based on Hutchinson's estimator modifications.
result Oracle complexity bounds for two-to-infinity and one-to-two norms.
Recent work shows that inference for Gaussian processes can be performed efficiently using iterative methods that rely only on matrix-vector multiplications (MVMs). Structured Kernel Interpolation (SKI) exploits these techniques by deriving approximate kernels with very fast MVMs. Unfortunately, such strategies suffer …
New method tackles high-dimensional SBL without covariance matrices.
problem Sparse coding problem in high-dimensional settings.
method Parallel solution of multiple linear systems using conjugate gradient algorithm.
result Our method scales better in computation time and memory.
Improved graph-based multiclass classification for multilayer data.
problem Efficient classification of multilayer data with limited labeled examples.
method Generalized diffuse interface methods applied to multilayer graphs, using spectral decomposition and fast matrix-vector products.
result Highly scalable and efficient classification for large, high-dimensional data sets.
Gaussian processes (GPs) with derivatives are useful in many applications, including Bayesian optimization, implicit surface reconstruction, and terrain reconstruction. Fitting a GP to function values and derivatives at n points in d dimensions requires linear solves and log determinants with an ${n(d+1) \times n(d…
New algorithms solve linear algebra problems in sublinear time.
problem Numerical linear algebra problems, especially with structured matrices.
method Sublinear time algorithms using matrix-vector multiplications.
result Solve problems like least squares regression and low rank approximation in sublinear time.
For applications as varied as Bayesian neural networks, determinantal point processes, elliptical graphical models, and kernel learning for Gaussian processes (GPs), one must compute a log determinant of an n×n positive definite matrix, and its derivatives - leading to prohibitive O(n3) computatio…
The Clifford torus is unique when its isoperimetric ratio is prescribed.
problem Proving the uniqueness of the Clifford torus with a prescribed isoperimetric ratio.
method Reduction to a positivity question of a polynomial recurrence.
result The conjecture can be reduced to a polynomial recurrence positivity question.
Efficient method estimates intrinsic dimension for big data.
problem Estimating intrinsic dimension for large datasets is costly and complex.
method Proposes a matrix-vector product-based approach for efficient intrinsic dimension estimation.
result Demonstrates superior performance compared to state-of-the-art methods.
Efficient kernel methods for large datasets using GPU acceleration.
problem Handling large-scale nonparametric learning problems efficiently.
method Preconditioned gradient solver, GPU acceleration, parallelization, out-of-core linear algebra, numerical precision optimization.
result Dramatic speedups on datasets with billions of points, maintaining state-of-the-art performance.
New method estimates large matrices' spectra from small sub-matrices.
problem Estimating large matrices' spectra when full matrix-vector products are not available.
method Free decompression based on free probability theory.
result Estimates eigenspectrum of impalpable matrices from small sub-matrices.
Paper develops IFTRR to solve sparse generalized eigenvalue problems efficiently.
problem Finding the leading eigenvector with at most k nonzero entries in sparse generalized eigenvalue problems.
method Inverse-free truncated Rayleigh-Ritz method (IFTRR) with a new truncation strategy.
result IFTRR efficiently finds the support set of the leading eigenvector for large scale problems.
This paper proposes a representational model for grid cells. In this model, the 2D self-position of the agent is represented by a high-dimensional vector, and the 2D self-motion or displacement of the agent is represented by a matrix that transforms the vector. Each component of the vector is a unit or a cell. The mode…
Improved analysis of accelerated noisy power method for PCA.
problem Inexact matrix-vector products in PCA settings.
method Improved analysis of Accelerated Noisy Power Method under milder perturbation conditions.
result Worst-case optimal convergence rate with relaxed noise conditions.
This paper speeds up PDV model calibration by learning SPX and VIX prices.
problem Slow calibration of the 4-factor PDV model due to expensive outer simulation.
method Learning SPX and VIX prices with neural networks to reduce outer simulation time.
result Calibration times reduced to just a few seconds.
The ability to decompose a signal in an orthonormal basis (a set of orthogonal components, each normalized to have unit length) using a fast numerical procedure rests at the heart of many signal processing methods and applications. The classic examples are the Fourier and wavelet transforms that enjoy numerically effic…
This paper uses supervised learning to predict optimal chunk-size for parallel linear algebra operations.
problem Finding the optimal chunk-size for parallel linear algebra operations.
method The paper uses supervised learning models (logistic regression, neural networks, decision trees) to predict the optimal chunk-size for multiple linear algebra operations.
result The custom decision tree model outperforms classical decision trees and other models in predicting optimal chunk-size for linear algebra operations.
New methods for sketching non-PSD matrices improve regression and optimization tasks.
problem Efficiently handling non-PSD matrices in computations.
method Developed novel matrix sketching techniques for non-PSD and complex matrices.
result Improved performance in convex and non-convex optimization, regression, and vector-matrix-vector queries.
This paper tackles fitting multilevel low rank matrices by addressing three problems.
problem Fitting a given matrix by an MLR matrix in the Frobenius norm.
method Factor fitting, rank allocation, and hierarchical partitioning.
result The proposed methods can fit a given matrix by an MLR matrix in the Frobenius norm.
Efficiently models learning curves using Gaussian processes with latent Kronecker structure.
problem Joint modeling of machine learning model performance across hyper-parameters and training progress.
method Imposes latent Kronecker structure to leverage efficient product kernels and handle missing values.
result Matches the performance of a Transformer on a learning curve prediction task.
Paper relaxes set-valued prediction in hierarchical classification by considering representation complexity.
problem Uncertainty in class labels in hierarchical multi-class classification problems.
method Introduces representation complexity for predicted sets, proposes three methods for inference.
result Recursive tree search method is computationally more efficient.
Analog arrays are a promising upcoming hardware technology with the potential to drastically speed up deep learning. Their main advantage is that they compute matrix-vector products in constant time, irrespective of the size of the matrix. However, early convolution layers in ConvNets map very unfavorably onto analog a…
Solving symmetric positive definite linear problems is a fundamental computational task in machine learning. The exact solution, famously, is cubicly expensive in the size of the matrix. To alleviate this problem, several linear-time approximations, such as spectral and inducing-point methods, have been suggested and a…
We propose a semismooth Newton algorithm for pathwise optimization (SNAP) for the LASSO and Enet in sparse, high-dimensional linear regression. SNAP is derived from a suitable formulation of the KKT conditions based on Newton derivatives. It solves the semismooth KKT equations efficiently by actively and continuously s…
In this paper we show that for the purposes of dimensionality reduction certain class of structured random matrices behave similarly to random Gaussian matrices. This class includes several matrices for which matrix-vector multiply can be computed in log-linear time, providing efficient dimensionality reduction of gene…
Efficient approximations reduce computation of matrix-based Renyi's entropy.
problem High computational complexity of matrix-based Renyi's entropy.
method Taylor, Chebyshev, and Lanczos approximations to reduce complexity.
result Reduced complexity to significantly less than O(n2) with negligible accuracy loss. We explore the robustness of recurrent neural networks when the computations within the network are noisy. One of the motivations for looking into this problem is to reduce the high power cost of conventional computing of neural network operations through the use of analog neuromorphic circuits. Traditional GPU/CPU-cen…
Study generalizes Möbius energy to non-smooth sets in arbitrary dimensions.
problem Investigate Möbius-invariant energies on non-smooth subsets of arbitrary dimensions.
method Show local finite energy implies embedded Lipschitz submanifold, and low fractional Sobolev regularity guarantees finite energy.
result Local graph structure of low fractional Sobolev regularity on a set is sufficient to guarantee finite energy.
Lower bounds show linear complexity for linear regression.
problem Computational complexity of linear regression.
method Reduction to estimating the least eigenvalue of a random Wishart matrix.
result Θ(d) calls to the oracle are necessary and sufficient for polynomial accuracy.
Memristors have recently received significant attention as ubiquitous device-level components for building a novel generation of computing systems. These devices have many promising features, such as non-volatility, low power consumption, high density, and excellent scalability. The ability to control and modify biasin…
A conformal metric g with constant curvature one and finite conical singularities on a compact Riemann surface Σ can be thought of as the pullback of the standard metric on the 2-sphere by a multi-valued locally univalent meromorphic function f on Σ\{singularities}, called the {\it developing …
The paper analyzes contraction rates for GP regression approximations.
problem Computational infeasibility of exact GP posterior in large-scale applications.
method Lanczos and conjugate gradient approximations of the posterior mean.
result Minimax contraction rates for these approximations in large-scale applications.
Statistical learning improves reactive power control in distribution systems.
problem Challenges in reactive power control due to renewable energy sources and flexible loads.
method A deep neural network parameterizes the input-output relationship between grid states and optimal reactive power control. Unknown weights are learned offline to minimize power loss, and inference is fast with matrix-vector multiplications.
result Computational efficiency and robustness to random input perturbations demonstrated in a 47-bus distribution network.
Efficient federated algorithm for calculating transportation barycenter.
problem Efficiently calculating the free-support transportation barycenter in a federated setting.
method Single-loop dual decomposition algorithm that uses only aggregated information.
result Significantly scalable and low-complexity algorithm for federated computation.
NeuralIF uses neural networks to improve preconditioning for faster CG convergence.
problem Improving convergence of conjugate gradient method for large-scale sparse systems.
method Data-driven approach using graph neural networks to generate incomplete factorization.
result Data-driven preconditioners accelerate convergence of conjugate gradient method.
Paper proposes a new ML approach using only additions and thresholding.
problem Energy efficiency and reduced complexity for IoT ML devices.
method Margin-Propagation (MP) network for inference and learning without MVMs.
result MP-based classifiers achieve comparable results to traditional ML methods with energy savings.