This paper solves matrix blind joint block diagonalization with noise.
problem Identifying the diagonalizer and block diagonal structure of matrices under noise.
method Bi-block diagonalization method.
result The method can identify the exact solution under certain conditions.
Paper proposes ABDR for convex subspace clustering with adaptive block diagonal representation.
problem Subspace clustering with block diagonal structure for noisy data.
method ABDR explicitly pursues block diagonality without sacrificing convexity, using a specially designed convex regularizer.
result Experimental results show ABDR outperforms state-of-the-arts.
New theory allows simultaneous block-diagonalization of commuting operator fields.
problem Normal forms of operator fields.
method Generalized Nijenhuis torsions and generalized Haantjes algebra.
result Simultaneous block-diagonalization of commuting operator fields.
Modular method simplifies curvature computation in neural nets.
problem Efficient computation of curvature matrices for training neural nets.
method Modular backpropagation for block-diagonal approximations.
result Compact notation and easy integration into machine learning libraries.
Randomized block-diagonal preconditioning improves parallel learning convergence.
problem Improving convergence of gradient-based optimization methods in parallel settings.
method Randomization of coordinates during optimization to repartition tasks.
result Randomization significantly improves convergence of block-diagonal preconditioned methods.
New adaptive methods improve deep learning performance.
problem Training deep networks efficiently and effectively.
method Block-diagonal matrix adaptation for gradient updates.
result Block-diagonal methods outperform adaptive diagonal methods and vanilla SGD.
Localized sketching improves matrix multiplication and ridge regression complexity.
problem Efficiently approximate matrix multiplication and ridge regression with limited data availability.
method Localized sketching matrices for block diagonal structure, reducing sample complexity.
result Localized sketching achieves sample complexity matching global sketching methods.
Improved Hessian-free method for neural networks reduces computational cost.
problem High computational cost and model-dependent algorithmic variations in second-order methods.
method Block-diagonal approximation of the generalized Gauss-Newton matrix, conjugate gradient updates for each block.
result Better convergence and generalization compared to original Hessian-free and Adam methods.
Efficiently approximates Sparse PCA with significant speedups and minor error.
problem Sparse Principal Component Analysis (Sparse PCA) is NP-hard and computationally expensive.
method Approximates the covariance matrix with block-diagonal form, solves sub-problems in each block, and reconstructs the solution.
result Significant computational speedups with minor additive error.
New insights into Hessian structure of neural networks reveal two forces.
problem Understanding the Hessian structure of neural networks.
method Analyzing the static and dynamic forces, comparing limit distributions using random matrix theory.
result The Hessian structure arises from a combination of static and dynamic forces, with C being a primary driver. Gaussian graphical models are widely utilized to infer and visualize networks of dependencies between continuous variables. However, inferring the graph is difficult when the sample size is small compared to the number of variables. To reduce the number of parameters to estimate in the model, we propose a non-asymptoti…
New model handles complex non-linear relationships with hidden graph structures.
problem Modeling non-linear relationships with hidden graph-structured interactions.
method Block-diagonal localized mixture of polynomial experts (BLoMPE) regression model with penalized maximum likelihood selection criterion.
result Strong theoretical guarantee for finite-sample oracle inequality.
Paper proposes a new Markov model for efficient PLC system design.
problem Efficient estimation of Markov model parameters for bursty error channels.
method Introduced a Block Diagonal Markov model and a modified Baum-Welch algorithm.
result Efficient estimation of state transition matrix Λ for PLC system design. This paper tackles model selection for MoE models in high-dimensional data.
problem Model selection for Gaussian-gated localized MoE and block-diagonal covariance localized MoE regression models in high-dimensional data.
method Penalized maximum likelihood estimation framework with non-asymptotic risk bounds.
result Established non-asymptotic risk bounds for model selection in MoE models.
New method for estimating financial covariance matrices efficiently.
problem Noisy covariance matrix estimation in high-dimensional financial data.
method Cluster financial time series into groups, apply shrinkage to ensure positive definiteness.
result Proposed methods provide reliable estimates and outperform other estimators.
Efficient Gauss-Newton optimisation for neural networks.
problem Optimizing deep learning models efficiently and effectively.
method Block-diagonal approximation of Gauss-Newton matrix for neural networks.
result The resulting algorithm outperforms state-of-the-art first-order optimisation methods.
Framework for incomplete multi-view learning improves efficiency and clustering accuracy.
problem Incomplete representation in multi-view data.
method Joint Embedding Learning and Low-Rank Approximation (JELLA) framework.
result Improves efficiency and clustering accuracy in incomplete multi-view data.
Efficient subspace clustering using Kronecker product reduces computational complexity.
problem Efficiency and scalability issues in traditional subspace clustering methods for large datasets.
method Proposes a subspace clustering model based on the Kronecker product to reduce computational complexity.
result Significantly improved efficiency compared to state-of-the-art methods on public datasets.
Develops efficient quasi-Newton methods for training deep neural networks.
problem Training deep neural networks with large-scale Hessian matrices.
method Approximates Hessian as block-diagonal Kronecker product of smaller matrices, applies damping.
result Outperforms or matches state-of-the-art methods in autoencoder models.
Improved sparse Gaussian processes using structured scaling matrices and Power-EP framework.
problem Scaling Gaussian processes for large datasets.
method Structured diagonal scaling matrix and Power-EP framework.
result Structured approximations improve performance without increasing computational cost.
The paper simplifies the Fisher information matrix for random deep networks, speeding up learning.
problem Learning deep neural networks efficiently with large parameter spaces.
method Statistical neurodynamical method to reveal Fisher information properties, proving unit-wise block diagonal structure and explicit inverse.
result Explicit natural gradient formula without matrix inversion, speeding up learning.
DKLM learns adaptive kernels for robust nonlinear subspace clustering.
problem Nonlinear structures in data and challenges with kernel-based clustering.
method Data-driven kernel learning with adaptive weighting and optimal block-diagonal affinity matrix.
result DKLM enhances robustness and preserves manifold structure in nonlinear space.
Ensembles of neural networks improve training dynamics and performance.
problem Improving neural network performance through model size increase.
method Defining collegial ensembles (CE) as multiple independent models trained as a single model, and using theoretical results on NTK to optimize architecture search.
result CE dynamics simplify and scale favorably, resembling wide models, and can be efficiently implemented using group convolutions and block diagonal layers.
Shrunk sample covariance matrix is a factor model of a special form combining some (typically, style) risk factor(s) and principal components with a (block-)diagonal factor covariance matrix. As such, shrinkage, which essentially inherits out-of-sample instabilities of the sample covariance matrix, is not an alternativ…
It is shown that, in four dimensions, it is possible to introduce coordinates so that an analytic metric locally takes block diagonal form. i.e. one can find coordinates such that gαβ=0 for (α,β)∈S where S=(1,3),(1,4),(2,3),(2,4). We call a coordinate system in which the metric takes this for…
New neural network scaling improves robustness.
problem Improving robustness of neural networks.
method Introduces a new rescaling technique for neural network training.
result Scales stepsizes to improve robustness.
Characterizes Anosov reducible representations in terms of eigenvalues.
problem Understanding Anosov representations in reducible settings.
method Characterizes Anosov representations using eigenvalue magnitudes of irreducible block factors.
result Connected components of character varieties do not contain reducible representations for many non-elementary hyperbolic groups.
The paper identifies redundant columns in matrices for feature selection and clustering.
problem Identifying redundant columns in matrices for feature selection and clustering.
method Proves that after re-ordering columns, a matrix can be block-diagonalized revealing linearly dependent columns.
result Identifies redundant columns in matrices, aiding in feature selection and clustering.
Quantum Natural Gradient uses quantum geometry for optimization.
problem Optimizing variational quantum circuits efficiently.
method Quantum generalization of Natural Gradient Descent using Quantum Information Geometry.
result Efficient algorithm for computing metric tensor approximations.
Proposes TFCL to mitigate negative transfer in MTL by collaborating across features and tasks.
problem Negative transfer in Multi-Task Learning (MTL) due to dissimilar tasks.
method Task-Feature Collaborative Learning (TFCL) with heterogeneous block-diagonal structure regularizer and optimization method.
result Global convergence and block-diagonal structure recovery guarantees.
The paper defines a Chern-Simons invariant for stably trivial vector bundles and uses it to obstruct conformal immersions.
problem Obstructing conformal immersions of Riemannian manifolds.
method Defining a Chern-Simons invariant for stably trivial vector bundles and using it to derive an obstruction.
result An obstruction for conformally immersing a n-dimensional Riemannian manifold in a translation manifold of dimension n+1.
We use a cluster ensemble to determine the number of clusters, k, in a group of data. A consensus similarity matrix is formed from the ensemble using multiple algorithms and several values for k. A random walk is induced on the graph defined by the consensus matrix and the eigenvalues of the associated transition proba…
The paper introduces a penalized matrix estimation procedure aiming at solutions which are sparse and low-rank at the same time. Such structures arise in the context of social networks or protein interactions where underlying graphs have adjacency matrices which are block-diagonal in the appropriate basis. We introduce…
Parallelizes DEC on curved meshes using group actions.
problem Efficiently solving DEC operators on curved and 3D meshes.
method Universal block-diagonalization framework for d and ⋆ operators, exploiting group actions. result Block-diagonal structure inherited by operators, enabling parallel solvers.
Bayesian online learning method improves neural network performance.
problem Overcoming catastrophic forgetting in neural networks.
method Kronecker factored online Laplace approximation for Bayesian online learning.
result Achieves over 90% test accuracy across 50 MNIST tasks.
One of the longstanding open problems in spectral graph clustering (SGC) is the so-called model order selection problem: automated selection of the correct number of clusters. This is equivalent to the problem of finding the number of connected components or communities in an undirected graph. We propose automated mode…
New brackets generalize Haantjes moduli and ensure integrability of operators.
problem Characterizing and integrating operators using Haantjes moduli.
method Introducing a new infinite class of brackets and proving integrability conditions.
result Vanishing of higher-level Nijenhuis torsions ensures integrability and block-diagonal form.
Haantjes algebras help in diagonalizing operators on manifolds.
problem Diagonalizing operators on differentiable manifolds.
method Introducing Haantjes algebra, a family of operator fields with vanishing Haantjes torsion and compatibility conditions.
result Simultaneous diagonalization of operators in local coordinates or block-diagonal form in general cases.
Alexandrov spaces with non-negative curvature are characterized by the matrix displacement convexity of an entropy tensor.
problem Characterizing non-negative curvature in Alexandrov spaces
method Constructing a parallel trivialization of the entropy tensor
result The entropy tensor is matrix displacement convex on Alexandrov spaces
Given a constant magnetic field on Euclidean space Rp determined by a skew-symmetric (p×p) matrix Θ, and a Zp-invariant probability measure μ on the disorder set Σ which is by hypothesis a Cantor set, where the action is assumed to be minimal, the corresponding Integrated Density…
A robust algorithm for forecasting vector time series with seasonal components.
problem Forecasting vector time series with seasonal patterns and handling missing data.
method Auto-regression with seasonal annual, weekly, and daily baselines, and a Gaussian process for residuals. Custom truncated eigendecomposition and low-rank plus block-diagonal Gaussian kernel. Schur complement and Tikhonov regularization for efficient inference.
result The model can scale to very large datasets and is efficient in terms of memory and computation.
A new iterative K-FAC algorithm reduces training time and memory usage.
problem Training deep learning models efficiently.
method Uses conjugate gradient to approximate Fisher information matrix without generating the matrix or factors.
result Time and memory complexity of iterative CG-FAC is less than standard K-FAC.
Simplified method for multi-set correlated component analysis.
problem Performing multi-set correlated component analysis efficiently.
method Single-step solution using eigenvectors of ${f D}^{-1} {f R}$, where ${f R}$ is concatenated data covariance and ${f D}$ is block-diagonal.
result The solution maximizes inter-set correlation without additional constraints.
Existing MAP inference algorithms for determinantal point processes (DPPs) need to calculate determinants or conduct eigenvalue decomposition generally at the scale of the full kernel, which presents a great challenge for real-world applications. In this paper, we introduce a class of DPPs, called BwDPPs, that are char…
Second-order optimization speeds up deep hedging for complex options.
problem Hedging exotic options with market frictions in realistic markets.
method Second-order optimization scheme leveraging pathwise differentiability and Kronecker-factoring.
result Our method optimizes the policy in 1/4 the steps of standard optimization.
ViViT efficiently computes curvature for deep networks without approximations.
problem Efficiently computing curvature for deep networks without approximations.
method Leverages the GGN's low-rank structure without further approximations.
result ViViT allows for efficient computation of eigenvalues, eigenvectors, and directional derivatives.
A new method for scalable inference in deep discrete LVMs.
problem Challenges in scalable inference for deep discrete latent variable models.
method Topic-layer-adaptive stochastic gradient Riemannian MCMC (TLASGR) for DLDA.
result State-of-the-art results on big data sets.
We introduce a general framework for estimation of inverse covariance, or precision, matrices from heterogeneous populations. The proposed framework uses a Laplacian shrinkage penalty to encourage similarity among estimates from disparate, but related, subpopulations, while allowing for differences among matrices. We p…