Paper proposes ABDR for convex subspace clustering with adaptive block diagonal representation.
problem Subspace clustering with block diagonal structure for noisy data.
method ABDR explicitly pursues block diagonality without sacrificing convexity, using a specially designed convex regularizer.
result Experimental results show ABDR outperforms state-of-the-arts.
This paper solves matrix blind joint block diagonalization with noise.
problem Identifying the diagonalizer and block diagonal structure of matrices under noise.
method Bi-block diagonalization method.
result The method can identify the exact solution under certain conditions.
New theory allows simultaneous block-diagonalization of commuting operator fields.
problem Normal forms of operator fields.
method Generalized Nijenhuis torsions and generalized Haantjes algebra.
result Simultaneous block-diagonalization of commuting operator fields.
Randomized block-diagonal preconditioning improves parallel learning convergence.
problem Improving convergence of gradient-based optimization methods in parallel settings.
method Randomization of coordinates during optimization to repartition tasks.
result Randomization significantly improves convergence of block-diagonal preconditioned methods.
New insights into Hessian structure of neural networks reveal two forces.
problem Understanding the Hessian structure of neural networks.
method Analyzing the static and dynamic forces, comparing limit distributions using random matrix theory.
result The Hessian structure arises from a combination of static and dynamic forces, with C being a primary driver. A method to simplify Gaussian graphical models for high-dimensional data.
problem Difficulty in inferring networks of dependencies between variables when sample size is small.
method Approximate covariance matrix as block-diagonal, select threshold using slope heuristic, infer network in blocks.
result The method reduces the number of parameters to estimate and improves network inference.
New model handles complex non-linear relationships with hidden graph structures.
problem Modeling non-linear relationships with hidden graph-structured interactions.
method Block-diagonal localized mixture of polynomial experts (BLoMPE) regression model with penalized maximum likelihood selection criterion.
result Strong theoretical guarantee for finite-sample oracle inequality.
New adaptive methods improve deep learning performance.
problem Training deep networks efficiently and effectively.
method Block-diagonal matrix adaptation for gradient updates.
result Block-diagonal methods outperform adaptive diagonal methods and vanilla SGD.
Localized sketching improves matrix multiplication and ridge regression complexity.
problem Efficiently approximate matrix multiplication and ridge regression with limited data availability.
method Localized sketching matrices for block diagonal structure, reducing sample complexity.
result Localized sketching achieves sample complexity matching global sketching methods.
Modular method simplifies curvature computation in neural nets.
problem Efficient computation of curvature matrices for training neural nets.
method Modular backpropagation for block-diagonal approximations.
result Compact notation and easy integration into machine learning libraries.
This paper tackles model selection for MoE models in high-dimensional data.
problem Model selection for Gaussian-gated localized MoE and block-diagonal covariance localized MoE regression models in high-dimensional data.
method Penalized maximum likelihood estimation framework with non-asymptotic risk bounds.
result Established non-asymptotic risk bounds for model selection in MoE models.
Improved sparse Gaussian processes using structured scaling matrices and Power-EP framework.
problem Scaling Gaussian processes for large datasets.
method Structured diagonal scaling matrix and Power-EP framework.
result Structured approximations improve performance without increasing computational cost.
Paper proposes a new Markov model for efficient PLC system design.
problem Efficient estimation of Markov model parameters for bursty error channels.
method Introduced a Block Diagonal Markov model and a modified Baum-Welch algorithm.
result Efficient estimation of state transition matrix Λ for PLC system design. Solves model order selection for spectral graph clustering.
problem Automated selection of the correct number of clusters in spectral graph clustering.
method AMOS, a selection criterion based on asymptotic phase transition analysis.
result Validates phase transition analysis and model selection procedure on real-world data.
New method for estimating financial covariance matrices efficiently.
problem Noisy covariance matrix estimation in high-dimensional financial data.
method Cluster financial time series into groups, apply shrinkage to ensure positive definiteness.
result Proposed methods provide reliable estimates and outperform other estimators.
Framework for incomplete multi-view learning improves efficiency and clustering accuracy.
problem Incomplete representation in multi-view data.
method Joint Embedding Learning and Low-Rank Approximation (JELLA) framework.
result Improves efficiency and clustering accuracy in incomplete multi-view data.
DKLM learns adaptive kernels for robust nonlinear subspace clustering.
problem Nonlinear structures in data and challenges with kernel-based clustering.
method Data-driven kernel learning with adaptive weighting and optimal block-diagonal affinity matrix.
result DKLM enhances robustness and preserves manifold structure in nonlinear space.
Improved Hessian-free method for neural networks reduces computational cost.
problem High computational cost and model-dependent algorithmic variations in second-order methods.
method Block-diagonal approximation of the generalized Gauss-Newton matrix, conjugate gradient updates for each block.
result Better convergence and generalization compared to original Hessian-free and Adam methods.
Efficiently approximates Sparse PCA with significant speedups and minor error.
problem Sparse Principal Component Analysis (Sparse PCA) is NP-hard and computationally expensive.
method Approximates the covariance matrix with block-diagonal form, solves sub-problems in each block, and reconstructs the solution.
result Significant computational speedups with minor additive error.
Develops efficient quasi-Newton methods for training deep neural networks.
problem Training deep neural networks with large-scale Hessian matrices.
method Approximates Hessian as block-diagonal Kronecker product of smaller matrices, applies damping.
result Outperforms or matches state-of-the-art methods in autoencoder models.
Efficient Gauss-Newton optimisation for neural networks.
problem Optimizing deep learning models efficiently and effectively.
method Block-diagonal approximation of Gauss-Newton matrix for neural networks.
result The resulting algorithm outperforms state-of-the-art first-order optimisation methods.
New neural network scaling improves robustness.
problem Improving robustness of neural networks.
method Introduces a new rescaling technique for neural network training.
result Scales stepsizes to improve robustness.
Efficient subspace clustering using Kronecker product reduces computational complexity.
problem Efficiency and scalability issues in traditional subspace clustering methods for large datasets.
method Proposes a subspace clustering model based on the Kronecker product to reduce computational complexity.
result Significantly improved efficiency compared to state-of-the-art methods on public datasets.
Proposes TFCL to mitigate negative transfer in MTL by collaborating across features and tasks.
problem Negative transfer in Multi-Task Learning (MTL) due to dissimilar tasks.
method Task-Feature Collaborative Learning (TFCL) with heterogeneous block-diagonal structure regularizer and optimization method.
result Global convergence and block-diagonal structure recovery guarantees.
The paper introduces a penalized matrix estimation procedure aiming at solutions which are sparse and low-rank at the same time. Such structures arise in the context of social networks or protein interactions where underlying graphs have adjacency matrices which are block-diagonal in the appropriate basis. We introduce…
The paper simplifies the Fisher information matrix for random deep networks, speeding up learning.
problem Learning deep neural networks efficiently with large parameter spaces.
method Statistical neurodynamical method to reveal Fisher information properties, proving unit-wise block diagonal structure and explicit inverse.
result Explicit natural gradient formula without matrix inversion, speeding up learning.
Parallelizes DEC on curved meshes using group actions.
problem Efficiently solving DEC operators on curved and 3D meshes.
method Universal block-diagonalization framework for d and ⋆ operators, exploiting group actions. result Block-diagonal structure inherited by operators, enabling parallel solvers.
Bayesian online learning method improves neural network performance.
problem Overcoming catastrophic forgetting in neural networks.
method Kronecker factored online Laplace approximation for Bayesian online learning.
result Achieves over 90% test accuracy across 50 MNIST tasks.
Unsupervised framework learns symmetry from time sequences.
problem Learning symmetry from time sequences without labeled data.
method Meta-sequential prediction (MSP) framework that leverages stationary properties.
result Hidden disentangled structure emerges as a by-product of training.
ViViT efficiently computes curvature for deep networks without approximations.
problem Efficiently computing curvature for deep networks without approximations.
method Leverages the GGN's low-rank structure without further approximations.
result ViViT allows for efficient computation of eigenvalues, eigenvectors, and directional derivatives.
Ensembles of neural networks improve training dynamics and performance.
problem Improving neural network performance through model size increase.
method Defining collegial ensembles (CE) as multiple independent models trained as a single model, and using theoretical results on NTK to optimize architecture search.
result CE dynamics simplify and scale favorably, resembling wide models, and can be efficiently implemented using group convolutions and block diagonal layers.
CoLA automates efficient numerical linear algebra for complex matrix structures.
problem Efficiently solving large-scale linear algebra problems with complex matrix structures.
method Combining linear operator abstraction with compositional dispatch rules.
result Automatic and efficient numerical algorithms for various linear algebra operations.
Haantjes algebras help in diagonalizing operators on manifolds.
problem Diagonalizing operators on differentiable manifolds.
method Introducing Haantjes algebra, a family of operator fields with vanishing Haantjes torsion and compatibility conditions.
result Simultaneous diagonalization of operators in local coordinates or block-diagonal form in general cases.
It is shown that, in four dimensions, it is possible to introduce coordinates so that an analytic metric locally takes block diagonal form. i.e. one can find coordinates such that gαβ=0 for (α,β)∈S where S=(1,3),(1,4),(2,3),(2,4). We call a coordinate system in which the metric takes this for…
Characterizes Anosov reducible representations in terms of eigenvalues.
problem Understanding Anosov representations in reducible settings.
method Characterizes Anosov representations using eigenvalue magnitudes of irreducible block factors.
result Connected components of character varieties do not contain reducible representations for many non-elementary hyperbolic groups.
The paper identifies redundant columns in matrices for feature selection and clustering.
problem Identifying redundant columns in matrices for feature selection and clustering.
method Proves that after re-ordering columns, a matrix can be block-diagonalized revealing linearly dependent columns.
result Identifies redundant columns in matrices, aiding in feature selection and clustering.
Framework estimates precision matrices for heterogeneous populations.
problem Estimating precision matrices in populations with subpopulations.
method Laplacian shrinkage penalty, ADMM algorithm, hierarchical clustering.
result Consistent estimation of precision matrices in heterogeneous populations.
Shrinkage covariance is a multifactor model with risk factors and principal components.
problem Out-of-sample instabilities in sample covariance matrices.
method Combines risk factors and principal components with a block-diagonal factor covariance matrix.
result Shrinkage is a regularization scheme with less out-of-sample instability.
Quantum Natural Gradient uses quantum geometry for optimization.
problem Optimizing variational quantum circuits efficiently.
method Quantum generalization of Natural Gradient Descent using Quantum Information Geometry.
result Efficient algorithm for computing metric tensor approximations.
The paper defines a Chern-Simons invariant for stably trivial vector bundles and uses it to obstruct conformal immersions.
problem Obstructing conformal immersions of Riemannian manifolds.
method Defining a Chern-Simons invariant for stably trivial vector bundles and using it to derive an obstruction.
result An obstruction for conformally immersing a n-dimensional Riemannian manifold in a translation manifold of dimension n+1.
We use a cluster ensemble to determine the number of clusters, k, in a group of data. A consensus similarity matrix is formed from the ensemble using multiple algorithms and several values for k. A random walk is induced on the graph defined by the consensus matrix and the eigenvalues of the associated transition proba…
New method estimates sparse covariance matrices in logit mixtures.
problem Estimating correlations among random coefficients in logit models.
method Mixed-integer optimization (MIO) with Markov Chain Monte Carlo (MCMC) for posterior draws.
result Correctly recovers true covariance structure from synthetic data.
HollowFlow speeds up likelihood evaluation for large-scale models.
problem Prohibitive scaling of sample likelihood computations in flow-based models.
method Introduces HollowFlow, a flow-based generative model using a NoBGNN with a block-diagonal Jacobian structure.
result Achieves up to O(n^2) speed-up in likelihood evaluation for large systems.
New algorithm for biclustering with improved performance.
problem Simultaneous clustering of rows and columns with similar patterns.
method Formulated new biclustering problem, developed alternating k-means algorithm.
result Our algorithm finds local minima efficiently and outperforms other methods.
A formula connects two algebraic structures derived from a category.
problem Connecting two algebraic structures derived from a category.
method Using differential graded modular functors and Calabi-Yau structures.
result The action of a specific mapping class group element transforms one algebraic structure into another.
BEGIN network models binary data without parametric assumptions.
problem Conditional independence in non-parametric families of binary data.
method BEGIN network models binary data using sparse linear representations and block factorizations.
result BEGIN network captures conditional independence for arbitrary binary and multinomial variables.
Paper proposes S2ConvSCN for robust subspace clustering and classification.
problem Insufficient handling of nonlinear manifolds, data corruptions, and out-of-sample data.
method Self-supervised convolutional subspace clustering network (S2ConvSCN) with FC layer, CIM for robustness, and BD regularization. result Robust S2ConvSCN outperforms baseline on unseen data. The paper explores warped-like product metrics with exceptional holonomy groups.
problem Exploring new metrics with exceptional holonomy groups.
method Presented a general ansatz of warped-like product metric and studied specific examples.
result Explicit examples of (3+3+2) and (3+3+1) warped-like product manifolds with Spin(7) and G2 holonomy, respectively.