Paper proposes ABDR for convex subspace clustering with adaptive block diagonal representation.
problem Subspace clustering with block diagonal structure for noisy data.
method ABDR explicitly pursues block diagonality without sacrificing convexity, using a specially designed convex regularizer.
result Experimental results show ABDR outperforms state-of-the-arts.
This paper solves matrix blind joint block diagonalization with noise.
problem Identifying the diagonalizer and block diagonal structure of matrices under noise.
method Bi-block diagonalization method.
result The method can identify the exact solution under certain conditions.
Characterizes Anosov reducible representations in terms of eigenvalues.
problem Understanding Anosov representations in reducible settings.
method Characterizes Anosov representations using eigenvalue magnitudes of irreducible block factors.
result Connected components of character varieties do not contain reducible representations for many non-elementary hyperbolic groups.
Subspace clustering is a useful technique for many computer vision applications in which the intrinsic dimension of high-dimensional data is often smaller than the ambient dimension. Spectral clustering, as one of the main approaches to subspace clustering, often takes on a sparse representation or a low-rank represent…
Randomized block-diagonal preconditioning improves parallel learning convergence.
problem Improving convergence of gradient-based optimization methods in parallel settings.
method Randomization of coordinates during optimization to repartition tasks.
result Randomization significantly improves convergence of block-diagonal preconditioned methods.
New theory allows simultaneous block-diagonalization of commuting operator fields.
problem Normal forms of operator fields.
method Generalized Nijenhuis torsions and generalized Haantjes algebra.
result Simultaneous block-diagonalization of commuting operator fields.
Modular method simplifies curvature computation in neural nets.
problem Efficient computation of curvature matrices for training neural nets.
method Modular backpropagation for block-diagonal approximations.
result Compact notation and easy integration into machine learning libraries.
New adaptive methods improve deep learning performance.
problem Training deep networks efficiently and effectively.
method Block-diagonal matrix adaptation for gradient updates.
result Block-diagonal methods outperform adaptive diagonal methods and vanilla SGD.
Framework for incomplete multi-view learning improves efficiency and clustering accuracy.
problem Incomplete representation in multi-view data.
method Joint Embedding Learning and Low-Rank Approximation (JELLA) framework.
result Improves efficiency and clustering accuracy in incomplete multi-view data.
New insights into Hessian structure of neural networks reveal two forces.
problem Understanding the Hessian structure of neural networks.
method Analyzing the static and dynamic forces, comparing limit distributions using random matrix theory.
result The Hessian structure arises from a combination of static and dynamic forces, with C being a primary driver. Efficiently approximates Sparse PCA with significant speedups and minor error.
problem Sparse Principal Component Analysis (Sparse PCA) is NP-hard and computationally expensive.
method Approximates the covariance matrix with block-diagonal form, solves sub-problems in each block, and reconstructs the solution.
result Significant computational speedups with minor additive error.
Deep latent-variable models learn representations of high-dimensional data in an unsupervised manner. A number of recent efforts have focused on learning representations that disentangle statistically independent axes of variation by introducing modifications to the standard objective function. These approaches general…
BEGIN network models binary data without parametric assumptions.
problem Conditional independence in non-parametric families of binary data.
method BEGIN network models binary data using sparse linear representations and block factorizations.
result BEGIN network captures conditional independence for arbitrary binary and multinomial variables.
Localized sketching improves matrix multiplication and ridge regression complexity.
problem Efficiently approximate matrix multiplication and ridge regression with limited data availability.
method Localized sketching matrices for block diagonal structure, reducing sample complexity.
result Localized sketching achieves sample complexity matching global sketching methods.
Second-order methods for neural network optimization have several advantages over methods based on first-order gradient descent, including better scaling to large mini-batch sizes and fewer updates needed for convergence. But they are rarely applied to deep learning in practice because of high computational cost and th…
Gaussian graphical models are widely utilized to infer and visualize networks of dependencies between continuous variables. However, inferring the graph is difficult when the sample size is small compared to the number of variables. To reduce the number of parameters to estimate in the model, we propose a non-asymptoti…
New method for estimating financial covariance matrices efficiently.
problem Noisy covariance matrix estimation in high-dimensional financial data.
method Cluster financial time series into groups, apply shrinkage to ensure positive definiteness.
result Proposed methods provide reliable estimates and outperform other estimators.
Improved sparse Gaussian processes using structured scaling matrices and Power-EP framework.
problem Scaling Gaussian processes for large datasets.
method Structured diagonal scaling matrix and Power-EP framework.
result Structured approximations improve performance without increasing computational cost.
DKLM learns adaptive kernels for robust nonlinear subspace clustering.
problem Nonlinear structures in data and challenges with kernel-based clustering.
method Data-driven kernel learning with adaptive weighting and optimal block-diagonal affinity matrix.
result DKLM enhances robustness and preserves manifold structure in nonlinear space.
Homogeneous links were introduced by Peter Cromwell, who proved that the projection surface of these links, that given by the Seifert algorithm, has minimal genus. Here we provide a different proof, with a geometric rather than combinatorial flavor. To do this, we first show a direct relation between the Seifert matrix…
A deep neural network is a hierarchical nonlinear model transforming input signals to output signals. Its input-output relation is considered to be stochastic, being described for a given input by a parameterized conditional probability distribution of outputs. The space of parameters consisting of weights and biases i…
New model handles complex non-linear relationships with hidden graph structures.
problem Modeling non-linear relationships with hidden graph-structured interactions.
method Block-diagonal localized mixture of polynomial experts (BLoMPE) regression model with penalized maximum likelihood selection criterion.
result Strong theoretical guarantee for finite-sample oracle inequality.
Paper proposes a new Markov model for efficient PLC system design.
problem Efficient estimation of Markov model parameters for bursty error channels.
method Introduced a Block Diagonal Markov model and a modified Baum-Welch algorithm.
result Efficient estimation of state transition matrix Λ for PLC system design. A fast metric learning framework using Gershgorin disc alignment.
problem Learning effective metrics for graph-based data.
method Fast projection-free metric learning via Gershgorin disc alignment.
result Efficiently computed graph metric matrices outperform competing methods.
This paper tackles model selection for MoE models in high-dimensional data.
problem Model selection for Gaussian-gated localized MoE and block-diagonal covariance localized MoE regression models in high-dimensional data.
method Penalized maximum likelihood estimation framework with non-asymptotic risk bounds.
result Established non-asymptotic risk bounds for model selection in MoE models.
New diagonal knots found with non-torus structure.
problem Identifying knots with diagonal grid diagrams.
method Analysis of knots represented by diagonal grid diagrams.
result All diagonal knots are positive, and a new non-torus example is found.
Develops efficient quasi-Newton methods for training deep neural networks.
problem Training deep neural networks with large-scale Hessian matrices.
method Approximates Hessian as block-diagonal Kronecker product of smaller matrices, applies damping.
result Outperforms or matches state-of-the-art methods in autoencoder models.
We introduce the notion of Haantjes algebra: It consists of an assignment of a family of operator fields on a differentiable manifold, each of them with vanishing Haantjes torsion. They are also required to satisfy suitable compatibility conditions. Haantjes algebras naturally generalize several known interesting geome…
We define a second-order neural network stochastic gradient training algorithm whose block-diagonal structure effectively amounts to normalizing the unit activations. Investigating why this algorithm lacks in robustness then reveals two interesting insights. The first insight suggests a new way to scale the stepsizes, …
Sharp pseudospectral bounds prevent transient amplification in coupled gradient descent.
problem Transient amplification in coupled gradient descent systems.
method Developed a sharp pseudospectral theory for block-triangular Jacobians, proving Kreiss constant bounds and matching minimax lower bounds.
result Obtained a finite-horizon iteration-complexity bound of O(K(J)2log(1/δ)) for stochastic coupled descent. Study spectral flow on a warped cylinder with special boundary conditions.
problem Analyzing spectral flow on a warped cylinder with specific boundary conditions.
method Complexifying the twisting bundle, diagonalizing the orthogonal twist, and regrouping conjugate and reflection-paired blocks.
result Explicit formula for RO(O(2))-valued spectral flow, refining ordinary spectral flow. Parallelizes DEC on curved meshes using group actions.
problem Efficiently solving DEC operators on curved and 3D meshes.
method Universal block-diagonalization framework for d and ⋆ operators, exploiting group actions. result Block-diagonal structure inherited by operators, enabling parallel solvers.
This paper develops dimension-agnostic inference methods for high-dimensional data.
problem Understanding how classical inference methods behave in high-dimensional settings.
method Using variational representations, sample splitting, and self-normalization to create a refined test statistic.
result The resulting statistic has a Gaussian limiting distribution regardless of how dimensionality scales with sample size.
It is shown that, in four dimensions, it is possible to introduce coordinates so that an analytic metric locally takes block diagonal form. i.e. one can find coordinates such that gαβ=0 for (α,β)∈S where S=(1,3),(1,4),(2,3),(2,4). We call a coordinate system in which the metric takes this for…
Mini-Hes improves LFA model performance on HDI tasks with missing data.
problem Effective representation of high-dimensional, incomplete data for user behavior understanding.
method Proposes Mini-Hes, a parallelizable second-order LFA model using mini-block diagonal Hessian-free optimization.
result Mini-Hes outperforms state-of-the-art models in missing data estimation tasks on recommender system datasets.
New estimators reduce computation for Kendall's tau and conditional Kendall's tau matrices under structural assumptions.
problem Efficient estimation of Kendall's tau and conditional Kendall's tau matrices for large dimensions.
method Averaging pairwise estimates over blocks or conditional estimates, exploiting structural assumptions.
result Improved estimators with reduced computational cost and similar error level.
Study local structure of knot group representations into SL(n,C).
problem Understanding the local structure of knot group representations.
method Analysis of tangent cone and use of Luna's slice theorem.
result Local structure of representation variety at diagonal representations.
A new metric learning framework for signed graphs using Gershgorin disc alignment.
problem Learning Mahalanobis metrics from signed graphs efficiently.
method Proposes a fast metric learning framework using Gershgorin disc perfect alignment (GDPA) to circumvent full eigen-decomposition.
result Proves that Gershgorin disc left-ends of similarity transform are perfectly aligned at the smallest eigenvalue, enabling efficient optimization.
The paper identifies redundant columns in matrices for feature selection and clustering.
problem Identifying redundant columns in matrices for feature selection and clustering.
method Proves that after re-ordering columns, a matrix can be block-diagonalized revealing linearly dependent columns.
result Identifies redundant columns in matrices, aiding in feature selection and clustering.
Triangle groups show rigidity in hyperbolic spaces.
problem Local rigidity of triangle groups generated by reflections.
method Geometric representation and diagonal embeddings in PGL(2,R) and PSp±(2n,R). result Triangle groups are locally rigid in hyperbolic spaces.
We simplify matrix computations for block matrices, especially useful for covariance and correlation matrices.
problem Complex computations for block matrices, especially for covariance and correlation matrices.
method Obtained a canonical representation for block matrices, facilitating computation of various matrix operations.
result Simplified computation of matrix operations for block matrices, particularly useful for covariance and correlation matrices.
Develops large-sample theory for non-stationary source separation.
problem Lack of large-sample results for non-stationary source separation methods.
method Large-sample theory for NSS-JD method under specific assumptions.
result Consistency of unmixing estimator and its convergence to Gaussian distribution.
Advanced optimization algorithms such as Newton method and AdaGrad benefit from second order derivative or second order statistics to achieve better descent directions and faster convergence rates. At their heart, such algorithms need to compute the inverse or inverse square root of a matrix whose size is quadratic of …
Algorithm solves robust linear regression with block Lewis weights.
problem Group distributionally robust least squares problem.
method Algorithm based on geometric construction and block Lewis weights, using accelerated proximal methods.
result Improves over known methods for moderate accuracy regimes and matches state-of-the-art guarantees.
Constructs six-dimensional braid group representations for knot detection.
problem Detecting braid vertibility of knots and links.
method Constructs six-dimensional block representations of B3 and uses them to detect braid vertibility. result Some representations can detect braid vertibility of known knots and links.
It is challenging to develop stochastic gradient based scalable inference for deep discrete latent variable models (LVMs), due to the difficulties in not only computing the gradients, but also adapting the step sizes to different latent factors and hidden layers. For the Poisson gamma belief network (PGBN), a recently …
Motivated by large-scale Collaborative-Filtering applications, we present a Non-Commuting Latent Factor (NCLF) tensor-completion approach for modeling three-way arrays, which is diagonal like the standard PARAFAC, but wherein different terms distinguish different kinds of three-way relations of co-clusters, as determin…
We present a method based on the orthogonal symmetric non-negative matrix tri-factorization of the normalized Laplacian matrix for community detection in complex networks. While the exact factorization of a given order may not exist and is NP hard to compute, we obtain an approximate factorization by solving an optimiz…