arXiv research

Many problems in machine learning and statistics can be formulated as (generalized) eigenproblems. In terms of the associated optimization problem, computing linear eigenvectors amounts to finding critical points of a quadratic function subject to quadratic constraints. In this paper we show that a certain class of con…

A new clustering method for simplicial complexes using homology.

We introduce a principled and theoretically sound spectral method for $k$-way clustering in signed graphs, where the affinity measure between nodes takes either positive or negative values. Our approach is motivated by social balance theory, where the task of clustering aims to decompose the network into disjoint group…

It has been recently shown that a large class of balanced graph cuts allows for an exact relaxation into a nonlinear eigenproblem. We review briefly some of these results and propose a family of algorithms to compute nonlinear eigenvectors which encompasses previous work as special cases. We provide a detailed analysis…

New moving average adapts weight dynamically based on polynomial and wavefunction.

New method scales features for better clustering.

The Riemann curvature tensor is a central mathematical tool in Einstein's theory of general relativity. Its related eigenproblem plays an important role in mathematics and physics. We extend M-eigenvalues for the elasticity tensor to the Riemann curvature tensor. The definition of M-eigenproblem of the Riemann curvatur…

Eigenvalues of Steklov eigenproblems change predictably with boundary tweaks.

Regularized spectral methods improve clustering in signed graphs, especially for sparse data.

Lower bounds for Dirac eigenvalues on manifolds with boundary.

The execution flow drives market dynamics, validated on real data.

An important application of Lebesgue integral quadrature arXiv:1807.06007 is developed. Given two random processes, $f(x)$ and $g(x)$, two generalized eigenvalue problems can be formulated and solved. In addition to obtaining two Lebesgue quadratures (for $f$ and $g$) from two eigenproblems, the projections of $f$- and…

We discuss the portfolio optimization problem with the obligatory deposits constraint. Recently it has been shown that as a consequence of this nonlinear constraint, the solution consists of an exponentially large number of optimal portfolios, completely different from each other, and extremely sensitive to any changes…

Additive principal components (APCs for short) are a nonlinear generalization of linear principal components. We focus on smallest APCs to describe additive nonlinear constraints that are approximately satisfied by the data. Thus APCs fit data with implicit equations that treat the variables symmetrically, as opposed t…

New method improves subspace iteration for eigenvectors in machine learning.

Two adaptive kernel selection methods improve the accuracy of Kernelized Diffusion Maps.

New asymmetric kernel methods improve feature learning.

PCA++ improves robustness to background noise in contrastive learning.

Problems of interpolation, classification, and clustering are considered. In the tenets of Radon--Nikodym approach $\langle f(\mathbf{x})ψ^2 \rangle / \langleψ^2\rangle$, where the $ψ(\mathbf{x})$ is a linear function on input attributes, all the answers are obtained from a generalized eigenproblem $|f|ψ^{[i]}\rangle =…

Unified framework for scale-invariant representation learning using MAPCA.

Sparse Hopfield model improves memory retrieval with fewer connections.

New quantum code lacks sparse lift.

In this paper, a sparse Markov decision process (MDP) with novel causal sparse Tsallis entropy regularization is proposed.The proposed policy regularization induces a sparse and multi-modal optimal policy distribution of a sparse MDP. The full mathematical analysis of the proposed sparse MDP is provided.We first analyz…

This work introduces a method to compare sparse neural network topologies using graph theory.

Sparse-RS framework efficiently attacks models with sparse perturbations.

Using a Bayesian approach, we consider the problem of recovering sparse signals under additive sparse and dense noise. Typically, sparse noise models outliers, impulse bursts or data loss. To handle sparse noise, existing methods simultaneously estimate the sparse signal of interest and the sparse noise of no interest.…

Sparse deep neural networks(DNNs) are efficient in both memory and compute when compared to dense DNNs. But due to irregularity in computation of sparse DNNs, their efficiencies are much lower than that of dense DNNs on regular parallel hardware such as TPU. This inefficiency leads to poor/no performance benefits for s…

Sparse DNNs face scalability issues; MIT/IEEE/Amazon challenge analyzes best solutions.

In compressed sensing, we wish to reconstruct a sparse signal $x$ from observed data $y$. In sparse coding, on the other hand, we wish to find a representation of an observed signal $y$ as a sparse linear combination, with coefficients $x$, of elements from an overcomplete dictionary. While many algorithms are competit…

Sparse coding approximates the data sample as a sparse linear combination of some basic codewords and uses the sparse codes as new presentations. In this paper, we investigate learning discriminative sparse codes by sparse coding in a semi-supervised manner, where only a few training samples are labeled. By using the m…

Dynamic Sparse Training finds efficient sparse networks from scratch.

New method for hyperparameter tuning in sparse matrix factorization.

We demonstrate the possibility of what we call sparse learning: accelerated training of deep neural networks that maintain sparse weights throughout training while achieving dense performance levels. We accomplish this by developing sparse momentum, an algorithm which uses exponentially smoothed gradients (momentum) to…

High dimensional sparse learning has imposed a great computational challenge to large scale data analysis. In this paper, we are interested in a broad class of sparse learning approaches formulated as linear programs parametrized by a {\em regularization factor}, and solve them by the parametric simplex method (PSM). O…

This paper explores loss landscapes of sparse neural networks, finding unique characteristics compared to dense networks.

Sparse APCA identifies sparse factors in financial returns over time.

Study confirms sparse coding in whole brain using MRI data.

A new method for sparse PCA using orthogonal rotations and soft-thresholding.

Develops sparse portfolio strategy for high-dimensional assets.

Picasso is a new library for sparse learning problems in R and Python.

The CUR decomposition provides an approximation of a matrix $X$ that has low reconstruction error and that is sparse in the sense that the resulting approximation lies in the span of only a few columns of $X$. In this regard, it appears to be similar to many sparse PCA methods. However, CUR takes a randomized algorithm…

DFR reduces the computational cost of sparse-group lasso and adaptive sparse-group lasso.

A variety of representation learning approaches have been investigated for reinforcement learning; much less attention, however, has been given to investigating the utility of sparse coding. Outside of reinforcement learning, sparse coding representations have been widely used, with non-convex objectives that result in…

Guarantees sparse recovery for neural networks with iterative hard thresholding.

New methods show sparse portfolios offer no advantage over mean-variance in diversification.

Computations for the softmax function are significantly expensive when the number of output classes is large. In this paper, we present a novel softmax inference speedup method, Doubly Sparse Softmax (DS-Softmax), that leverages sparse mixture of sparse experts to efficiently retrieve top-k classes. Different from most…

msPCA solves sparse PCA for multiple components efficiently.

We consider the scenario where one observes an outcome variable and sets of features from multiple assays, all measured on the same set of samples. One approach that has been proposed for dealing with this type of data is ``sparse multiple canonical correlation analysis'' (sparse mCCA). All of the current sparse mCCA t…