arXiv research

New derivation shows how a three-factor learning rule is derived from Oja's rule.

Oja's rule improves neural network training without engineered tricks.

In this paper we are interested in the problem of learning an over-complete basis and a methodology such that the reconstruction or inverse problem does not need optimization. We analyze the optimality of the presented approaches, their link to popular already known techniques s.a. Artificial Neural Networks,k-means or…

Oja's algorithm has been the cornerstone of streaming methods in Principal Component Analysis (PCA) since it was first proposed in 1982. However, Oja's algorithm does not have a standardized choice of learning rate (step size) that both performs well in practice and truly conforms to the online streaming setting. In th…

A new single-pass algorithm improves sparse PCA under limited computational resources.

We quantify uncertainty in Oja's algorithm's leading eigenvector estimation.

In this paper, we propose to adopt the diffusion approximation tools to study the dynamics of Oja's iteration which is an online stochastic gradient descent method for the principal component analysis. Oja's iteration maintains a running estimate of the true principal component from streaming data and enjoys less tempo…

Paper improves Oja's algorithm for Markovian data streams.

Low-precision streaming PCA estimates the leading eigenvector with limited precision.

In this paper, we study the problems of principal Generalized Eigenvector computation and Canonical Correlation Analysis in the stochastic setting. We propose a simple and efficient algorithm, Gen-Oja, for these problems. We prove the global convergence of our algorithm, borrowing ideas from the theory of fast-mixing M…

EigenGame reinterprets PCA as a game to find eigenvectors.

We study streaming principal component analysis (PCA), that is to find, in $O(dk)$ space, the top $k$ eigenvectors of a $d\times d$ hidden matrix $\bf Σ$ with online vectors drawn from covariance matrix $\bf Σ$. We provide $\textit{global}$ convergence for Oja's algorithm which is popularly used in practice but lacks t…

This work provides improved guarantees for streaming principle component analysis (PCA). Given $A_1, \ldots, A_n\in \mathbb{R}^{d\times d}$ sampled independently from distributions satisfying $\mathbb{E}[A_i] = Σ$ for $Σ\succeq \mathbf{0}$, this work provides an $O(d)$-space linear-time single-pass streaming algorithm …

A neuron is a basic physiological and computational unit of the brain. While much is known about the physiological properties of a neuron, its computational role is poorly understood. Here we propose to view a neuron as a signal processing device that represents the incoming streaming data matrix as a sparse vector of …

We present a high-dimensional analysis of three popular algorithms, namely, Oja's method, GROUSE and PETRELS, for subspace estimation from streaming and highly incomplete observations. We show that, with proper time scaling, the time-varying principal angles between the true subspace and its estimates given by the algo…

We study the statistical and computational aspects of kernel principal component analysis using random Fourier features and show that under mild assumptions, $O(\sqrt{n} \log n)$ features suffices to achieve $O(1/ε^2)$ sample complexity. Furthermore, we give a memory efficient streaming algorithm based on classical Oja…

pPCA speeds up PCA by priming initial estimates for faster, more accurate results.

We shed new insights on the two commonly used updates for the online $k$-PCA problem, namely, Krasulina's and Oja's updates. We show that Krasulina's update corresponds to a projected gradient descent step on the Stiefel manifold of the orthonormal $k$-frames, while Oja's update amounts to a gradient descent step using…

We consider a situation in which we see samples in $\mathbb{R}^d$ drawn i.i.d. from some distribution with mean zero and unknown covariance A. We wish to compute the top eigenvector of A in an incremental fashion - with an algorithm that maintains an estimate of the top eigenvector in O(d) space, and incrementally adju…

New algorithm learns principal subspace from random samples.

We consider streaming principal component analysis when the stochastic data-generating model is subject to perturbations. While existing models assume a fixed covariance, we adopt a robust perspective where the covariance matrix belongs to a temporal uncertainty set. Under this setting, we provide fundamental limits on…

Stochastic optimization naturally arises in machine learning. Efficient algorithms with provable guarantees, however, are still largely missing, when the objective function is nonconvex and the data points are dependent. This paper studies this fundamental challenge through a streaming PCA problem for stationary time s…

Principal Component Analysis is a novel way of of dimensionality reduction. This problem essentially boils down to finding the top k eigen vectors of the data covariance matrix. A considerable amount of literature is found on algorithms meant to do so such as an online method be Warmuth and Kuzmin, Matrix Stochastic Gr…

We study the problem of recovering the subspace spanned by the first $k$ principal components of $d$-dimensional data under the streaming setting, with a memory bound of $O(kd)$. Two families of algorithms are known for this problem. The first family is based on the framework of stochastic gradient descent. Nevertheles…

AgFlow speeds up model selection in penalized PCA.

A new method for streaming PCA provides confidence intervals for eigenvector entries.

DeepCTRL integrates rules into deep learning models, allowing flexible control at inference.

New methods prune unpromising rules from KGs, improving scalability and runtime.

In this study, we give the relationships between the conical curvatures of ruled surfaces drawn by the unit vectors of the ruling, central normal and central tangent of a regular ruled surface in the Euclidean -space. We obtain the differential equations characterizing slant ruled surfaces and if the reference ruled su…

New framework learns interpretable rule ensembles without sacrificing accuracy.

R2N learns interpretable rules and literals from numerical features.

Study ruled surfaces with finite multiplicity, focusing on their curves and singularities.

Paper classifies ruled surfaces in Lorentz-Minkowski space for a specific flow.

In this study, we define some new types of ruled surfaces called slant ruled surfaces. We give some characterizations for a regular ruled surface to be a slant ruled surface in Euclidean 3- space. We show that if the slant ruled surface is developable then the striction curve is a general helix or a slant helix accordi…

In this paper we use fuzzy systems theory to convert the technical trading rules commonly used by stock practitioners into excess demand functions which are then used to drive the price dynamics. The technical trading rules are recorded in natural languages where fuzzy words and vague expressions abound. In Part I of t…

Subdivision rules create sequences of nested cell structures on CW-complexes, and they frequently arise from groups. In this paper, we develop several tools for classifying subdivision rules. We give a criterion for a subdivision rule to represent a Gromov hyperbolic space, and show that a subdivision rule for a hyperb…

In this article supervised learning problems are solved using soft rule ensembles. We first review the importance sampling learning ensembles (ISLE) approach that is useful for generating hard rules. The soft rules are then obtained with logistic regression from the corresponding hard rules. In order to deal with the p…

Advances rule-based multi-label classification using conformal prediction.

In this paper, we consider non developable ruled surface with spacelike ruling, timelike ruling, respectively. We give the relations between the structure functions with the curvature and torsion of the striction line of the timelike and spacelike non developable ruled surfaces. Also, we have calculated the gaussian an…

The Lamarle Formula, given by Kruppa in \cite{Kr}, is known as a relationship between the Gaussian curvature and the distribution parameter of a ruled surface in the surface theory. The ruled surfaces were investigated in 3 different classes with respect to the character of base curves and rulings, \cite{Tu1},\cite{Tu2…

In this study, we introduce Darboux slant ruled surfaces in the Euclidean 3-space which is defined by the property that the Darboux vector of orthonormel frame of ruled surface makes a constant angle with a fixed, non-zero direction. We obtain the characterizations of Darboux slant ruled surfaces regarding the conical …

Generative models learn rules at different timescales, revealing a 'innovation window'.

In this study, we define some new types of non-null ruled surfaces called slant ruled surfaces in the Minkowski 3-space E_1^3. We introduce some characterizations for a non-null ruled surface to be a slant ruled surface in E_1^3. Moreover, we obtain some corollaries which give the relationships between a non-null slant…

FedRule uses graph neural networks to recommend rules for smart homes without centralizing data.

Paper explores folding patterns of curved creases preserving their geometric properties.

Study on hypersurfaces with minimized distance between rulings.

NeuRules learns interpretable rule lists from data without pre-discretization.

Study embeds ruled surfaces into symplectic manifolds, finds Stein fillability results.