A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
New analysis enables inversion of deep generative models with unique solutions.
Proposes QEP to mitigate quantization error propagation in layer-wise post-training quantization.
Layer-wise preconditioning methods improve neural network optimization and feature learning.
Unified framework LPCD optimizes quantization of complex submodules.
Layer-wise networks have a closed-form solution and a stopping criterion.
Layer-wise networks have a closed-form solution and a stopping criterion.
Isometry pursuit identifies orthonormal submatrices from wide matrices.
We study a form of cyclic pursuit on Riemannian manifolds with positive injectivity radius. We conjecture that on a compact manifold, the piecewise geodesic loop formed by connecting consecutive pursuit agents either collapses in finite time or converges to a closed geodesic. The main result is that this conjecture is …
Study of pursuit-evasion game on sphere and its relation to planar Apollonius circle.
This paper closes the gap on matching pursuit's convergence rate.
In this paper, we tackle the problem of explanations in a deep-learning based model for recommendations by leveraging the technique of layer-wise relevance propagation. We use a Deep Convolutional Neural Network to extract relevant features from the input images before identifying similarity between the images in featu…
LEWIS merges LLMs without training, improving performance on specific tasks.
A fundamental question in deep learning concerns the role played by individual layers in a deep neural network (DNN) and the transferable properties of the data representations which they learn. To the extent that layers have clear roles, one should be able to optimize them separately using layer-wise loss functions. S…
Random layer-wise pruning profiles are as effective as metric-based ones for various datasets.
Pursuit-evasion is a multi-agent sequential decision problem wherein a group of agents known as pursuers coordinate their traversal of a spatial domain to locate an agent trying to evade them. Pursuit evasion problems arise in a number of import application domains including defense and route planning. Learning to opti…
Two popular examples of first-order optimization methods over linear spaces are coordinate descent and matching pursuit algorithms, with their randomized variants. While the former targets the optimization by moving along coordinates, the latter considers a generalized notion of directions. Exploiting the connection be…
Generative ML learns optimal pursuit trajectories in pursuit-evasion games.
Matching pursuit algorithms are an important class of algorithms in signal processing and machine learning. We present a blended matching pursuit algorithm, combining coordinate descent-like steps with stronger gradient descent steps, for minimizing a smooth convex function over a linear space spanned by a set of atoms…
The paper finds non-Gaussian directions in high-dimensional data using Wasserstein distance.
In this correspondence, we obtain exact recovery conditions for regularized modified basis pursuit (reg-mod-BP) and discuss when the obtained conditions are weaker than those for modified-CS or for basis pursuit (BP). The discussion is also supported by simulation comparisons. Reg-mod-BP provides a solution to the spar…
We propose a new optimization method for training feed-forward neural networks. By rewriting the activation function as an equivalent proximal operator, we approximate a feed-forward neural network by adding the proximal operators to the objective function as penalties, hence we call the lifted proximal operator machin…
Analyzes layer-wise quantization effects in neural networks.
Automatically learns flexible symmetry constraints in neural networks using gradients.
Insurance firms use RL to optimize customer offers for desired target portfolios.
Enhances projection pursuit tree classifier with visual diagnostics for better multi-class classification.
Pre-training is crucial for learning deep neural networks. Most of existing pre-training methods train simple models (e.g., restricted Boltzmann machines) and then stack them layer by layer to form the deep structure. This layer-wise pre-training has found strong theoretical foundation and broad empirical support. Howe…
A new method improves few-shot image classification by updating top layers.
Product models of low dimensional experts are a powerful way to avoid the curse of dimensionality. We present the ``under-complete product of experts' (UPoE), where each expert models a one dimensional projection of the data. The UPoE is fully tractable and may be interpreted as a parametric probabilistic model for pro…
Sparse Subspace Clustering (SSC) is a state-of-the-art method for clustering high-dimensional data points lying in a union of low-dimensional subspaces. However, while optimization-based SSC algorithms suffer from high computational complexity, other variants of SSC, such as Orthogonal Matching Pursuit-based S…
Recently, the principal component pursuit has received increasing attention in signal processing research ranging from source separation to video surveillance. So far, all existing formulations are real-valued and lack the concept of phase, which is inherent in inputs such as complex spectrograms or color images. Thus,…
Orthogonal Matching Pursuit (OMP) has long been considered a powerful heuristic for attacking compressive sensing problems; however, its theoretical development is, unfortunately, somewhat lacking. This paper presents an improved Restricted Isometry Property (RIP) based performance guarantee for T-sparse signal reconst…
Sharp results link DLN gradient flow to basis pursuit optimization and GHA phase transitions.
Two of the most fundamental prototypes of greedy optimization are the matching pursuit and Frank-Wolfe algorithms. In this paper, we take a unified view on both classes of methods, leading to the first explicit convergence rates of matching pursuit methods in an optimization sense, for general sets of atoms. We derive …
Paper proposes a novel approach to density ratio estimation using projection pursuit.
Speeds up deep neural networks training by 10x using GPU concurrency.
Study on shortcuts in deep networks, revealing their layer-wise distribution and impact.
The problem of pursuing a moving target is always one of the main topics in navigation. In the literatures, there are two well-known algorithms called Pure Pursuit and Pure Rendezvous navigation in the 3-dimensional space . In this paper, these two methods are combined to introduce a novel family of pursu…
A geometric approach to differential game theory is illustrated. The parallel pursuit is considered as a two-player zero-sum differential game. The optimal strategies of each player is designed based on Riemann-Finsler geometry. Our approach incorporates a closed loop optimal control and the presentation is familiar wi…
Optimal Control Theory optimizes neural networks, improving robustness and efficiency.
Projection pursuit model improves Gaussian process regression for high-dimensional data.
We propose NovoGrad, an adaptive stochastic gradient descent method with layer-wise gradient normalization and decoupled weight decay. In our experiments on neural networks for image classification, speech recognition, machine translation, and language modeling, it performs on par or better than well tuned SGD with mom…
In this paper, we explore the limitations of PCA as a dimension reduction technique and study its extension, projection pursuit (PP), which is a broad class of linear dimension reduction methods. We first discuss the relevant concepts and theorems and then apply PCA and PP (with negative standardized Shannon's entropy …
This work analyzes how different layers in deep neural networks contribute to generalization error.
OOMP selects features online for sparse linear regression.
Paper proposes PPMM for fast estimation of large-scale OTM.
This paper presents a new ensemble learning method for classification problems called projection pursuit random forest (PPF). PPF uses the PPtree algorithm introduced in Lee et al. (2013). In PPF, trees are constructed by splitting on linear combinations of randomly chosen variables. Projection pursuit is used to choos…
Lower bounds show OLS outperforms basis pursuit in overparameterized linear regression.
We introduce a new convex formulation for stable principal component pursuit (SPCP) to decompose noisy signals into low-rank and sparse representations. For numerical solutions of our SPCP formulation, we first develop a convex variational framework and then accelerate it with quasi-Newton methods. We show, via synthet…