A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
MOBO-OSD optimizes multi-objective functions using orthogonal search directions.
In this paper, we propose the use of a black-box optimization method called deterministic Mesh Adaptive Direct Search (MADS) algorithm with orthogonal directions (Ortho-MADS) for the selection of hyperparameters of Support Vector Machines with a Gaussian kernel. Different from most of the methods in the literature that…
Scientific discovery is limited by hypothesis redundancy, and hybrid methods can exploit non-local exploration.
We introduce Block Sparse Canonical Correlation Analysis which estimates multiple pairs of canonical directions (together a "block") at once, resulting in significantly improved orthogonality of the sparse directions which, we demonstrate, translates to more interpretable solutions. Our approach builds on the sparse CC…
Inverted file and asymmetric distance computation (IVFADC) have been successfully applied to approximate nearest neighbor search and subsequently maximum inner product search. In such a framework, vector quantization is used for coarse partitioning while product quantization is used for quantizing residuals. In the ori…
A new method reduces complexity for optimizing large-scale problems with orthogonality constraints.
The paper finds non-Gaussian directions in high-dimensional data using Wasserstein distance.
A new hashing method improves accuracy by learning an orthogonal transform.
New method selects direct causal parents from large sets of variables.
SOLAR improves search efficiency and accuracy with sparse, orthogonal embeddings.
In this paper, we consider derivative free optimization problems, where the objective function is smooth but is computed with some amount of noise, the function evaluations are expensive and no derivative information is available. We are motivated by policy optimization problems in reinforcement learning that have rece…
Geodesics in symmetrized bidisc have intrinsic orthogonality and distinguished directions.
In optimization, the negative gradient of a function denotes the direction of steepest descent. Furthermore, traveling in any direction orthogonal to the gradient maintains the value of the function. In this work, we show that these orthogonal directions that are ignored by gradient descent can be critical in equilibri…
New stochastic gradient descent with random search directions improves efficiency and convergence.
We classify six-dimensional Lie groups which admit a left-invariant half-flat SU(3)-structure and which split in a direct product of three-dimensional factors. Moreover, a complete list of those direct products is obtained which admit a left-invariant half-flat SU(3)-structure such that the three-dimensional factors ar…
A new knot invariant measures crossings in three orthogonal directions.
The problem of inferring the direct causal parents of a response variable among a large set of explanatory variables is of high practical importance in many disciplines. Recent work exploits stability of regression coefficients or invariance properties of models across different experimental conditions for reconstructi…
A new pruning method reduces neural network computation without retraining.
This paper investigates the use of multiple directions of stratification as a variance reduction technique for Monte Carlo simulations of path-dependent options driven by Gaussian vectors. The precision of the method depends on the choice of the directions of stratification and the allocation rule within each strata. S…
Equivalent tests for SGD batch size selection found.
Introduces new curvature concept for Kähler manifolds.
Early stopping method saves up to 75% computation time in policy search tasks.
A new algorithm avoids retractions to optimize orthogonal matrices efficiently.
This work proposes searching for optimal operation distribution in neural architecture search.
The vast majority of successful deep neural networks are trained using variants of stochastic gradient descent (SGD) algorithms. Recent attempts to improve SGD can be broadly categorized into two approaches: (1) adaptive learning rate schemes, such as AdaGrad and Adam, and (2) accelerated schemes, such as heavy-ball an…
Paper develops online statistical inference methods for stochastic optimization using Kiefer-Wolfowitz algorithms.
SONIA optimizes machine learning problems with a novel algorithm.
By this short preface we show the main idea and we will bring some definitions and concepts in each section.
Deep learning algorithms have achieved excellent performance lately in a wide range of fields (e.g., computer version). However, a severe challenge faced by deep learning is the high dependency on hyper-parameters. The algorithm results may fluctuate dramatically under the different configuration of hyper-parameters. A…
Introduces MSW distances to improve SW metrics.
New framework detects directional influence in multivariate time series.
New findings on Kähler manifolds restrict orthogonal coordinates existence.
LOFT separates subspace rotation and transformation for orthogonal fine-tuning.
Yau's Affine Normal Descent optimizes smooth unconstrained problems with geometrically adapted directions.
This letter presents a new spectral-clustering-based approach to the subspace clustering problem. Underpinning the proposed method is a convex program for optimal direction search, which for each data point d finds an optimal direction in the span of the data that has minimum projection on the other data points and non…
Choosing appropriate step sizes is critical for reducing the computational cost of training large-scale neural network models. Mini-batch sub-sampling (MBSS) is often employed for computational tractability. However, MBSS introduces a sampling error, that can manifest as a bias or variance in a line search. This is bec…
This paper provides a block coordinate descent algorithm to solve unconstrained optimization problems. In our algorithm, computation of function values or gradients is not required. Instead, pairwise comparison of function values is used. Our algorithm consists of two steps; one is the direction estimate step and the o…
New differential geometry perspective on orthogonal RNNs.
Paper finds conditions for benign overfitting in neural networks.
We compute approximate solutions to L0 regularized linear regression using L1 regularization, also known as the Lasso, as an initialization step. Our algorithm, the Lass-0 ("Lass-zero"), uses a computationally efficient stepwise search to determine a locally optimal L0 solution given any L1 regularization solution. We …
Modified Laplacian connects to Yang-Mills instantons on manifolds.
NeuralArTS categorizes neural ops in a type system for NAS.
We analyse the fine convergence properties of one parameter families of hyperbolic metrics, on a fixed underlying surface, that move always in a horizontal direction, i.e. orthogonal to the action of diffeomorphisms.
A new method optimizes neural sequence models for better task performance.
Learning rates in stochastic neural network training are currently determined a priori to training, using expensive manual or automated iterative tuning. This study proposes gradient-only line searches to resolve the learning rate for neural network training algorithms. Stochastic sub-sampling during training decreases…
Neural Architecture Search (NAS) enabled the discovery of state-of-the-art architectures in many domains. However, the success of NAS depends on the definition of the search space. Current search spaces are defined as a static sequence of decisions and a set of available actions for each decision. Each possible sequenc…
The radar experiment connects the geometry of spacetime with an observers measurement of spatial length. We investigate the radar experiment on Finsler spacetimes which leads to a general definition of radar orthogonality and radar length. The directions radar orthogonal to an observer form the spatial equal time surfa…
Improves RL planning by proposing sub-goals hierarchically.