arXiv research

Adaptive reduction scheme approximates optimal policy in regularized MDPs.

New algorithm reduces optimization complexity in adaptive mirror descent.

Improved variance reduction for Riemannian non-convex optimization with adaptive batch size.

Ant colonies and boosting algorithms both reduce bias and variance through adaptive mechanisms.

Adaptive framework improves nonparametric dimensionality reduction.

Adapts DR objectives for both sample and feature size reduction.

Adaptive importance sampling for stochastic optimization is a promising approach that offers improved convergence through variance reduction. In this work, we propose a new framework for variance reduction that enables the use of mixtures over predefined sampling distributions, which can naturally encode prior knowledg…

DFR reduces the computational cost of sparse-group lasso and adaptive sparse-group lasso.

In this paper, we propose a novel lower dimensional representation of a shape sequence. The proposed dimension reduction is invertible and computationally more efficient in comparison to other related works. Theoretically, the differential geometry tools such as moving frame and parallel transportation are successfully…

Dimensionality reduction is an important operation in information visualization, feature extraction, clustering, regression, and classification, especially for processing noisy high dimensional data. However, most existing approaches preserve either the global or the local structure of the data, but not both. Approache…

AdaSVRG combines adaptive gradient with SVRG for robust optimization.

Deep neural networks are vulnerable to adversarial examples, i.e., carefully-perturbed inputs aimed to mislead classification. This work proposes a detection method based on combining non-linear dimensionality reduction and density estimation techniques. Our empirical findings show that the proposed approach is able to…

New risk factors improve stress testing accuracy.

We study adaptive data-dependent dimensionality reduction in the context of supervised learning in general metric spaces. Our main statistical contribution is a generalization bound for Lipschitz functions in metric spaces that are doubling, or nearly doubling. On the algorithmic front, we describe an analogue of PCA f…

This paper explores how effective sample size, dimensionality, and model performance are related in covariate shift adaptation.

Bayesian optimization reduces hyperparameters for mixed variable design problems.

We present an adaptive approach for valuing the European call option on assets with stochastic volatility. The essential feature of the method is a reduction of uncertainty in latent volatility due to a Bayesian learning procedure. Starting from a discrete-time stochastic volatility model, we derive a recurrence equati…

A general study of symmetries in optimal control theory is given, starting from the presymplectic description of this kind of system. Then, Noether's theorem, as well as the corresponding reduction procedure (based on the application of the Marsden-Weinstein theorem adapted to the presymplectic case) are stated both in…

Scalability of statistical estimators is of increasing importance in modern applications and dimension reduction is often used to extract relevant information from data. A variety of popular dimension reduction approaches can be framed as symmetric generalized eigendecomposition problems. In this paper we outline how t…

A new deep neural network tackles nonlinear functional regression with improved dimensionality reduction.

The scalability of statistical estimators is of increasing importance in modern applications. One approach to implementing scalable algorithms is to compress data into a low dimensional latent space using dimension reduction methods. In this paper we develop an approach for dimension reduction that exploits the assumpt…

Nonparametric adaptive robust control tackles model uncertainty in stochastic processes.

We present the first adaptive strategy for active learning in the setting of classification with smooth decision boundary. The problem of adaptivity (to unknown distributional parameters) has remained opened since the seminal work of Castro and Nowak (2007), which first established (active learning) rates for this sett…

Adaptive Monte Carlo methods are recent variance reduction techniques. In this work, we propose a mathematical setting which greatly relaxes the assumptions needed by for the adaptive importance sampling techniques presented by Vazquez-Abad and Dufresne, Fu and Su, and Arouna. We establish the convergence and asymptoti…

This paper discusses the exact simulation of the stock price process underlying the 3/2 model. Using a result derived by Craddock and Lennox using Lie Symmetry Analysis, we adapt the Broadie-Kaya algorithm for the simulation of affine processes to the 3/2 model. We also discuss variance reduction techniques and find th…

We introduce several new black-box reductions that significantly improve the design of adaptive and parameter-free online learning algorithms by simplifying analysis, improving regret guarantees, and sometimes even improving runtime. We reduce parameter-free online learning to online exp-concave optimization, we reduce…

Generative adversarial networks benefit from optimal input dimension and adaptive generator architecture.

We present local discriminative Gaussian (LDG) dimensionality reduction, a supervised dimensionality reduction technique for classification. The LDG objective function is an approximation to the leave-one-out training error of a local quadratic discriminant analysis classifier, and thus acts locally to each training po…

New neural network method simplifies high-dimensional data.

Stochastic convex optimization algorithms are the most popular way to train machine learning models on large-scale data. Scaling up the training process of these models is crucial, but the most popular algorithm, Stochastic Gradient Descent (SGD), is a serial method that is surprisingly hard to parallelize. In this pap…

Let $P$ be a parabolic subgroup of a connected simply connected complex semisimple Lie group $G$. Given a compact Kähler manifold $X$, the dimensional reduction of $G$-equivariant holomorphic vector bundles over $X\times G/P$ was carried out by the first and third authors. This raises the question of dimensional reduct…

The main goal of this work is equipping convex and nonconvex problems with Barzilai-Borwein (BB) step size. With the adaptivity of BB step sizes granted, they can fail when the objective function is not strongly convex. To overcome this challenge, the key idea here is to bridge (non)convex problems and strongly convex …

Flag manifolds are in general not symmetric spaces. But they are provided with a structure of $\mathbb{Z}_2^k$-symmetric space. We describe the Riemannian metrics adapted to this structure and some properties of reducibility. We detail for the flag manifold $SO(5)/SO(2)\times SO(2) \times SO(1)$ what are the conditions…

A characterization of maximal domains of existence of adapted complex structures for Riemannian homogeneous manifolds under certain extensibility assumptions on their geodesic flow is given. This is applied to generalized Heisenberg groups and naturally reductive Riemannian homogeneous spaces. As an application it is s…

New algorithm reduces TV-denoising to adaptive online learning.

Paper reduces expensive financial risk simulations through efficient MOR.

In this paper we propose the macroblock scaling (MBS) algorithm, which can be applied to various CNN architectures to reduce their model size. MBS adaptively reduces each CNN macroblock depending on its information redundancy measured by our proposed effective flops. Empirical studies conducted with ImageNet and CIFAR-…

JADAI optimizes design and inference for parameter estimation.

C-ADAM is a new adaptive solver for complex nested problems.

MARS optimizes large model training by reducing variance, outperforming AdamW.

Improves MARS for nonparametric multivariate regression with dimension reduction.

Paper speeds up large foundation models for time series data.

New method for multi-modal depth prediction challenges.

Reduces path integrals for interacting systems using dependent coordinates.

Mime algorithm improves federated learning by adapting centralized methods.

Paper improves PINNs' extrapolation by TL and adaptive AFs.

Proposes a flexible feature allocation model for sparse factor analysis.

The principal support vector machines method (Li et al., 2011) is a powerful tool for sufficient dimension reduction that replaces original predictors with their low-dimensional linear combinations without loss of information. However, the computational burden of the principal support vector machines method constrains …