arXiv research

Data-driven optimization improves mean-variance portfolios by penalizing norms.

Improved GP bandit algorithms for noiseless, varying noise, and RKHS norms.

A new method detects anomalies in multivariate streams without unit dependence.

Noise injection before gradient steps helps in regularization for neural networks.

AdaGrad-Norm achieves optimal convergence rates for non-convex objectives without tuning.

New algorithms reduce regret in both stochastic and deterministic environments.

New methods solve graph sparsity optimization problems faster.

Extends Mahalanobis distance to Banach spaces for anomaly detection.

New algorithms solve stochastic variational inequalities without bounded variance assumption.

We consider the problem of distributed mean estimation (DME), in which $n$ machines are each given a local $d$-dimensional vector $x_v \in \mathbb{R}^d$, and must cooperate to estimate the mean of their inputs $μ= \frac 1n\sum_{v = 1}^n x_v$, while minimizing total communication cost. DME is a fundamental construct in …

In this short report, we discuss how coordinate-wise descent algorithms can be used to solve minimum variance portfolio (MVP) problems in which the portfolio weights are constrained by $l_{q}$ norms, where $1\leq q \leq 2$. A portfolio which weights are regularised by such norms is called a sparse portfolio (Brodie et …

The paper relaxes assumptions for analyzing stochastic optimization algorithms.

A new ratio, the Hansen ratio, simplifies mean-variance portfolio theory.

We address the problem of defining a group sparse formulation for Principal Components Analysis (PCA) - or its equivalent formulations as Low Rank approximation or Dictionary Learning problems - which achieves a compromise between maximizing the variance explained by the components and promoting sparsity of the loading…

Deep linear networks can closely approximate interpolants without improving risk.

The study analyzes how covariance estimation errors affect the global minimum-variance portfolio under heavy-tailed distributions.

GRAIN: Group Aggregation via Min-Norm Objective

Principal component analysis (PCA) is often used to reduce the dimension of data by selecting a few orthonormal vectors that explain most of the variance structure of the data. L1 PCA uses the L1 norm to measure error, whereas the conventional PCA uses the L2 norm. For the L1 PCA problem minimizing the fitting error of…

SignSVRG improves SignSGD by reducing variance, achieving similar convergence rates.

Recently, the \textit{Tensor Nuclear Norm~(TNN)} regularization based on t-SVD has been widely used in various low tubal-rank tensor recovery tasks. However, these models usually require smooth change of data along the third dimension to ensure their low rank structures. In this paper, we propose a new definition of da…

New method tightens sub-Gaussian concentration inequalities.

New algorithms adapt to both gradient norms and comparator norms in online learning.

It has been noted in existing literature that over-parameterization in ReLU networks generally improves performance. While there could be several factors involved behind this, we prove some desirable theoretical properties at initialization which may be enjoyed by ReLU networks. Specifically, it is known that He initia…

Given a multivariate data set, sparse principal component analysis (SPCA) aims to extract several linear combinations of the variables that together explain the variance in the data as much as possible, while controlling the number of nonzero loadings in these combinations. In this paper we consider 8 different optimiz…

Interpolation hurts robust generalization even without noise.

VRCQ algorithm reduces variance in Q-learning for MDPs, achieving optimal sample complexity.

Normative modeling has recently been proposed as an alternative for the case-control approach in modeling heterogeneity within clinical cohorts. Normative modeling is based on single-output Gaussian process regression that provides coherent estimates of uncertainty required by the method but does not consider spatial c…

Humans are able to accelerate their learning by selecting training materials that are the most informative and at the appropriate level of difficulty. We propose a framework for distributing deep learning in which one set of workers search for the most informative examples in parallel while a single worker updates the …

GAS-Norm improves deep learning time series forecasting in non-stationary settings.

Adam converges to stationary points under relaxed conditions.

New method relaxes PCA orthogonality constraints using explained variance of correlated components.

After presenting Actor Critic Methods (ACM), we show ACM are control variate estimators. Using the projection theorem, we prove that the Q and Advantage Actor Critic (A2C) methods are optimal in the sense of the $L^2$ norm for the control variate estimators spanned by functions conditioned by the current state and acti…

Study variance-reduced method for estimating fixed points in Banach spaces.

This paper addresses the problem of segmenting a time-series with respect to changes in the mean value or in the variance. The first case is when the time data is modeled as a sequence of independent and normal distributed random variables with unknown, possibly changing, mean value but fixed variance. The main assumpt…

We consider a group of mean-variance investors with mimicking desire such that each investor is willing to penalize deviations of his portfolio composition from compositions of other group members. Penalizing norm constraints are already applied for statistical improvement of Markowitz portfolio procedure in order to c…

We introduce and analyze a form of variance-reduced $Q$-learning. For $γ$-discounted MDPs with finite state space $\mathcal{X}$ and action space $\mathcal{U}$, we prove that it yields an $ε$-accurate estimate of the optimal $Q$-function in the $\ell_\infty$-norm using $\mathcal{O} \left(\left(\frac{D}{ ε^2 (1-γ)^3} \ri…

The paper refines and generalizes worst-case law invariant convex risk measures.

A method predicts GNS of transformer layers using normalization layer norms.

This work studies the implicit bias of mini-batch SGD in classification.

This paper analyzes convergence of RMSProp and Adam in non-convex optimization with tight complexity bounds.

A conventional wisdom in statistical learning is that large models require strong regularization to prevent overfitting. Here we show that this rule can be violated by linear regression in the underdetermined $n\ll p$ situation under realistic conditions. Using simulations and real-life high-dimensional data sets, we d…

The study computes Bergman kernels and point process asymptotics on Kähler manifolds.

In this paper, we propose $\ell_p$-norm regularized models to seek near-optimal sparse portfolios. These sparse solutions reduce the complexity of portfolio implementation and management. Theoretical results are established to guarantee the sparsity of the second-order KKT points of the $\ell_p$-norm regularized models…

Study variance-optimal hedging of forward curve derivatives under stochastic volatility.

Sharp concentration inequalities for sub-Orlicz random variables with phase transition at α=2.

We lower bound the complexity of finding $ε$-stationary points (with gradient norm at most $ε$) using stochastic first-order methods. In a well-studied model where algorithms access smooth, potentially non-convex functions through queries to an unbiased stochastic gradient oracle with bounded variance, we prove that (i…

Improved stochastic Halpern iteration for fixed-point approximation in normed spaces.

Study ridge regression for non-identically distributed data with varying variances.