arXiv research

New algorithms for online learning without boundedness or Lipschitz loss assumptions.

New bounds for online portfolio selection without smoothness assumptions.

Study proves existence and uniqueness for differential equations with non-Lipschitz coefficients.

New algorithm reduces pricing error by a factor of T^2/3.

We consider the finite sample properties of the regularized high-dimensional Cox regression via lasso. Existing literature focuses on linear models or generalized linear models with Lipschitz loss functions, where the empirical risk functions are the summations of independent and identically distributed (iid) losses. T…

New research shows many batch selection methods for training work just as well as full batch training.

We consider a class of constrained optimization problems with a possibly nonconvex non-Lipschitz objective and a convex feasible set being the intersection of a polyhedron and a possibly degenerate ellipsoid. Such problems have a wide range of applications in data science, where the objective is used for inducing spars…

New algorithm reduces prediction errors across various loss functions.

Paper proves convergence for private FL on non-Lipschitz convex objectives using normalization instead of clipping.

Extends involutivity to non-Lipschitz subbundles and proves the Frobenius Theorem.

We consider the approximation of stochastic differential equations (SDEs) with non-Lipschitz drift or diffusion coefficients. We present a modified explicit Euler-Maruyama discretisation scheme that allows us to prove strong convergence, with a rate. Under some regularity and integrability conditions, we obtain the opt…

Paper analyzes robustness of non-Lipschitz networks, proving powerful adversarial attacks but offering solutions.

Improved MLMC method for barrier options with non-Lipschitz coefficients.

Paper tackles robust control of SDEs with ambiguity, proving value function existence and applying to investment problems.

The (global) Lipschitz smoothness condition is crucial in establishing the convergence theory for most optimization methods. Unfortunately, most machine learning and signal processing problems are not Lipschitz smooth. This motivates us to generalize the concept of Lipschitz smoothness condition to the relative smoothn…

Full-batch GD achieves generalization close to any stationary point with fewer assumptions.

The predict-then-optimize framework is fundamental in many practical settings: predict the unknown parameters of an optimization problem, and then solve the problem using the predicted values of the parameters. A natural loss function in this environment is to consider the cost of the decisions induced by the predicted…

New method models fat-tailed distributions with anisotropic tail-adaptive flows.

The CEV model is given by the stochastic differential equation $X_t=X_0+\int_0^tμX_sds+\int_0^tσ(X^+_s)^pdW_s$, $\frac{1}{2}\le p<1$. It features a non-Lipschitz diffusion coefficient and gets absorbed at zero with a positive probability. We show the weak convergence of Euler-Maruyama approximations $X_t^n$ to the proc…

Unified analysis of online optimization with self-concordant barriers, improving regret bounds.

We prove strong existence and uniqueness, and Hölder regularity, of a large class of stochastic Volterra equations, with singular kernels and non-Lipschitz diffusion coefficient. Extending Yamada-Watanabe's theorem, our proof relies on an approximation of the process by a sequence of semimartingales with regularised ke…

A new algorithm reduces communication in deep learning training.

We study the use of the multilevel Monte Carlo technique in the context of the calculation of Greeks. The pathwise sensitivity analysis differentiates the path evolution and reduces the payoff's smoothness. This leads to new challenges: the inapplicability of pathwise sensitivities to non-Lipschitz payoffs often makes …

We are interested in strong approximations of one-dimensional SDEs which have non-Lipschitz coefficients and which take values in a domain. Under a set of general assumptions we derive an implicit scheme that preserves the domain of the SDEs and is strongly convergent with rate one. Moreover, we show that this general …

Study on non-negative solutions for stochastic Volterra equations with jumps.

We study online optimization of smoothed piecewise constant functions over the domain [0, 1). This is motivated by the problem of adaptively picking parameters of learning algorithms as in the recently introduced framework by Gupta and Roughgarden (2016). Majority of the machine learning literature has focused on Lipsc…

A new macroscopic market making model connects market making and optimal execution.

We study the robustness of active learning (AL) algorithms against prior misspecification: whether an algorithm achieves similar performance using a perturbed prior as compared to using the true prior. In both the average and worst cases of the maximum coverage setting, we prove that all $α$-approximate algorithms are …

New shuffling methods improve convergence without Lipschitz smoothness.

NES optimizes discrete structured VAEs effectively without gradient propagation.

This work closes the theory-practice gap for distributed optimization methods by introducing a new regularity condition.

The paper relaxes assumptions for analyzing stochastic optimization algorithms.

Deep NURBS improves PINNs for solving PDEs on arbitrary geometries.

Optimization in the presence of sharp (non-Lipschitz), unpredictable (w.r.t. time and amount) changes is a challenging and largely unexplored problem of great significance. We consider the class of piecewise Lipschitz functions, which is the most general online setting considered in the literature for the problem, and …

Meta-learning improves performance across similar tasks in adversarial bandit settings.

In this paper we consider a general matrix factorization model which covers a large class of existing models with many applications in areas such as machine learning and imaging sciences. To solve this possibly nonconvex, nonsmooth and non-Lipschitz problem, we develop a non-monotone alternating updating method based o…

The heavy-tailed distributions of corrupted outliers and singular values of all channels in low-level vision have proven effective priors for many applications such as background modeling, photometric stereo and image alignment. And they can be well modeled by a hyper-Laplacian. However, the use of such distributions g…

Nonparametric Thompson Sampling achieves optimal regret for risk-averse bandits with sub-Gaussian rewards.

Novel Bayesian framework for Poisson inverse problems using Bregman geometry.

In this paper, we consider a class of possibly nonconvex, nonsmooth and non-Lipschitz optimization problems arising in many contemporary applications such as machine learning, variable selection and image processing. To solve this class of problems, we propose a proximal gradient method with extrapolation and line sear…

Study proves existence, uniqueness, and positivity of solutions to a complex volatility model.

The one-dimensional SDE with non Lipschitz diffusion coefficient $dX_{t} = b(X_{t})dt + σX_{t}^γ dB_{t}, \ X_{0}=x, \ γ<1$ is widely studied in mathematical finance. Several works have proposed asymptotic analysis of densities and implied volatilities in models involving instances of this equation, based on a careful i…

In this paper, we consider a well-known sparse optimization problem that aims to find a sparse solution of a possibly noisy underdetermined system of linear equations. Mathematically, it can be modeled in a unified manner by minimizing $\|\bf{x}\|_p^p$ subject to $\|A\bf{x}-\bf{b}\|_q\leqσ$ for given $A \in \mathbb{R}^…

Introduces Fitzpatrick losses, tighter than Fenchel-Young losses.

We study losses for binary classification and class probability estimation and extend the understanding of them from margin losses to general composite losses which are the composition of a proper loss with a link function. We characterise when margin losses can be proper composite losses, explicitly show how to determ…

We present the Tamed Cross Entropy (TCE) loss function, a robust derivative of the standard Cross Entropy (CE) loss used in deep learning for classification tasks. However, unlike other robust losses, the TCE loss is designed to exhibit the same training properties than the CE loss in noiseless scenarios. Therefore, th…

Unified surrogate loss framework for multi-label learning with strong consistency guarantees.

This paper introduces new loss functions for balanced multi-class classification.