arXiv research

The paper solves optimal bounds for separating data points in high dimensions.

Stochastic Neighbor Embedding and its variants are widely used dimensionality reduction techniques -- despite their popularity, no theoretical results are known. We prove that the optimal SNE embedding of well-separated clusters from high dimensions to any Euclidean space R^d manages to successfully separate the cluste…

New algorithm improves source separation with multi-trial supervision.

The paper analyzes condition numbers for logistic regression to understand first-order methods' performance.

slimTrain simplifies DNN training by separating features and adapting hyperparameters.

SPI-Optimizer separates momentum term to eliminate oscillation in stochastic optimization.

Study models deep learning training dynamics using locally elastic SDEs to reveal feature separability.

Paper analyzes normal approximation for two-timescale stochastic algorithms, revealing interaction between fast and slow timescales.

This work proposes external correctors for quick AI error corrections without system modification.

The Douglas Rachford algorithm is an algorithm that converges to a minimizer of a sum of two convex functions. The algorithm consists in fixed point iterations involving computations of the proximity operators of the two functions separately. The paper investigates a stochastic version of the algorithm where both funct…

New algorithm AG-OG optimizes separable convex-concave problems efficiently.

Optimizes CM for stochastic convex optimization with progressive precision.

In this paper, a simple, general method of adding auxiliary stochastic neurons to a multi-layer perceptron is proposed. It is shown that the proposed method is a generalization of recently successful methods of dropout (Hinton et al., 2012), explicit noise injection (Vincent et al., 2010; Bishop, 1995) and semantic has…

Gradient methods avoid overfitting on separable data.

Deviation inequalities for stochastic approximation methods.

Localized sum-of-norms clustering separates balls in data.

This work investigates implicit bias in multiclass separable data using a novel geometry-aware optimizer.

Adaptive clustering and personalization algorithms minimize regret in multi-agent stochastic linear bandits.

Graph convolution improves linear separability and generalizes to out-of-distribution data.

Adam optimizes linear classifiers with separable data.

DDICA separates nonlinear mixed signals robustly.

We develop randomized (block) coordinate descent (CD) methods for linearly constrained convex optimization. Unlike most CD methods, we do not assume the constraints to be separable, but let them be coupled linearly. To our knowledge, ours is the first CD method that allows linear coupling constraints, without making th…

We consider convex-concave saddle point problems with a separable structure and non-strongly convex functions. We propose an efficient stochastic block coordinate descent method using adaptive primal-dual updates, which enables flexible parallel optimization for large-scale problems. Our method shares the efficiency an…

Separates estimation and control in risk-sensitive investment problems with partial observation.

The paper explores how different patterns of heterophily affect Graph Neural Networks.

We propose a new stochastic coordinate descent method for minimizing the sum of convex functions each of which depends on a small number of coordinates only. Our method (APPROX) is simultaneously Accelerated, Parallel and PROXimal; this is the first time such a method is proposed. In the special case when the number of…

Efficiently clusters large datasets using low-density hyperplanes.

Investigates optimal insurance and reinsurance strategies with incomplete market information.

New method disentangles sources of different timescales in planetary seismic data.

We consider a generic convex-concave saddle point problem with separable structure, a form that covers a wide-ranged machine learning applications. Under this problem structure, we follow the framework of primal-dual updates for saddle point problems, and incorporate stochastic block coordinate descent with adaptive st…

New graph kernels capture spatio-temporal interactions.

Stochastic Gradient Descent phases explained for deep networks.

AI systems can be corrected without rebuilding them, using simple linear methods.

Poly-GNNs achieve similar performance regardless of depth, highlighting graph noise's dominance.

We develop a scalable method for Bayesian neural networks with stochastic differential equations.

New method solves nonseparable stochastic control problems.

Efficient private algorithms for estimating block models and mixture models.

This paper introduces the probabilistic module interface, which allows encapsulation of complex probabilistic models with latent variables alongside custom stochastic approximate inference machinery, and provides a platform-agnostic abstraction barrier separating the model internals from the host probabilistic inferenc…

This work analyzes and improves stochastic gradient methods for GAN training.

We present a stochastic setting for optimization problems with nonsmooth convex separable objective functions over linear equality constraints. To solve such problems, we propose a stochastic Alternating Direction Method of Multipliers (ADMM) algorithm. Our algorithm applies to a more general class of nonsmooth convex …

We propose HAMSI (Hessian Approximated Multiple Subsets Iteration), which is a provably convergent, second order incremental algorithm for solving large-scale partially separable optimization problems. The algorithm is based on a local quadratic approximation, and hence, allows incorporating curvature information to sp…

Stochastic Gradient Descent (SGD) is a central tool in machine learning. We prove that SGD converges to zero loss, even with a fixed (non-vanishing) learning rate - in the special case of homogeneous linear classifiers with smooth monotone loss functions, optimized on linearly separable data. Previous works assumed eit…

Most models for barrier pricing are designed to let a market maker tune the model-implied covariance between moves in the asset spot price and moves in the implied volatility skew. This is often implemented with a local volatility/stochastic volatility mixture model, where the mixture parameter tunes that covariance. T…

We address the problem of causal discovery from data, making use of the recently proposed causal modeling framework of modular structural causal models (mSCM) to handle cycles, latent confounders and non-linearities. We introduce σ-connection graphs (σ-CG), a new class of mixed graphs (containing undirected, bidirected…

This study analyzes adversarial training on linearly separable data and finds that gradient updates can achieve large margins in polynomial iterations.

Develops tests for Markowitz stochastic dominance spanning using saddle points.

How can we efficiently propagate uncertainty in a latent state representation with recurrent neural networks? This paper introduces stochastic recurrent neural networks which glue a deterministic recurrent neural network and a state space model together to form a stochastic and sequential neural generative model. The c…

We study the Combinatorial Pure Exploration problem with Continuous and Separable reward functions (CPE-CS) in the stochastic multi-armed bandit setting. In a CPE-CS instance, we are given several stochastic arms with unknown distributions, as well as a collection of possible decisions. Each decision has a reward accor…