arXiv research

Optimizes random forest inference by defining step order to maximize accuracy.

Random Function Descent improves optimization in high dimensions.

In this paper we propose using the principle of boosting to reduce the bias of a random forest prediction in the regression setting. From the original random forest fit we extract the residuals and then fit another random forest to these residuals. We call the sum of these two random forests a \textit{one-step boosted …

Single gradient step finds adversarial examples in random neural networks.

We show that time-dependent fluctuations $\{Δx\}$ in foreign exchange rates are accurately described by a random walk in a complex plane that is demarcated into the gain (+) and loss (-) sectors. $\{Δx\}$ is the outcome of $N$ random steps from the origin and $|Δx|$ is the square of the Euclidean distance of the final …

Study shows how mini-batch GD with random reshuffling affects least squares regression dynamics.

State-of-the-art adversarial attacks on neural networks use expensive iterative methods and numerous random restarts from different initial points. Iterative FGSM-based methods without restarts trade off performance for computational efficiency because they do not adequately explore the image space and are highly sensi…

As a testament to their success, the theory of random forests has long been outpaced by their application in practice. In this paper, we take a step towards narrowing this gap by providing a consistency result for online random forests.

In empirical risk optimization, it has been observed that stochastic gradient implementations that rely on random reshuffling of the data achieve better performance than implementations that rely on sampling the data uniformly. Recent works have pursued justifications for this behavior by examining the convergence rate…

We investigate the statistics of records in a random sequence $\{x_B(0)=0,x_B(1),\cdots, x_B(n)=x_B(0)=0\}$ of $n$ time steps. The sequence $x_B(k)$'s represents the position at step $k$ of a random walk `bridge' of $n$ steps that starts and ends at the origin. At each step, the increment of the position is a random ju…

A random walk $w_n$ on a separable, geodesic hyperbolic metric space $X$ converges to the boundary $\partial X$ with probability one when the step distribution supports two independent loxodromics. In particular, the random walk makes positive linear progress. Progress is known to be linear with exponential decay when …

We analyze two novel randomized variants of the Frank-Wolfe (FW) or conditional gradient algorithm. While classical FW algorithms require solving a linear minimization problem over the domain at each iteration, the proposed method only requires to solve a linear minimization problem over a small \emph{subset} of the or…

New method improves adversarial training efficiency and robustness.

The paper analyzes the randomized midpoint method for Langevin diffusions, revealing biases and asymptotic properties.

The paper extends risk measures to two-step approximations and studies log-concave distributions.

A two-step nonparametric method estimates financial systemic risk.

Study shows singularity of stationary measure on Furstenberg boundary for certain random walks.

A new graph neural network tackles oversmoothing and generalization issues.

New method solves stochastic optimization problems with random models.

We prove that a random group of the graph model associated with a sequence of expanders has fixed-point property for a certain class of CAT(0) spaces. We use Gromov's criterion for fixed-point property in terms of the growth of n-step energy of equivariant maps from a finitely generated group into a CAT(0) space, to wh…

The paper corrects for node degree in spectral clustering using random walk Laplacian.

DynForest R package predicts outcomes with time-dependent predictors.

Improved neural network training in low-dimensional random bases.

We study the statistics of records of a one-dimensional random walk of n steps, starting from the origin, and in presence of a constant bias c. At each time-step the walker makes a random jump of length ηdrawn from a continuous distribution f(η) which is symmetric around a constant drift c. We focus in particular on th…

The weights of a neural network are typically initialized at random, and one can think of the functions produced by such a network as having been generated by a prior over some function space. Studying random networks, then, is useful for a Bayesian understanding of the network evolution in early stages of training. In…

We highlight a pitfall when applying stochastic variational inference to general Bayesian networks. For global random variables approximated by an exponential family distribution, natural gradient steps, commonly starting from a unit length step size, are averaged to convergence. This useful insight into the scaling of…

Let T(x,r) denote the first hitting time of the disc of radius r centered at x for Brownian motion on the two dimensional torus. We prove that sup_{x} T(x,r)/|log r|^2 --> 2/pi as r --> 0. The same applies to Brownian motion on any smooth, compact connected, two-dimensional, Riemannian manifold with unit area and no bo…

The paper uses regression trees/random forests to price Bermudan options more efficiently.

We present an affine-invariant random walk for drawing uniform random samples from a convex body $\mathcal{K} \subset \mathbb{R}^n$ that uses maximum volume inscribed ellipsoids, known as John's ellipsoids, for the proposal distribution. Our algorithm makes steps using uniform sampling from the John's ellipsoid of the …

With increasing concerns about security, the need for highly secure physical biometrics-based authentication systems utilizing \emph{cancelable biometric} technologies is on the rise. Because the problem of cancelable template generation deals with the trade-off between template security and matching performance, many …

Joint sparsity regularization in multi-task learning has attracted much attention in recent years. The traditional convex formulation employs the group Lasso relaxation to achieve joint sparsity across tasks. Although this approach leads to a simple convex formulation, it suffers from several issues due to the loosenes…

RSGDA improves convergence rates for nonconvex-strongly concave optimization.

LightOn OPUs accelerate randomized numerical linear algebra, reducing computational costs.

Poisson Midpoint Method improves Langevin Dynamics for diffusion models.

New method uses randomized sparse neural networks to solve time-dependent PDEs more accurately and efficiently.

Optimal weighted random forests improve prediction accuracy.

Spectral clustering has become a popular technique due to its high performance in many contexts. It comprises three main steps: create a similarity graph between N objects to cluster, compute the first k eigenvectors of its Laplacian matrix to define a feature vector for each object, and run k-means on these features t…

Aggregates predictions from multiple regression models using random projections and kernel methods.

One gradient step improves neural network feature learning by aligning weights with the teacher model.

Extends causal discovery to group variables, improving performance in real-world applications.

Neural networks can overfit perfectly to noisy data and then grok near-optimal generalization.

We relate some basic constructions of stochastic analysis to differential geometry, via random walk approximations. We consider walks on both Riemannian and sub-Riemannian manifolds in which the steps consist of travel along either geodesics or integral curves associated to orthonormal frames, and we give particular at…

Hamiltonian Monte Carlo (HMC) is a state-of-the-art Markov chain Monte Carlo sampling algorithm for drawing samples from smooth probability densities over continuous spaces. We study the variant most widely used in practice, Metropolized HMC with the Störmer-Verlet or leapfrog integrator, and make two primary contribut…

Abstract: Nonlinear random walk with distributionally robust transition probabilities.

The Douglas Rachford algorithm is an algorithm that converges to a minimizer of a sum of two convex functions. The algorithm consists in fixed point iterations involving computations of the proximity operators of the two functions separately. The paper investigates a stochastic version of the algorithm where both funct…

Enhanced Gaussian process models accelerate optimization and posterior approximation.

We show that the largest subsurface projection distance between a marking and its image under the nth step of a random walk grows logarithmically in n, with probability approaching 1 as n tends to infinity. Our setup is general and also applies to (relatively) hyperbolic groups and to $\mathrm{Out}(F_n)$. We then use t…

Study on RF regression with SGD shows double descent phenomenon.