arXiv research

The paper defines conditions for Gaussian process sample path regularity.

This paper presents the asymptotic behavior of a linear instrumental variables (IV) estimator that uses a ridge regression penalty. The regularization tuning parameter is selected empirically by splitting the observed data into training and test samples. Conditional on the tuning parameter, the training sample creates …

We provide theoretical analysis of the statistical and computational properties of penalized $M$-estimators that can be formulated as the solution to a possibly nonconvex optimization problem. Many important estimators fall in this category, including least squares regression with nonconvex regularization, generalized …

Study shows how feature weighting affects neural network regularization.

This work derives closed-form expressions computing the expectation of co-presence and of number of co-occurrences of nodes on paths sampled from a network according to general path weights (a bag of paths). The underlying idea is that two nodes are considered as similar when they often appear together on (preferably s…

New algorithmic view of ℓ2 regularization using ODEs and path-following methods.

Diagonal linear networks converge to lasso regularization path during training.

We propose an algorithm for exploring the entire regularization path of asymmetric-cost linear support vector machines. Empirical evidence suggests the predictive power of support vector machines depends on the regularization parameters of the training algorithms. The algorithms exploring the entire regularization path…

Recently, path norm was proposed as a new capacity measure for neural networks with Rectified Linear Unit (ReLU) activation function, which takes the rescaling-invariant property of ReLU into account. It has been shown that the generalization error bound in terms of the path norm explains the empirical generalization b…

Path regularization improves GFlowNets exploration and generalization.

Neural Jump ODEs model Itô processes without adversarial training.

Deep networks with path norm regularization can approximate analytic functions.

Algorithm approximates regularization path for deep neural networks efficiently.

New algorithm speeds up path computation for optimal models.

We revisit the choice of SGD for training deep neural networks by reconsidering the appropriate geometry in which to optimize the weights. We argue for a geometry invariant to rescaling of weights that does not affect the output of the network, and suggest Path-SGD, which is an approximate steepest descent method with …

PSiLON Net uses $L_1$ weight normalization and 1-path-norm regularization for efficient learning and sparsity.

The regularization path of the Lasso can be shown to be piecewise linear, making it possible to "follow" and explicitly compute the entire path. We analyze in this paper this popular strategy, and prove that its worst case complexity is exponential in the number of variables. We then oppose this pessimistic result to a…

For a variety of regularized optimization problems in machine learning, algorithms computing the entire solution path have been developed recently. Most of these methods are quadratic programs that are parameterized by a single parameter, as for example the Support Vector Machine (SVM). Solution path algorithms do not …

Diffusion models' sampling paths lie in a low-dimensional subspace, resembling boomerangs.

Paper presents a copula-based method to efficiently generate correlated sample paths from multi-step time series models.

Sparse model selection is ubiquitous from linear regression to graphical models where regularization paths, as a family of estimators upon the regularization parameter varying, are computed when the regularization parameter is unknown or decided data-adaptively. Traditional computational methods rely on solving a set o…

In this work we establish the equivalence of algorithmic regularization and explicit convex penalization for generic convex losses. We introduce a geometric condition for the optimization path of a convex function, and show that if such a condition is satisfied, the optimization path of an iterative algorithm on the un…

Path regularization reveals convex optimization in deep ReLU networks.

In this paper we outline methodology to efficiently simulate (jump) diffusion bridge sample paths without discretisation error. We achieve this by considering the simulation of conditioned (jump) diffusion bridge sample paths in light of recent work developing a mathematical framework for simulating finite dimensional …

We consider the generic regularized optimization problem $\hat{\mathsfβ}(λ)=\arg \min_βL({\sf{y}},X{\sfβ})+λJ({\sfβ})$. Efron, Hastie, Johnstone and Tibshirani [Ann. Statist. 32 (2004) 407--499] have shown that for the LASSO--that is, if $L$ is squared error loss and $J(β)=\|β\|_1$ is the $\ell_1$ norm of $β$--the opti…

Optimal a priori estimates are derived for the population risk, also known as the generalization error, of a regularized residual network model. An important part of the regularized model is the usage of a new path norm, called the weighted path norm, as the regularization term. The weighted path norm treats the skip c…

Convex clustering is a promising new approach to the classical problem of clustering, combining strong performance in empirical studies with rigorous theoretical foundations. Despite these advantages, convex clustering has not been widely adopted, due to its computationally intensive nature and its lack of compelling v…

Paper proposes method for generating paths of stochastic volatility CGMY process for option pricing.

A regularized optimization problem over a large unstructured graph is studied, where the regularization term is tied to the graph geometry. Typical regularization examples include the total variation and the Laplacian regularizations over the graph. When applying the proximal gradient algorithm to solve this problem, t…

Proposes a neural network framework for feature selection in high-dimensional settings.

The aim of this paper is to establish two fundamental measure-metric properties of particular random geometric graphs. We consider $\varepsilon$-neighborhood graphs whose vertices are drawn independently and identically distributed from a common distribution defined on a regular submanifold of $\mathbb{R}^K$. We show t…

Derives functional Itô formula for non-anticipative maps of rough paths.

New algorithms sample from complex path measures using neural networks.

Proposes a thermodynamic work minimization framework for guiding generative models.

Generative Flow Networks solve shortest path problems in graphs.

AgFlow speeds up model selection in penalized PCA.

Flow Matching enables robust training of CNFs with various probability paths.

Gradient descent implicitly follows regularization for general losses.

We consider supervised learning problems where the features are embedded in a graph, such as gene expressions in a gene network. In this context, it is of much interest to automatically select a subgraph with few connected components; by exploiting prior knowledge, one can indeed improve the prediction performance or o…

A bounded curvature path is a continuously differentiable piecewise $C^2$ path with a bounded absolute curvature that connects two points in the tangent bundle of a surface. In this work, we analyze the homotopy classes of bounded curvature paths for points in the tangent bundle of the Euclidean plane. We show the exis…

The recently developed bag-of-paths (BoP) framework consists in setting a Gibbs-Boltzmann distribution on all feasible paths of a graph. This probability distribution favors short paths over long ones, with a free parameter (the temperature $T$) controlling the entropic level of the distribution. This formalism enables…

Regularizes decision trees to reduce inference time by up to 4x with minimal accuracy loss.

New methods improve Monte Carlo estimation of partition functions.

Efficient algorithms for clustered Lasso and OSCAR reduce computational costs.

In this paper, we recover sparse signals from their noisy linear measurements by solving nonlinear differential inclusions, which is based on the notion of inverse scale space (ISS) developed in applied mathematics. Our goal here is to bring this idea to address a challenging problem in statistics, \emph{i.e.} finding …

PSLR classifies functional data with scalar covariates using path signatures.

A framework for hypothesis testing on attributed graphs using sampling.

Improved sampling efficiency for molecular systems using path gradients after Flow Matching.