Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,657 papers · 148 categories

Trend · papers per month

51102153204 · Jun 202019922001200920172026
48 results for forward-mode differentiation

The paper proposes a method to compute higher infinitesimals in numerical and symbolic analysis.

problem Computing higher-order derivatives with higher infinitesimals.
method Automatic differentiation in terms of C-infinity rings and Weil algebras.
result A unifying theoretical framework for multivariate higher-order derivatives.

This work speeds up hyperparameter selection for non-smooth convex models using implicit differentiation.

problem Optimizing hyperparameters of non-smooth convex models.
method Implicit differentiation of proximal gradient and coordinate descent methods.
result Implicit differentiation can speed up hyperparameter optimization, especially for non-smooth problems.

We study two procedures (reverse-mode and forward-mode) for computing the gradient of the validation error with respect to the hyperparameters of any iterative learning algorithm such as stochastic gradient descent. These procedures mirror two methods of computing gradients for recurrent neural networks and have differ…

2017-03-06abs ↗pdf ↗

Backpropagation-free trunk training improves model performance on various benchmarks.

problem Memory inefficiency and noisy gradient estimates in deep network training.
method Split Forward Gradient (Split-FG) method that splits network into trunk and head, estimating only trunk gradient.
result Split-FG achieves better performance than pure forward-gradient training and backpropagation on various benchmarks.

Develops accelerated methods for optimization using low-dimensional projected-gradient information.

problem Optimization with low-dimensional projected-gradient information and Nesterov acceleration.
method Randomized-subspace Nesterov accelerated gradient methods for smooth convex and strongly convex optimization.
result Established accelerated oracle-complexity guarantees and unified basis for comparing sketch families.

ACA method improves gradient estimation for neural ODEs, reducing error and training time.

problem Inaccurate gradient estimation methods hinder the performance of neural ODEs on benchmark tasks.
method Adaptive Checkpoint Adjoint (ACA) method that applies trajectory checkpointing, deletes redundant components, and supports adaptive solvers.
result ACA reduces error rate by half and training time by half compared to adjoint and naive methods on image classification tasks.

We propose a fast second-order method that can be used as a drop-in replacement for current deep learning solvers. Compared to stochastic gradient descent (SGD), it only requires two additional forward-mode automatic differentiation operations per iteration, which has a computational cost comparable to two standard for…

2018-05-21abs ↗pdf ↗

A new stochastic method tackles bi-level optimization problems in deep learning.

problem Bi-level optimization problems in deep learning, including hyperparameter optimization and meta learning.
method Turning a BLO problem into a stochastic optimization, using SGLD MCMC and a recurrent algorithm to compute MC-estimated hypergradient.
result Our method is more robust to suboptimal inner optimization and non-unique inner minima, leading to more reliable solutions.

Cross validation (CV) and the bootstrap are ubiquitous model-agnostic tools for assessing the error or variability of machine learning and statistical estimators. However, these methods require repeatedly re-fitting the model with different weighted versions of the original dataset, which can be prohibitively time-cons…

2019-07-28abs ↗pdf ↗

Neural network model improves leaf spectral reflectance prediction for grapevines.

problem Inaccurate modeling of grapevine leaf spectral reflectance from traits.
method Multi-head attention neural network trained on grapevine-specific data.
result Model achieved high accuracy (R^2=0.84, NRMSE=1.52%) and outperformed PROSPECT-PRO.

The closure conditions of the inexact exterior differential form and dual form (an equality to zero of differentials of these forms) can be treated as a definition of some differential-geometrical structure. Such a connection discloses the properties and specific features of the differential-geometrical structures. The…

2004-04-12abs ↗pdf ↗

In this paper we give explicit formulas of differential characteristic classes of principal GG-bundles with connections and prove their expected properties. In particular, we obtain explicit formulas for differential Chern classes, differential Pontryagin classes and differential Euler class. Furthermore, we show that…

2013-11-15abs ↗pdf ↗

We generalize stochastic smoothing for gradient estimation of non-differentiable functions.

problem Gradient estimation for non-differentiable functions.
method Developed a general framework for relaxation and gradient estimation of non-differentiable black-box functions using stochastic smoothing with reduced assumptions.
result Empirically validated the effectiveness of variance reduction strategies for various non-differentiable tasks.

The paper proves Gorenstein contractions for multiscale differentials on nodal curves.

problem Proving Gorenstein contractions for multiscale differentials on nodal curves.
method Addressing the conjecture by Ranganathan and Wise, showing contractions level by level.
result Multiscale differentials can be contracted to Gorenstein singularities, level by level, from the top down.

Classifies components of strata of k-differentials on Riemann surfaces.

problem Classifying connected components of strata of k-differentials.
method Developed new techniques to study connected components of strata of k-differentials for general k.
result Complete classification of connected components of the strata of quadratic differentials with arbitrary poles.

Given a unital associatve graded algebra we construct the graded q-differential algebra by means of a graded q-commutator, where q is a primitive N-th root of unity. The N-th power (N>1) of the differential of this graded q-differential algebra is equal to zero. We use our approach to construct the graded q-differentia…

2005-09-21abs ↗pdf ↗

Finite intersection numbers between horizontal foliations of quadratic differentials.

problem Intersection properties of horizontal foliations in quadratic differentials.
method Joint continuity of intersection number in L1L^1-norm.
result Intersection number is finite and jointly continuous.

We consider differentiable maps in the setting of Abstract Differential Geometry and we study the conditions that ensure the uniqueness of differentials in this setting. In particular, we prove that smooth maps between smooth manifolds admit a unique differential, coinciding with the usual one. Thus smooth manifolds fo…

2013-11-25abs ↗pdf ↗

We study two notions of relative differential cohomology, using the model of differential characters. The two notions arise from the two options to construct relative homology, either by cycles of a quotient complex or of a mapping cone complex. We discuss the relation of the two notions of relative differential cohomo…

2013-10-10abs ↗pdf ↗

Studies projective geometry and partial differential equations prolongation.

problem Understanding the prolongation of overdetermined geometric partial differential equations.
method Introduction to differential geometry and tractor calculus, study of prolongation of equations.
result Recovery of projective tractor and cotractor connections via partial differential equations prolongation.

Develops differential K-theory for noncommutative algebras.

problem Creating a differential extension of algebraic K-theory for noncommutative algebras.
method Introduces secondary transgression forms and a differential refinement of the smooth Serre--Swan correspondence.
result Subsumes differential K-theory for smooth manifolds and fits into a noncommutative differential cohomology hexagon diagram.

Extends differential geometry concepts to manifolds with super tangent bundles.

problem No specific problem stated; extending differential geometry to super tangent bundles.
method Introduces super tangent bundle and extends differential geometry concepts.
result Basic notions of differential geometry extended to manifolds with super tangent bundles.

Generalized differential cohomology theories, in particular differential K-theory (often called "smooth K-theory"), are becoming an important tool in differential geometry and in mathematical physics. In this survey, we describe the developments of the recent decades in this area. In particular, we discuss axiomatic ch…

2010-11-30abs ↗pdf ↗

Differentiable pipeline replaces non-differentiable CAE components for shape optimization.

problem Gradient-based optimization is limited by non-differentiable components in CAE workflows.
method Surrogate models replace non-differentiable pipeline components, enabling gradient-based optimization.
result Gradient-based shape optimization possible without differentiable solvers.

Differentiable programming aids in solving differential equations and their sensitivities.

problem Computing gradients of numerical solutions of differential equations.
method Review of existing techniques and mathematical foundations.
result Established a coherent framework for combining differential equations with data-driven approaches.

The purpose of this paper is to connect two subjects: the theory of quantum integrable systems (complete commutative rings of differential operators), and differential Galois theory. We define quantum completely integrable systems (QCIS), algebraically integrable QCIS, the differential Galois group of a QCIS. We show t…

1996-07-12abs ↗pdf ↗

A brief introduction to exterior differential systems for graduate students familiar with manifolds and differential forms. For complete files, see https://github.com/Ben-McKay/introduction-to-exterior-differential-systems

2017-06-29abs ↗pdf ↗

DiffEqFlux.jl is a library for fusing neural networks and differential equations. In this work we describe differential equations from the viewpoint of data science and discuss the complementary nature between machine learning models and differential equations. We demonstrate the ability to incorporate DifferentialEqua…

2019-02-06abs ↗pdf ↗

We study Cheeger-Simons differential characters and provide geometric descriptions of the ring structure and of the fiber integration map. The uniqueness of differential cohomology (up to unique natural transformation) is proved by deriving an explicit formula for any natural transformation between a differential cohom…

2013-03-26abs ↗pdf ↗

Defines semi-symmetric metric connections on differential forms.

problem Analyzing connections on differential forms.
method Defined and studied semi-symmetric metric connections, computed their curvature and Ricci tensors, and analyzed Lie derivatives.
result Derived Gauss-Codazzi-Ricci equations and properties of canonical, Schouten, and Vrancreanu connections.