Local SGD outperforms conventional methods in LLM training.
problem Training large language models on distributed devices.
method Local SGD algorithm applied to distributed training.
result Local SGD achieves competitive results in LLM training.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,742 papers · 148 categories
Trend · papers per month
48 results for “local computation”
New methods for federated learning reduce communication costs.
problem Efficiently solving optimization problems in a distributed setting.
method Developed two strategies for achieving consensus in federated learning: fixed number of local steps and randomized computations.
result Convergence analysis and experiments show benefits of the proposed methods.
Hogwild! adapts to distributed data by varying mini-batch sizes.
problem Efficiently parallelize SGD over distributed local data sets.
method Asynchronous SGD with varying mini-batch sizes, aggregated by an aggregator.
result Improved convergence for heterogeneous data, reducing communication rounds.
Communication bottleneck has been identified as a significant issue in distributed optimization of large-scale learning models. Recently, several approaches to mitigate this problem have been proposed, including different forms of gradient compression or computing local models and mixing them iteratively. In this paper…
The paper extends a variance gamma model to quadratic functions, reducing arbitrage and computational costs.
problem Creating an arbitrage-free interpolation for option pricing models.
method Generalizing the local variance gamma model to a piecewise quadratic local variance function.
result The quadratic model results in an arbitrage-free interpolation of class C3, reducing knots and computational cost.
One of the limiting factors of using support vector machines (SVMs) in large scale applications are their super-linear computational requirements in terms of the number of training samples. To address this issue, several approaches that train SVMs on many small chunks of large data sets separately have been proposed in…
We introduce a local algorithm for Khovanov Homology computations - that is, we explain how it is possible to "cancel" terms in the Khovanov complex associated with a ("local") tangle, hence canceling the many associated "global" terms in one swoosh early on. This leads to a dramatic improvement in computational effici…
Machine learning and geostatistics are powerful mathematical frameworks for modeling spatial data. Both approaches, however, suffer from poor scaling of the required computational resources for large data applications. We present the Stochastic Local Interaction (SLI) model, which employs a local representation to impr…
We review the properties of the Morse-Novikov cohomology and compute it for all known compact complex surfaces with locally conformally Kähler metrics. We present explicit computations for the Inoue surfaces , , and classify the locally conformally Kähler (and the tamed loc…
Integrable symmetries of diffieties are studied, leading to local morphisms.
problem Understanding local integrable symmetries of diffieties.
method Integrable infinitesimal symmetries defined as a one-parameter pseudogroup of local diffiety morphisms. Reduction of computation to solving PDEs.
result Preliminary results and examples show integrable symmetries can be reduced to solving linear systems.
This paper proposes an in-depth re-thinking of neural computation that parallels apparently unrelated laws of physics, that are formulated in the variational framework of the least action principle. The theory holds for neural networks that are also based on any digraph, and the resulting computational scheme exhibits …
A new TwinGP framework for efficient large-scale GP modeling.
problem Efficiently modeling large-scale Gaussian processes with computational constraints.
method Combines global and local approximations using a subset-of-data approach.
result TwinGP framework performs on par or better than state-of-the-art methods at a fraction of the computational cost.
We compute the local Lipschitz constant of ReLU networks precisely.
problem Estimating the local Lipschitz constant of ReLU networks is hard.
method We use a novel approach involving the generalized Jacobian and backpropagation.
result We provide an algorithm to compute the exact Lipschitz constant of ReLU networks.
FALL improves local model training with anchor regularization.
problem Efficient local model training in regression tasks.
method Regularization with precomputed anchor models, closed-form solution.
result FALL outperforms network Lasso in accuracy with significantly less training time.
We consider a defaultable asset whose risk-neutral pricing dynamics are described by an exponential Lévy-type martingale. This class of models allows for a local volatility, local default intensity and a locally dependent Lévy measure. We present a pricing method for Bermudan options based on an analytical approximatio…
A new method reduces communication costs in distributed learning.
problem Reduces communication bottlenecks in distributed learning.
method Local SGD with communication-computation overlap and delay-corrected sparse model averaging.
result Theoretical convergence guarantees for smooth non-convex objectives.
Efficient Decomposition of Forman-Ricci Curvature on Vietoris-Rips Complexes and Data Applicationsmath.GT
Researchers decompose Forman-Ricci curvature for efficient computation in VR complexes.
problem Efficiently computing Forman-Ricci curvature in higher-dimensional data.
method Decomposition and set-theoretical proof for local computation of FRC in VR complexes.
result Reveals critical geometric insights overlooked by conventional techniques.
Manifold Markov chain Monte Carlo algorithms have been introduced to sample more effectively from challenging target densities exhibiting multiple modes or strong correlations. Such algorithms exploit the local geometry of the parameter space, thus enabling chains to achieve a faster convergence rate when measured in n…
Proposes a framework to incorporate global sensitivity into local surrogate models.
problem Narrowing focus to local scale in surrogate modeling leads to re-learning global trends.
method Integrates global sensitivity analysis into local surrogate models through input warping.
result Local models become equally sensitive to all input directions, focusing on local dynamics.
In this note we compute low degree rational Pontryagin classes for every closed locally symmetric manifold of noncompact type. In particular, we answer the question: Which locally symmetric M have at least one nonzero Pontryagin class?
We study distributed algorithms for expected loss minimization where the datasets are large and have to be stored on different machines. Often we deal with minimizing the average of a set of convex functions where each function is the empirical risk of the corresponding part of the data. In the distributed setting wher…
Twisted Morse complexesmath.AT
In this paper we study Morse homology and cohomology with local coefficients, i.e. "twisted" Morse homology and cohomology, on closed finite dimensional smooth manifolds. We prove a Morse theoretic version of Eilenberg's Theorem, and we prove isomorphisms between twisted Morse homology, Steenrod's CW-homology with loca…
The recent decades have seen a surge of interests in distributed computing. Existing work focus primarily on either distributed computing platforms, data query tools, or, algorithms to divide big data and conquer at individual machines etc. It is, however, increasingly often that the data of interest are inherently dis…
In our physically inspired in-tree (IT) based clustering algorithm and the series after it, there is only one free parameter involved in computing the potential value of each point. In this work, based on the Delaunay Triangulation or its dual Voronoi tessellation, we propose a nonparametric process to compute potentia…
Local GP approach improves simulation efficiency for large datasets.
problem High computational cost of traditional Gaussian processes for large-scale simulations.
method Hybridizes global and local GP approximations with strategic placement of inducing points.
result Local inducing points enhance accuracy and computational efficiency.
Overlap-Local-SGD improves distributed SGD by overlapping communication and computation.
problem High communication delay and node slowdown in distributed SGD.
method Adding an anchor model to synchronize local updates and pull them towards the anchor model.
result Overlap-Local-SGD speeds up distributed training and mitigates straggler effects.
Large scale agglomerative clustering is hindered by computational burdens. We propose a novel scheme where exact inter-instance distance calculation is replaced by the Hamming distance between Kernelized Locality-Sensitive Hashing (KLSH) hashed values. This results in a method that drastically decreases computation tim…
We extend the traditional worst-case, minimax analysis of stochastic convex optimization by introducing a localized form of minimax complexity for individual functions. Our main result gives function-specific lower and upper bounds on the number of stochastic subgradient evaluations needed to optimize either the functi…
We give a new algorithm computing local system cohomology groups for complexified real line arrangements. Using it, we obtain several conditions for the first local system cohomology to vanish and to be at most one-dimensional, which generalize a result by Cohen-Dimca-Orlik. The conditions are described in terms of dis…
We prove a local index formula in conformal geometry by computing the Connes-Chern character for the conformal Dirac (twisted) spectral triple recently constructed by Connes-Moscovici. Following an observation of Moscovici, the computation reduces to the computation of the CM cocycle of an equivariant Dirac (ordinary) …
Compute local cohomology of vector fields on manifolds.
problem Understanding cohomology of vector fields on manifolds and complex manifolds.
method Compute local cohomology, use descent for cocycles.
result Explicit representatives for cocycles constructed.
We propose a reduction for non-convex optimization that can (1) turn an stationary-point finding algorithm into an local-minimum finding one, and (2) replace the Hessian-vector product computations with only gradient computations. It works both in the stochastic and the deterministic settings, without hurting the algor…
This paper is the first part of a series of three papers about the fundamental groups of conic-line arrangements consist of two tangented conics and up to two additional tangented lines. In this part, we compute the local braid monodromies and the local fundamental groups of the singularities which appear in such arran…
Researchers compute quasi-local mass of Kerr black hole horizon.
problem Computing the quasi-local mass of Kerr black hole horizons.
method Isometric embedding of Kerr black hole horizon in hyperbolic space.
result Quasi-local mass varies with Kerr black hole spin parameter.
Surveying locally homogeneous almost-Hermitian spaces with formulas for curvature.
problem Understanding the geometry of locally homogeneous almost-Hermitian spaces.
method Using the framework of varying Lie brackets to compute curvature of Gauduchon connections.
result Explicit formulas and examples for curvature of Gauduchon connections on locally homogeneous almost-Hermitian spaces.
We define the "localized index" of longitudinal elliptic operators on Lie groupoids associated to Lie algebroid cohomology classes. We derive a topological expression for these numbers using the algebraic index theorem for Poisson manifolds on the dual of the Lie algebroid. Underlying the definition and computation of …
The paper calculates the full asymptotics of analytic torsions for compact orbifolds.
problem Analytic torsions of compact locally symmetric orbifolds.
method Using Selberg's trace formula and geometric localization, the paper evaluates the heat trace and orbital integrals.
result Explicit formula for the asymptotic Ray-Singer analytic torsion of compact orbifolds.
New approach improves computational efficiency of Bass Local Volatility model.
problem Eliminate interpolation and improve computational efficiency in local volatility models.
method Combines local quadratic estimation and lognormal mixture tails for state price densities; uses trapezoidal rule for numerical convolutions.
result Proposed method outperforms traditional numerical methods in option pricing and market case studies.
The curve graphs are not locally finite. In this paper, we show that the curve graphs satisfy a property which is equivalent to graphs being uniformly locally finite via Masur--Minsky's subsurface projections. As a direct application of this study, we show that there exist computable bounds for Bowditch's slices on tig…
New research limits what GNNs can compute and generalizes their performance.
problem Limits of GNNs in computing graph properties and generalization bounds.
method Novel graph-theoretic formalism and data-dependent generalization bounds.
result Proves GNNs can't compute certain graph properties and provides tighter generalization bounds.
New local ID estimators based on data separability.
problem Estimating intrinsic dimensionality locally in multi-dimensional data.
method Local estimators based on concentration of measure.
result Empirical comparison with other ID estimators.
Localized sketching improves matrix multiplication and ridge regression complexity.
problem Efficiently approximate matrix multiplication and ridge regression with limited data availability.
method Localized sketching matrices for block diagonal structure, reducing sample complexity.
result Localized sketching achieves sample complexity matching global sketching methods.
In this paper, we propose a communication- and computation-efficient algorithm to solve a convex consensus optimization problem defined over a decentralized network. A remarkable existing algorithm to solve this problem is the alternating direction method of multipliers (ADMM), in which at every iteration every node up…
With the wealth of information produced by social networks, smartphones, medical or financial applications, speculations have been raised about the sensitivity of such data in terms of users' personal privacy and data security. To address the above issues, Federated Learning (FL) has been recently proposed as a means t…
Deep Local Volatilityq-fin.CP
Develops a deep learning method for enforcing no-arbitrage in local volatility surfaces.
problem No-arbitrage conditions not enforced in deep learning approaches for local volatility.
method Jointly interpolates European vanilla option prices, enforcing no-arbitrage through modified loss functions or network architectures.
result Demonstrates the effectiveness of enforcing no-arbitrage in local volatility surfaces using deep learning.
Debiasing Distributed Second Order Optimization with Surrogate Sketching and Scaled Regularizationcs.LG
New technique debiases distributed optimization, improving convergence rate.
problem Bias in local estimates limits effectiveness of distributed second order optimization.
method Surrogate sketching and scaled regularization to eliminate bias.
result The debiased local estimates lead to faster convergence in distributed optimization.
InstantEmbedding efficiently generates node representations with less computation and memory.
problem Efficiently generating local node representations for large graphs.
method Local PageRank computations in sublinear time.
result Significantly faster and less memory-intensive than traditional methods.
Convolutional networks outperform fully-connected ones in certain tasks.
problem Understanding the computational advantage of convolutional networks over fully-connected networks.
method Demonstrated a computational advantage through a specific problem class.
result Convolutional networks can solve certain problems that fully-connected networks cannot, even with gradient descent.