This technical report describes an efficient technique for computing the norm of the gradient of the loss function for a neural network with respect to its parameters. This gradient norm can be computed efficiently for every example.
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,657 papers · 148 categories
Trend · papers per month
48 results for “Efficient computation”
Efficient WKNN-Shapley computation improves data valuation accuracy.
problem Efficient computation of Data Shapley for WKNN algorithm.
method Reframed WKNN-Shapley as a counting problem, introduced quadratic-time algorithm.
result Quadratic-time WKNN-Shapley computation, improving from .
SCQRNN prevents quantile crossing and improves computational efficiency.
problem Quantile crossing issue in regression models.
method Integrates ad hoc sorting in training to prevent quantile crossing and enhance computational efficiency.
result SCQRNN achieves faster convergence and non-intersecting quantiles.
Quantum computing offers energy savings over classical computing.
problem Energy efficiency in computing services.
method Cournot competition model constrained by energy usage.
result Quantum computing firms can outperform classical counterparts in energy efficiency.
Efficiently scales continuous kernels with sparse Fourier domain learning.
problem High computational and memory demands, spectral bias in continuous kernels.
method Sparse learning in the Fourier domain.
result Efficient scaling of continuous kernels, reduced computational and memory requirements, mitigated spectral bias.
Efficiently approximates higher-order derivatives for generative models.
problem Expensive computation of higher-order derivatives in generative models.
method Rewrite SM objective in terms of directional derivatives and use finite difference for efficient approximation.
result Comparable results to gradient-based methods but significantly more computationally efficient.
Efficient Decomposition of Forman-Ricci Curvature on Vietoris-Rips Complexes and Data Applicationsmath.GT
Researchers decompose Forman-Ricci curvature for efficient computation in VR complexes.
problem Efficiently computing Forman-Ricci curvature in higher-dimensional data.
method Decomposition and set-theoretical proof for local computation of FRC in VR complexes.
result Reveals critical geometric insights overlooked by conventional techniques.
HeteroFL trains diverse clients with varying capabilities efficiently.
problem Training models on clients with different computational and communication capacities.
method Proposes HeteroFL framework to adaptively distribute subnetworks based on client capabilities.
result Adaptive subnetwork distribution leads to efficient computation and communication.
This paper improves computational efficiency in kernel ridge regression under covariate shift.
problem Covariate shift in nonparametric regression.
method Random projections in RKHS to reduce computational demands.
result Significant computational savings can be achieved without compromising learning performance under covariate shift.
BiQGEMM efficiently multiplies quantized DNN weights using lookup tables.
problem Efficiently multiplying quantized DNN weights on CPUs/GPUs with limited memory.
method BiQGEMM pre-computes and stores redundant intermediate results in lookup tables.
result BiQGEMM achieves lower overall computations and higher performance.
Efficiently discovers Bayesian network structure with reduced memory usage.
problem Bayesian network structure discovery is NP-hard and memory-intensive.
method Progressively leveled scoring approach with hierarchical computation.
result Achieved processing of a 28-variable Bayesian network using only memory.
A new method for efficient computation of Knowledge Gradient in Bayesian optimization.
problem Efficient computation of the Knowledge Gradient for Bayesian optimization.
method One-shot Hybrid KG, a new approach combining previous ideas.
result The new method is cheap to compute and preserves theoretical properties of previous methods.
AI progress measured by reduced compute needed to reach past performance.
problem Quantifying algorithmic progress in AI.
method Analysis of floating-point operations required to train neural networks.
result Algorithmic efficiency doubled every 16 months over 7 years.
Optical Echo State Network Reservoir Computingphysics.optics
Optical ESNs enable flexible, efficient machine learning with reduced energy.
problem Implementing universal computational capabilities in machine learning.
method Optical implementation of ESNs leveraging stimulated Brillouin scattering.
result Efficient, scalable, and memory-capable optical reservoir computing.
Efficient algorithm computes knot invariants quickly.
problem Computing finite type invariants efficiently for knots.
method Create look-up tables for subdiagrams indexed by dyadic intervals, then compute invariants in time.
result Finite type invariants can be computed on an -crossing knot in time, significantly faster than previous methods.
A new method uses neural tangent kernel to efficiently compute MMD statistic.
problem Efficiently computing Maximum Mean Discrepancy (MMD) statistic with low memory and computational complexity.
method Identifies a connection between neural tangent kernel (NTK) and MMD to develop a computationally and memory-efficient approach.
result The proposed NTK-MMD statistic is validated through numerical experiments on synthetic and real-world datasets.
This paper provides efficient algorithms for computing entropy and KL divergence in Bayesian networks.
problem Computing entropy and KL divergence for Bayesian networks efficiently.
method Leveraging the graphical structure of Bayesian networks, the paper provides computationally efficient algorithms.
result Reduces computational complexity of KL divergence from cubic to quadratic for Gaussian BNs.
E2GC optimizes energy efficiency in DNNs by balancing computational and data movement costs.
problem Imbalance between computational complexity and data reuse in GConv leads to suboptimal energy efficiency.
method Developed an optimum group size model and proposed E2GC module with constant group size.
result E2GC modules improve energy efficiency by 10.8% and 4.73% on P100 and P4000 GPUs, respectively.
New methods improve Reservoir Computing for chaotic time series prediction.
problem Chaotic time series prediction in Reservoir Computing.
method Established Recurrent Kernel limit, introduced Structured Reservoir Computing.
result Structured Reservoir Computing is faster and more memory-efficient.
Fiber simplifies RL and population-based methods for distributed training.
problem Challenges in RL and population-based methods, including frequent interaction with simulations and dynamic scaling.
method Introducing Fiber, a scalable distributed computing framework.
result Significantly expands accessibility of large-scale parallel computation.
PALMS reconstructs large-scale networks efficiently with parallel computing.
problem Reconstructing large-scale latent networks from observed dynamics is computationally challenging.
method PALMS (Parallel Adaptive Lasso with Multi-directional Signals) framework for distributed network reconstruction.
result PALMS substantially reduces computational complexity and storage requirements.
New findings show limitations in converting private learning to online learning efficiently.
problem Limitations in converting private learning to online learning efficiently.
method Assuming one-way functions, we show an efficient conversion from pure-private learners to online learners is impossible.
result Efficient conversion from pure-private learners to online learners is impossible under certain assumptions.
Study efficient interactive learning for structured outputs with reliable computation.
problem Interactive learning with noisy labels and structured output spaces.
method Identify and utilize CRISPs (probabilistic models) that guarantee reliable and efficient computation of probabilistic quantities.
result CRISPs enable robust and efficient active and skeptical learning in large structured output spaces.
Efficiently calibrates computationally expensive models using vine copulas.
problem Computational models are expensive and hard to calibrate with real data.
method Variational Bayes inference with vine copulas for dependent data.
result Computational scalability and efficiency of the proposed algorithm.
Memory-efficient learning for large-scale imaging systems.
problem Memory limitations in GPUs for real-world large-scale inverse problems.
method Exploits reversibility of network layers to enable data-driven design.
result Demonstrated on small-scale and large-scale real-world systems.
A new method for computing image curvature efficiently and accurately.
problem Low performance, low accuracy, and requirement of second order differentiability in conventional computation schemes.
method Proposes a novel discrete computation scheme for weighted Gaussian curvature.
result More accurate, computationally more efficient, and does not require second order differentiability.
Existing methods for reducing the computational burden of neural networks at run-time, such as parameter pruning or dynamic computational path selection, focus solely on improving computational efficiency during inference. On the other hand, in this work, we propose a novel method which reduces the memory footprint and…
An Efficient Algorithm for Computing Interventional Distributions in Latent Variable Causal Modelscs.LG
Probabilistic inference in graphical models is the task of computing marginal and conditional densities of interest from a factorized representation of a joint probability distribution. Inference algorithms such as variable elimination and belief propagation take advantage of constraints embedded in this factorization …
Many neural speech enhancement and source separation systems operate in the time-frequency domain. Such models often benefit from making their Short-Time Fourier Transform (STFT) front-ends trainable. In current literature, these are implemented as large Discrete Fourier Transform matrices; which are prohibitively inef…
Deep RNNs compute American option prices and deltas efficiently.
problem Computing prices and deltas of high-dimensional American options.
method Two deep RNNs, one for price and one for delta, learn over spacetime.
result Linear time and constant memory cost compared to feedforward networks.
Information-Computation Tradeoffs for Learning Margin Halfspaces with Random Classification Noisecs.LG
Study shows a tradeoff between sample complexity and computational efficiency for learning halfspaces with random noise.
problem PAC learning γ-margin halfspaces with Random Classification Noise.
method Established an information-computation tradeoff and provided a simple efficient algorithm with sample complexity O(1/(γ^2 ε^2)). Also, proved lower bounds for SQ algorithms and low-degree polynomial tests.
result Inherent gap between sample complexity and computational efficiency for learning halfspaces with random noise.
MBQC linked to CQCA, yielding efficient Ansätze.
problem Quantum computation efficiency and Ansatz adaptation.
method Relating MBQC to CQCA and constructing Ansätze.
result MBQC Ansätze can lead to different performances on learning tasks.
Data-driven configuration tuning of glmnet for balancing accuracy and computational efficiencystat.CO
Optimizes glmnet configuration for better accuracy and efficiency.
problem Inappropriate glmnet configuration leads to inaccurate solutions and increased computation time.
method Data-driven framework using neural networks to predict accuracy and computation time from dataset characteristics and configuration.
result Automatic selection of optimal configuration maximizing accuracy under a time constraint.
Gradients of neural networks can be computed efficiently for any architecture, but some applications require differential operators with higher time complexity. We describe a family of restricted neural network architectures that allow efficient computation of a family of differential operators involving dimension-wise…
Communication-Efficient Federated Hypergradient Computation via Aggregated Iterative Differentiationcs.LG
Paper proposes AggITD for efficient federated hypergradient computation.
problem Computing hypergradient in federated settings is challenging due to distributed and nonlinear construction of global Hessian matrices.
method AggITD: a novel communication-efficient federated hypergradient estimator via aggregated iterative differentiation.
result AggITD achieves the same sample complexity as AID-based approaches but with fewer communication rounds, especially in heterogeneous data environments.
Market competition depends on computational complexity, P != NP makes it impossible.
problem Competitive market outcomes require computational intractability.
method Analyzes the computational hardness of collusion detection in markets.
result If P != NP, collusion detection is computationally infeasible, making collusion unstable.
We discuss the relative merits of optimistic and randomized approaches to exploration in reinforcement learning. Optimistic approaches presented in the literature apply an optimistic boost to the value estimate at each state-action pair and select actions that are greedy with respect to the resulting optimistic value f…
Sparsity helps reduce diffusion model costs.
problem High computational costs in diffusion models.
method Introduced sparsity concept to reduce input dimensionality.
result Sparsity reduces computational complexity to intrinsic data dimension.
Matrix profile has been recently proposed as a promising technique to the problem of all-pairs-similarity search on time series. Efficient algorithms have been proposed for computing it, e.g., STAMP, STOMP and SCRIMP++. All these algorithms use the z-normalized Euclidean distance to measure the distance between subsequ…
EvoGrad improves efficiency in meta-learning and hyperparameter optimization.
problem Efficiently compute hypergradients for larger network architectures.
method Uses evolutionary techniques to estimate hypergradients without second-order derivatives or longer computational graphs.
result Significant improvements in efficiency, enabling scaling to bigger architectures.
DREAM model improves computational efficiency for non-linear effects in relational event models.
problem Efficiently modeling non-linear effects in dynamic relational networks.
method Introduces Deep Relational Event Additive Model (DREAM) using Neural Additive Models.
result Demonstrates superior computational efficiency compared to traditional REM approaches.
Efficiently computes quasiconcave envelope with limited data.
problem Approximating unknown quasiconcave function with partial information.
method Solves value problem and interpolation problem with polynomial and logarithmic LPs.
result Efficiently computes quasiconcave envelope with limited data.
SIM-Shapley improves SV approximation efficiency and stability.
problem High computational costs of Shapley value methods in high-dimensional settings.
method Stochastic Iterative Momentum for Shapley Value Approximation (SIM-Shapley).
result Reduced computation time by up to 85% while maintaining feature attribution quality.
Photonic chip speeds up option pricing with GAN for financial efficiency.
problem Bottleneck in classical computing limits financial industry development.
method Unary approach, photonic chip, quantum amplitude estimation, GAN for asset distribution.
result Quadratic speedup over classical Monte Carlo methods.
Optimizes model training efficiency with core subset selection.
problem Escalating computational costs in large dataset training.
method Core subset selection for reweighting.
result Efficiently minimizes computational time and improves model performance.
The Signature Kernelmath.PR
Signature kernel handles sequential data with theoretical and practical advantages.
problem Handling sequential data efficiently and accurately.
method Positive definite kernel based on stochastic analysis with efficient computation.
result Strong empirical performance and theoretical guarantees.
New method approximates CV efficiently for large-scale problems.
problem High computational cost of standard CV in large-scale problems.
method Iterative first-order algorithm to approximate CV solution.
result Extends CV approximation guarantees to non-converged solutions.
Efficiently solves MRF inference problems with semidefinite programming.
problem Computing partition function or MAP estimate in binary and multi-class MRFs.
method Coordinate-descent-based fast semidefinite solver for SDPs.
result Substantially outperforms existing state-of-the-art methods in approximate inference.