Deep learning methods are useful for high-dimensional data and are becoming widely used in many areas of software engineering. Deep learners utilizes extensive computational power and can take a long time to train-- making it difficult to widely validate and repeat and improve their results. Further, they are not the b…
On-device research index
arXiv research
A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.
168,695 papers · 148 categories
Trend · papers per month
48 results for “Faster Learning”
Deeper neural networks learn lower frequency functions faster, according to a new principle.
problem Understanding why deeper learning is faster.
method Fourier analysis and filtering method to separate and analyze the frequency distribution of neural network outputs.
result Deeper hidden layers in neural networks bias towards lower frequency functions during training.
New machine learning model faster, more accurate, and can identify hard-to-classify samples.
problem Improving classification models in machine learning.
method Quadratic Multiform Separation approach.
result Produces comparable predictive accuracy, runs faster, and identifies hard-to-classify samples.
FIRE PBT improves neural network training by focusing on long-term performance.
problem Greedy decision mechanisms in PBT lead to poor long-term performance.
method FIRE PBT uses a fitness metric to encourage long-term performance over short-term improvements.
result FIRE PBT outperforms PBT on ImageNet and matches hand-tuned learning rates.
Faster Tsetlin Machines use clause indexing to speed inference and learning.
problem Overfitting and slow inference in Tsetlin Machines.
method Introduced a look-up table that indexes clauses based on feature falsification, enabling faster evaluation of clauses.
result Up to 15 times faster classification and three times faster learning on MNIST and Fashion-MNIST.
We propose a novel training algorithm for reinforcement learning which combines the strength of deep Q-learning with a constrained optimization approach to tighten optimality and encourage faster reward propagation. Our novel technique makes deep reinforcement learning more practical by drastically reducing the trainin…
Faster policy learning via continuous-time gradients.
problem Efficiently estimating policy gradients for continuous-time systems.
method Approximating continuous-time gradients directly, using adaptive discretization.
result More efficient policy gradient estimator leads to faster learning.
In this paper, we present the Bennett-type generalization bounds of the learning process for i.i.d. samples, and then show that the generalization bounds have a faster rate of convergence than the traditional results. In particular, we first develop two types of Bennett-type deviation inequality for the i.i.d. learning…
Many currently deployed Reinforcement Learning agents work in an environment shared with humans, be them co-workers, users or clients. It is desirable that these agents adjust to people's preferences, learn faster thanks to their help, and act safely around them. We argue that most current approaches that learn from hu…
Faster algorithms solve convex function learning problems.
problem Approximating arbitrary convex functions in various learning tasks.
method 2-block ADMM method with closed-form block computations.
result Converges with for convex Lipschitz regression.
This paper investigates the nonparametric regression problem using SVMs with anisotropic Gaussian RBF kernels. Under the assumption that the target functions are resided in certain anisotropic Besov spaces, we establish the almost optimal learning rates, more precisely, optimal up to some logarithmic factor, presented …
BPNNs learn to solve combinatorial problems faster and more accurately.
problem Generalizing belief propagation for efficient problem solving.
method BPNNs are parameterized operators that operate on factor graphs, generalizing BP. BPNN-D is a learned iterative operator that provably maintains BP's properties.
result BPNN-D converges 1.7x faster on Ising models and provides tighter bounds.
LALR adapts learning rate for faster convergence in regression and neural nets.
problem Finding optimal learning rates for faster convergence in regression and neural networks.
method Lipschitz continuity theory applied to Mean Absolute Error and Quantile loss functions.
result Adaptive learning rate policy enables up to 20x faster convergence.
We develop nested automatic differentiation (AD) algorithms for exact inference and learning in integer latent variable models. Recently, Winner, Sujono, and Sheldon showed how to reduce marginalization in a class of integer latent variable models to evaluating a probability generating function which contains many leve…
MP-Boost boosts accuracy faster and more interpretable than AdaBoost.
problem Developing a faster, more interpretable boosting method.
method Adaptive selection of minipatches (small subsets of instances and features) at each iteration.
result Achieves comparable accuracy to AdaBoost and gradient boosting but faster and more interpretable.
Faster convergence in federated learning for non-convex problems.
problem Accelerating convergence in federated learning for non-convex models.
method Reformulated federated learning as gradient-based method with biased gradients, proving convergence for non-convex problems and proposing an accelerated algorithm.
result Proved federated averaging algorithm converges for non-convex problems and proposed an accelerated federated learning algorithm with convergence guarantee.
New approach uses PDE learning for faster RL fine-tuning.
problem Learning optimal control policy for diffusion process.
method Solves variational inequality based on HJB equations.
result Shows fine-tuning can be done via supervised regression.
Simpler algorithm learns shallow networks faster.
problem Learning a linear combination of ReLU activations.
method A simpler one-stage algorithm with improved runtime.
result Runs in time.
FROST speeds up and stabilizes one-shot semi-supervised learning.
problem Slow training and sensitivity to labeled data choices in semi-supervised learning.
method Combines semi-supervised learning with a one-stage, single network self-training approach.
result FROST trains up to 10x faster and is more robust to labeled data choices.
Enhanced Markov chain sampler learns network statistics faster.
problem Learning network statistics efficiently.
method Integrates graph Forman curvature into Markov chain transition probabilities and stationary distribution.
result Curved Markov chain Monte Carlo achieves faster convergence.
There is a general trend towards solving problems suited to deep learning with more complex deep learning architectures trained on larger training sets. This requires longer compute times and greater data parallelization or model parallelization. Both data and model parallelism have been historically faster in paramete…
We design a non-convex second-order optimization algorithm that is guaranteed to return an approximate local minimum in time which scales linearly in the underlying dimension and the number of training examples. The time complexity of our algorithm to find an approximate local minimum is even faster than that of gradie…
FedCluster accelerates federated learning convergence by cycling device groups.
problem Federated learning convergence issues with device-level data heterogeneity.
method FedCluster groups devices into clusters that cycle through learning rounds, boosting convergence with meta-updates.
result FedCluster achieves faster convergence in nonconvex optimization compared to FedAvg.
New algorithm learns disjunctions faster than previous methods.
problem Learning Boolean disjunctions in the agnostic PAC model.
method Developed an agnostic learner with complexity .
result First separation between SQ and CSQ models in distribution-free agnostic learning.
New method speeds up lifelong learning of complex tasks.
problem Catastrophic forgetting in lifelong learning.
method Directly trains lifelong function approximators via policy gradients.
result Faster convergence and better policies achieved.
Proposes a method to train neural networks that solve differential equations faster.
problem Training neural networks that solve differential equations becomes computationally expensive.
method Introduces a differentiable surrogate for numerical solver time cost using higher-order derivatives.
result Trains models that are faster to solve while maintaining nearly the same accuracy.
New algorithms optimize faster for simpler problems in machine learning.
problem Design faster optimization algorithms for practical machine learning problems.
method Adaptive gradient algorithms with refined conditions based on Hessian singular values.
result Achieved optimal gradient complexity for linear regression.
Simplifies IV regression for high-dimensional instruments.
problem Nonlinear instrumental variable regression with high-dimensional instruments.
method Combines kernelized IV methods with an adaptive regression algorithm.
result Faster convergence and adaptability to feature dimensionality.
Determinantal point processes (DPPs) have garnered attention as an elegant probabilistic model of set diversity. They are useful for a number of subset selection tasks, including product recommendation. DPPs are parametrized by a positive semi-definite kernel matrix. In this work we present a new method for learning th…
Increasing urban concentration raises operational challenges that can benefit from integrated monitoring and decision support. Such complex systems need to leverage the full stack of analytical methods, from state estimation using multi-sensor fusion for situational awareness, to prediction and computation of optimal r…
Riemannian stochastic gradient descent converges faster with increasing batch size.
problem Improving convergence rate of Riemannian stochastic gradient descent.
method Theoretical analysis and numerical investigation of increasing batch size effects.
result Riemannian stochastic gradient descent converges faster with increasing batch size.
Learning-based hashing algorithms are ``hot topics" because they can greatly increase the scale at which existing methods operate. In this paper, we propose a new learning-based hashing method called ``fast supervised discrete hashing" (FSDH) based on ``supervised discrete hashing" (SDH). Regressing the training exampl…
Two new algorithms speed up TreeSHAP computation for tree-based models.
problem Slow computation of SHAP values on tree-based models.
method Two new algorithms, Fast TreeSHAP v1 and v2, designed to improve computational efficiency.
result Fast TreeSHAP v2 is 2.5x faster than TreeSHAP, with slightly higher memory usage.
Stochastic methods with coordinate-wise adaptive stepsize (such as RMSprop and Adam) have been widely used in training deep neural networks. Despite their fast convergence, they can generalize worse than stochastic gradient descent. In this paper, by revisiting the design of Adagrad, we propose to split the network par…
We investigate the learning rate of multiple kernel learning (MKL) with and elastic-net regularizations. The elastic-net regularization is a composition of an -regularizer for inducing the sparsity and an -regularizer for controlling the smoothness. We focus on a sparse setting where the total …
Faster, better sparse model estimation for large datasets.
problem Sparse model estimation for large datasets with millions of samples and features.
method Coordinate descent, working sets, Anderson acceleration.
result Significantly faster and more efficient than state-of-the-art algorithms.
When applied to training deep neural networks, stochastic gradient descent (SGD) often incurs steady progression phases, interrupted by catastrophic episodes in which loss and gradient norm explode. A possible mitigation of such events is to slow down the learning process. This paper presents a novel approach to contro…
Paper proposes a faster SPIDER-EM variant for large-scale nonconvex optimization.
problem High computational cost of EM algorithm in large-scale learning.
method Extension of SPIDER-EM for nonconvex finite-sum optimization problems.
result Achieves state-of-the-art complexity bounds and linear convergence under certain conditions.
SAPPHIRE: Preconditioned Stochastic Variance Reduction for Faster Large-Scale Statistical Learningstat.ML
SAPPHIRE tackles ill-conditioned rERM problems with faster convergence.
problem Ill-conditioned objectives and non-smooth regularizers in rERM.
method Sketch-based preconditioning and scaled proximal mapping.
result Achieves condition-number-free linear convergence.
A novel neural network training method reduces gradient variance for faster and better reinforcement learning.
problem Improving convergence and generalization in deep reinforcement learning.
method Gradient Monitoring (GM) approach to dynamically adjust the learning process based on feedback.
result The proposed methods, especially AM-WGM, significantly enhance model performance and generalization.
MDLdroid improves mobile deep learning for personal sensing with faster training.
problem Continuous local changes and resource constraints in personal mobile sensing affect global model performance.
method ChainSGD-reduce approach to reduce overhead and balance resources.
result 2x to 3.5x faster training on off-the-shelf mobile devices compared to single-device training.
Low-rank extended Kalman filtering for online learning of neural networks from streaming datastat.ML
Efficiently learns neural network parameters from streaming data.
problem Online learning of neural networks from non-stationary data streams.
method Low-rank extended Kalman filtering for approximate Bayesian inference.
result Significantly faster learning and adaptation to changing distributions.
Object detection in streaming images is a major step in different detection-based applications, such as object tracking, action recognition, robot navigation, and visual surveillance applications. In mostcases, image quality is noisy and biased, and as a result, the data distributions are disturbed and imbalanced. Most…
Quantum Banditscs.LG
Quantum algorithm solves best arm identification problem faster.
problem Identifying the best option in a set of choices.
method Quantum amplitude amplification algorithm.
result Quadratically faster than classical methods.
Faster weak supervision framework using triplet methods.
problem Computational inefficiency in weak supervision models.
method Closed-form solution for latent variable models, avoiding iterative methods.
result Orders of magnitude faster than previous approaches.
Stagewise training strategy is widely used for learning neural networks, which runs a stochastic algorithm (e.g., SGD) starting with a relatively large step size (aka learning rate) and geometrically decreasing the step size after a number of iterations. It has been observed that the stagewise SGD has much faster conve…
A new algorithm speeds up matrix multiplication without actual multiplication.
problem Efficiently multiplying matrices in machine learning.
method Learning-based algorithm that uses hashing, averaging, and byte shuffling.
result Often runs 100x faster than exact matrix products and 10x faster than current approximate methods.
The paper introduces MDP homomorphic networks for faster reinforcement learning.
problem Current reinforcement learning approaches do not exploit symmetries in the joint state-action space.
method Equivariant neural networks with group-structured symmetries (reflections, rotations).
result MDP homomorphic networks converge faster than unstructured baselines on various tasks.