Research
On-device research index

arXiv research

A locally-built, LLM-digested index of recent arXiv papers in quant finance, geometry/topology, and statistical ML — keyword search served straight from SQLite on this machine.

168,695 papers · 148 categories

Trend · papers per month

3987951,1931,590 · Jun 202019922001200920172026
48 results for large-scale machine learning

New algorithms estimate Jacobian matrices for large-scale machine learning.

problem Efficiently computing search directions for large nonlinear least squares.
method Exploit low-rank structure in Hessian to estimate Jacobian matrices.
result Two algorithms perform well compared to state-of-the-art methods.

This paper compresses large datasets for efficient machine learning.

problem Efficiently processing large datasets for machine learning.
method Constructing a sketch of the dataset using random features and averaging, then learning from the sketch.
result The approach can perform machine learning tasks without full dataset access, preserving both information and privacy.

New machine learning method detects quantum separability in large-scale systems.

problem Deciding quantum separability of large-scale bipartite density matrices.
method Frank-Wolfe-based algorithm for finding nearest separable density matrices and classification of density matrices as separable or entangled.
result The method scales up to thousands of density matrices and achieves high quantum entanglement detection accuracy.

This paper provides a review and commentary on the past, present, and future of numerical optimization algorithms in the context of machine learning applications. Through case studies on text classification and the training of deep neural networks, we discuss how optimization problems arise in machine learning and what…

2016-06-15abs ↗pdf ↗

The paper simplifies influence computations for large-scale machine learning models.

problem Improving training efficiency and accuracy in large-scale models.
method Study influence functions, define memorization, simplify computations.
result Influence functions can be practical for large-scale models, indicating memorization.

Paper proposes data quality measures for large-scale high-dimensional data.

problem Lack of practical data quality measures for large-scale high-dimensional data.
method Proposes two data quality measures: class separability and in-class variability. Efficient algorithms based on random projections and bootstrapping are provided.
result Efficient algorithms for computing data quality measures on large-scale high-dimensional data.

Machine learning improves learning and memory retention by optimizing study sessions.

problem Improving learning and memory retention methods for factual material.
method Large-scale randomized controlled trial with machine learning optimization of study sessions.
result Study sessions optimized with machine learning lead to 67% longer retention and 50% higher return rate.

Accelerates MCMC sampling for large-scale problems using machine learning.

problem Efficiently sampling large-scale Bayesian inference problems with high computational cost.
method Integrates low-fidelity machine learning models into a multilevel MCMC framework.
result Significantly accelerates multilevel sampling by a factor of two with similar accuracy.

Feature selection is an important challenge in machine learning. It plays a crucial role in the explainability of machine-driven decisions that are rapidly permeating throughout modern society. Unfortunately, the explosion in the size and dimensionality of real-world datasets poses a severe challenge to standard featur…

2018-06-12abs ↗pdf ↗

New method speeds up learning of complex dynamical systems.

problem Efficiently learning large-scale dynamical systems from finite data.
method Random projections (sketching) to boost kernel-based Koopman operator estimators.
result The proposed estimators maintain accuracy while significantly reducing computation time.

Federated learning linked to mean-field games for large-scale learning.

problem Large-scale distributed and privacy-preserving learning algorithms.
method Established a connection between federated learning and mean-field games, presenting federated learning as a differential game.
result Properties of the equilibrium of the federated learning game were discussed.

New method uses tensor decompositions to overcome the curse of dimensionality for large-scale learning.

problem Large-scale machine learning problems with kernel methods.
method Deterministic Fourier features combined with low-rank tensor decomposition for tensor product structure.
result Demonstrated consistent performance and superior results compared to random Fourier features.

SOL is an open-source library for scalable online learning algorithms, and is particularly suitable for learning with high-dimensional data. The library provides a family of regular and sparse online learning algorithms for large-scale binary and multi-class classification tasks with high efficiency, scalability, porta…

2016-10-28abs ↗pdf ↗

In this paper we present MLaut (Machine Learning AUtomation Toolbox) for the python data science ecosystem. MLaut automates large-scale evaluation and benchmarking of machine learning algorithms on a large number of datasets. MLaut provides a high-level workflow interface to machine algorithm algorithms, implements a l…

2019-01-11abs ↗pdf ↗

Study evaluates machine learning methods for large-scale network reliability, revealing ANN's and PR's performance.

problem Tackles the NP-hard problem of approximating binary-state network reliability for large-scale systems.
method Compares 20 machine learning methods across three reliability regimes and evaluates their performance on large-scale networks.
result Large-scale networks with arc reliability ≥ 0.9 exhibit near-unity system reliability, enabling computational simplifications.

GGDA simplifies DA for large models, speeding up attribution by up to 50x.

problem Computational intensity of existing DA methods limits their applicability to large-scale models.
method Generalized Group Data Attribution (GGDA) framework attributing to groups of training points.
result GGDA achieves up to 50x speedups over standard DA methods while maintaining effectiveness.

Unified quadrature framework for large-scale kernel machines.

problem Efficiently approximating kernel functions for large-scale machine learning.
method Deterministic and randomized interpolatory rules for numerical integration of kernel functions.
result The proposed method reduces the number of nodes needed for accurate kernel approximation.

Deep learning methods have shown great promise in many practical applications, ranging from speech recognition, visual object recognition, to text processing. However, most of the current deep learning methods suffer from scalability problems for large-scale applications, forcing researchers or users to focus on small-…

2015-08-28abs ↗pdf ↗

Proposes MamBO for efficient high-dimensional large-scale optimization.

problem High-dimensional and large-scale optimization problems in machine learning and simulation.
method Combines subsampling and subspace embeddings with model aggregation to address uncertainty in surrogate models.
result Improves robustness of Bayesian optimization algorithm and achieves superior performance.

Framework for discovering treatment benefits in user segments.

problem Discovering differential impacts of treatments across user subgroups.
method Combines causal inference and machine learning for user segment discovery.
result Unified approach for treatment benefit discovery and assignment.

Hybrid approach combines user feedback and machine learning for predicting user satisfaction.

problem Measuring user satisfaction in large-scale conversational agent systems.
method Fusion of explicit user feedback and predictions from two machine-learned models trained on different data types.
result Hybrid approach significantly improves user satisfaction predictions.

Mango automates hyperparameter tuning for large-scale ML training.

problem Manual hyperparameter tuning is tedious and inefficient for large-scale machine learning.
method Parallel hyperparameter tuning with intelligent search strategies and flexible abstractions.
result Mango achieves comparable performance to Hyperopt while supporting distributed computing.

In this paper, we consider the problem of minimizing the average of a large number of nonsmooth and convex functions. Such problems often arise in typical machine learning problems as empirical risk minimization, but are computationally very challenging. We develop and analyze a new algorithm that achieves robust linea…

2018-05-11abs ↗pdf ↗

New approach for large-scale distributed learning systems that improve generalization performance.

problem Transitioning from centralized to distributed AI systems for complex learning tasks.
method Self-organizing hierarchical structuring mechanism based on agglomerative clustering, hierarchical generalization, and personalized learning.
result Demonstrates better generalization performance compared to conventional federated learning algorithms.

Novel LRMC tackles missing data and outliers in large-scale low-rank data recovery.

problem Missing data and extreme outliers in low-rank data analysis.
method Learned Robust Matrix Completion (LRMC) using deep unfolding and flexible neural network framework.
result LRMC achieves optimum performance with low computational complexity and linear convergence.

A new training method improves MLIPs for faster, lighter simulations.

problem High computational and memory costs of complex MLIPs for large-scale MD simulations.
method Teacher-student training framework using latent atomic energy knowledge.
result Lightweight student MLIPs achieve faster MD speeds and comparable accuracy to teachers.

This work aims to create a large-scale model for critical care time series data.

problem Lack of large-scale datasets and distribution shifts in critical care time series data.
method Harmonized dataset creation and transfer learning research.
result Established a foundation for large-scale multi-variate time series models in critical care.

New method infers causal factors from large-scale data without full graph reconstruction.

problem Inferring causal variables from large-scale systems without full causal graph reconstruction.
method Supervised learning on simulated data using a neural network and subsampled-ensemble inference.
result Efficiently identifies causal relationships in large-scale gene regulatory networks.

New method estimates bidirectional causal effects in large-scale systems.

problem Estimating bidirectional causal effects in systems with mutual dependence and heteroskedasticity.
method Heteroskedasticity-based identification with online kernel learning and random Fourier features.
result Superior accuracy and stability compared to single equation and polynomial approximations.

We present a hybrid algorithm for optimizing a convex, smooth function over the cone of positive semidefinite matrices. Our algorithm converges to the global optimal solution and can be used to solve general large-scale semidefinite programs and hence can be readily applied to a variety of machine learning problems. We…

2012-06-18abs ↗pdf ↗