This paper improves parallel belief propagation for scalable machine learning.
problem Efficient parallelization of belief propagation for large-scale machine learning tasks.
method Use of scalable relaxed schedulers to parallelize belief propagation.
result Our approach outperforms previous methods in scalability and convergence time.
SAFE automates feature engineering for industrial tasks efficiently and scalably.
problem Efficiency and scalability of automatic feature engineering methods for industrial tasks.
method SAFE (Scalable Automatic Feature Engineering) method, which provides excellent efficiency and scalability.
result SAFE method provides prominent efficiency and competitive effectiveness in industrial tasks.
The paper discusses scalable learning for wireless data-driven systems.
problem Expanding data volume and model complexity limit centralized learning solutions.
method Discusses scalable architecture and local learning strategies.
result Promising research directions in scalable data-driven wireless communications.
This study improves scalability of randomized smoothing for certifying classifier robustness.
problem Certifying machine learning classifiers against adversarial attacks is challenging and scalable solutions are needed.
method The study reviews and explores randomized smoothing and its derivatives, focusing on scalability.
result The study provides theoretical guarantees and discusses scalability challenges of randomized smoothing.
New framework uses conformal predictions for robust, scalable machine learning classification.
problem Developing robust and reliable machine learning models for classification.
method Introducing scalable classifiers linked to statistical order theory and probabilistic learning theory, defining a score function and conformal safety set.
result Demonstrated practical implications in cybersecurity for identifying DNS tunneling attacks.
Scalable machine learning with path signatures for time series and graphs.
problem Challenges in real-world time series and graph data.
method Combines rough path theory with probabilistic, deep, and kernel methods.
result Scalable models for time series and graph data.
SOL is an open-source library for scalable online learning algorithms, and is particularly suitable for learning with high-dimensional data. The library provides a family of regular and sparse online learning algorithms for large-scale binary and multi-class classification tasks with high efficiency, scalability, porta…
MLI is an Application Programming Interface designed to address the challenges of building Machine Learn- ing algorithms in a distributed setting based on data-centric computing. Its primary goal is to simplify the development of high-performance, scalable, distributed algorithms. Our initial results show that, relativ…
Explosive growth in data and availability of cheap computing resources have sparked increasing interest in Big learning, an emerging subfield that studies scalable machine learning algorithms, systems, and applications with Big Data. Bayesian methods represent one important class of statistic methods for machine learni…
Applying machine learning techniques to the quickly growing data in science and industry requires highly-scalable algorithms. Large datasets are most commonly processed "data parallel" distributed across many nodes. Each node's contribution to the overall gradient is summed using a global allreduce. This allreduce is t…
A scalable method for econometric inference using machine learning for big data.
problem Interpreting large, often black-box, economic data.
method Variational Bayesian Inference for time-varying parameter auto-regressive models.
result The model can handle large datasets and is scalable for big data.
VQC-MLPNet combines quantum and classical elements for scalable quantum machine learning.
problem Challenges in expressivity, trainability, and noise resilience of VQCs.
method Hybrid architecture with a VQC generating weights for a classical MLP during training.
result Improved expressivity, trainability, and robustness compared to standalone quantum or hybrid approaches.
We propose a distributed approach to train deep neural networks (DNNs), which has guaranteed convergence theoretically and great scalability empirically: close to 6 times faster on instance of ImageNet data set when run with 6 machines. The proposed scheme is close to optimally scalable in terms of number of machines, …
In an effort to overcome the data deluge in computational biology and bioinformatics and to facilitate bioinformatics research in the era of big data, we identify some of the most influential algorithms that have been widely used in the bioinformatics community. These top data mining and machine learning algorithms cov…
LIBS2ML is a library based on scalable second order learning algorithms for solving large-scale problems, i.e., big data problems in machine learning. LIBS2ML has been developed using MEX files, i.e., C++ with MATLAB/Octave interface to take the advantage of both the worlds, i.e., faster learning using C++ and easy I/O…
Quantum machine learning: Adiabatic quantum SVM outperforms classical methods.
problem Training support vector machines efficiently on large datasets.
method Adiabatic quantum computing for SVM training.
result Quantum approach outperforms classical methods in accuracy and scalability.
We propose scalable methods to execute counting queries in machine learning applications. To achieve memory and computational efficiency, we abstract counting queries and their context such that the counts can be aggregated as a stream. We demonstrate performance and scalability of the resulting approach on random quer…
Improved scalable machine learning under heavy-tailed data.
problem Machine learning scalability under heavy-tailed data without strong convexity.
method Simple robust validation sub-routine to boost confidence in gradient-based sub-processes.
result Substantial improvement in dimension dependence without strong convexity.
Develops probabilistic safety regions for scalable classifiers.
problem Minimizing misclassification errors in supervised classification.
method Introduces probabilistic safety regions and scalable classifiers.
result Probabilistic certifications for classifier performance.
New approach to bilevel optimization for machine learning using functional methods.
problem Solving bilevel optimization problems in machine learning, especially with over-parameterized neural networks.
method Functional point of view, scalable and efficient algorithms for functional bilevel optimization.
result Demonstrates benefits of functional approach on instrumental regression and reinforcement learning tasks.
In this paper, we present iPrescribe, a scalable low-latency architecture for recommending 'next-best-offers' in an online setting. The paper presents the design of iPrescribe and compares its performance for implementations using different real-time streaming technology stacks. iPrescribe uses an ensemble of deep lear…
New scalable methods for robust model learning from large datasets.
problem Training robust models resistant to data distribution shifts.
method Composite optimization for distributionally robust optimization (DRO).
result Scalable methods for learning robust models from large datasets.
Quantum Gaussian processes enable scalable quantum learning.
problem Lack of simple, interpretable, scalable learning frameworks for quantum data.
method Bayesian framework using Gaussian processes with quantum kernels.
result Provable and scalable quantum Gaussian processes for quantum learning.
New algorithm discovers causal graphs efficiently from observational data.
problem Discovering causal graphs from observational data efficiently.
method Approximating the score function using machine learning and applying scalable techniques.
result DAS algorithm reduces complexity and achieves competitive accuracy.
Enhances parallelism in decentralized learning for larger networks.
problem Scalability limitations in decentralized learning with increasing number of machines.
method Proposes Decentralized Anytime SGD, a novel algorithm that extends parallelism threshold.
result Establishes a theoretical upper bound on parallelism surpassing current state-of-the-art.
This dissertation tackles challenges in reliable machine learning measurement.
problem Challenges in reproducibility, scalability, and uncertainty quantification in machine learning.
method Develops criteria for meaningful metrics and methodologies for scalable, reliable measurement.
result Provides methods for evaluating generative-AI systems and quantifying memorization.
New method for scalable learning of IRT models from large datasets.
problem Efficiently learning latent variables in IRT models from large numbers of examinees and items.
method Leveraging logistic regression and coresets for scalable IRT training.
result Scalable learning of IRT models from large data is achieved.
The book covers scalable MCMC methods for Bayesian learning.
problem Scalability issues in Bayesian learning with large datasets.
method Advanced MCMC algorithms, including stochastic gradient, non-reversible, and continuous time methods.
result Substantial advances in practical and theoretical Bayesian computation.
Bayesian learning made scalable with posteriors library.
problem Computational challenges in Bayesian learning with modern models.
method Introducing posteriors library and tempered MCMC.
result Bayesian approximations are useful and scalable.
FSGD uses latent factors to scale SGD for high-dimensional learning.
problem Scalable optimization in high-dimensional machine learning.
method Factor-Augmented SGD (FSGD) that operates on streaming data.
result Established theoretical framework for latent factor estimation error in SGD.
Quantum ELMs use a quantum reservoir to learn from data, with limits on expressivity and scalability.
problem Understanding the limits of quantum ELMs for machine learning tasks.
method Decomposed QELM predictions into Fourier series to analyze expressivity and scalability.
result Expressivity of QELMs is limited by the number of Fourier frequencies and observables, and scalability is hindered by hardware noise and entanglement.
Combines ML and KB modeling for large chaotic systems.
problem Predicting large, complex, spatiotemporal systems with limited data.
method Parallel ML prediction and hybrid approach combining ML and KB.
result Excellent performance and reduced training data needed.
This paper surveys large-scale machine learning methods for efficient data analysis.
problem Efficiently processing large-scale data with machine learning models.
method Divided into three categories: model simplification, optimization approximation, and computation parallelism.
result Blueprint for future developments in large-scale machine learning.
Improved particle-flow event reconstruction for future colliders using scalable neural networks.
problem Efficient and accurate particle reconstruction in future particle detectors.
method Comparative study of scalable machine learning models (graph neural network and kernel-based transformer) for event reconstruction.
result Graph neural network model improves jet transverse momentum resolution by up to 50%.
New method speeds up deep learning optimization.
problem Scalable second-order optimization for deep learning.
method Second-order optimization with algorithmic and numerical improvements.
result Significant convergence and wall-clock time improvements.
A scalable parallel BO method for asynchronous settings.
problem Expensive-to-evaluate problems in machine learning.
method Simple and scalable Bayesian optimization method for asynchronous parallel settings.
result Demonstrated promising performance on benchmark functions and hyperparameter optimization.
Develops coresets for scalable multivariate distribution estimation.
problem Handling large-scale data in non-parametric or semi-parametric regression and density estimation.
method Novel coreset construction for multivariate conditional transformation models (MCTMs).
result Substantial data reduction with high log-likelihood accuracy.
COPML framework securely trains models across multiple data owners without revealing individual data.
problem Privacy-preserving collaborative machine learning with multiple data owners.
method Securely encodes data, distributes computation, performs distributed training.
result Achieves up to 16x speedup in training time while maintaining strong privacy.
Develops Bayesian filtering for online learning and related problems.
problem Sequential machine learning challenges, especially non-stationarity, model misspecification, and high dimensionality.
method Modular adaptive framework, provably robust filter, and sequential parameter updates.
result Improved performance in dynamic, high-dimensional, and misspecified models.
The optimization of expensive-to-evaluate black-box functions over combinatorial structures is an ubiquitous task in machine learning, engineering and the natural sciences. The combinatorial explosion of the search space and costly evaluations pose challenges for current techniques in discrete optimization and machine …
We describe TF-Replicator, a framework for distributed machine learning designed for DeepMind researchers and implemented as an abstraction over TensorFlow. TF-Replicator simplifies writing data-parallel and model-parallel research code. The same models can be effortlessly deployed to different cluster architectures (i…
New algorithm improves clustering accuracy without sacrificing scalability.
problem Improving clustering accuracy for large datasets.
method Nonnegative low-rank semidefinite programming with Burer-Monteiro factorization.
result Significantly smaller mis-clustering errors compared to existing methods.
EASTER improves OCR efficiency and scalability.
problem Efficient and scalable Optical Character Recognition (OCR) for machine printed and handwritten text.
method 1-D convolutional layers without recurrence, parallel training, synthetic dataset generation.
result EASTER achieves comparable performance to complex RNN models with less data and outperforms them on benchmark datasets.
Study evaluates scalability and real-world impact of disentangled representations.
problem Scalability and real-world impact of disentangled representations.
method New high-resolution dataset and architectures for disentangled representation learning.
result Disentanglement predicts out-of-distribution task performance.
New method for efficient sketching of gradients and Hessians.
problem Memory constraints in training machine learning models.
method A novel framework for scalable gradient and HVP sketching tailored for modern hardware.
result Theoretical guarantees and practical applications in training data attribution and Hessian spectrum analysis.
New algorithm reduces bias in trained models, near-optimal performance proven.
problem Reduction of bias in trained machine learning models.
method Scalable post-processing algorithm for debiasing trained models, including deep neural networks (DNNs).
result Proven to be near-optimal by bounding its excess Bayes risk.
Kolmogorov-Arnold Networks promise scalable performance in high dimensions.
problem Curse of dimensionality in multilayer perceptrons.
method Kolmogorov-Arnold representation theorem and interpolation methods.
result Kolmogorov-Arnold Networks achieve true freedom from the curse of dimensionality.
A variety of large-scale machine learning problems can be cast as instances of constrained submodular maximization. Existing approaches for distributed submodular maximization have a critical drawback: The capacity - number of instances that can fit in memory - must grow with the data set size. In practice, while one c…