Bayesian optimization sped up with scalable Gaussian processes.
problem Optimizing functions with derivative information and large datasets.
method Combines derivative acceleration and scalable Gaussian process models.
result Significant speedup in optimization convergence for large datasets.
\emph{Scalable spaces} are simply connected compact manifolds or finite complexes whose real cohomology algebra embeds in their algebra of (flat) differential forms. This is a rational homotopy invariant property and all scalable spaces are formal; indeed, scalability can be thought of as a metric version of formality.…
As a non-parametric Bayesian model which produces informative predictive distribution, Gaussian process (GP) has been widely used in various fields, like regression, classification and optimization. The cubic complexity of standard GP however leads to poor scalability, which poses challenges in the era of big data. Hen…
Scalable3-BO tackles scalability issues in Bayesian optimization for big data and high dimensions.
problem Bayesian optimization scalability issues in big data and high dimensions.
method Sparse Gaussian process, random embedding, asynchronous parallelization.
result Scalable3-BO framework optimizes high-dimensional problems with 1 million data points and 10,000 dimensions.
New scalable Lipschitz bounds improve neural network robustness analysis.
problem Computing tight Lipschitz bounds for deep neural networks is challenging and computationally expensive.
method Derived new closed-form Lipschitz bounds using more general feasible points of LipSDP, avoiding SDP solvers.
result Improved scalability and precision of Lipschitz estimation for large neural networks.
New GP-VAE model improves scalability and performance.
problem Inability of conventional VAEs to model correlations between data points.
method Principled sparse inference approaches to improve scalability of GP-VAEs.
result New model outperforms existing approaches in runtime and memory usage.
Paper introduces a fast, robust, scalable method for detecting changes in data streams.
problem Detecting changes in data streams efficiently and reliably.
method Bayesian online changepoint detection with provable robustness and scalability.
result The proposed method is more than 10 times faster than previous approaches and provides provable robustness.
SIGL learns scalable graphons from graphs.
problem Estimating graphons from graphs of varying sizes.
method Combines INRs and GNNs for scalable graphon estimation.
result SIGL learns consistent graphons at arbitrary resolutions.
Scalable TS using sparse GPs improves efficiency without sacrificing performance.
problem Efficiently applying TS to complex, multi-modal problems.
method Sparse Gaussian Process models for scalable TS.
result Theoretical and empirical validation of scalable TS's effectiveness.
A scalable parallel BO method for asynchronous settings.
problem Expensive-to-evaluate problems in machine learning.
method Simple and scalable Bayesian optimization method for asynchronous parallel settings.
result Demonstrated promising performance on benchmark functions and hyperparameter optimization.
Improving scalability and stability of Stein discrepancies for scalable goodness-of-fit testing
problem Improving scalability and stability of Stein discrepancies for scalable goodness-of-fit testing
method Reformulating Stein discrepancy construction as an explicit SNR^2 maximisation problem
result Avoiding exponential SNR^2 collapse and achieving stable SNR^2
Gaussian process classification (GPC) provides a flexible and powerful statistical framework describing joint distributions over function space. Conventional GPCs however suffer from (i) poor scalability for big data due to the full kernel matrix, and (ii) intractable inference due to the non-Gaussian likelihoods. Henc…
New scalable GP approximation using Fourier series decomposition.
problem Scalability and accuracy in Gaussian process approximations.
method Harmonic kernel decomposition (HKD) to decompose kernels orthogonally.
result Significantly outperforms standard variational methods in scalability and accuracy.
Graphs are ubiquitous real-world data structures, and generative models that approximate distributions over graphs and derive new samples from them have significant importance. Among the known challenges in graph generation tasks, scalability handling of large graphs and datasets is one of the most important for practi…
The marriage of wireless big data and machine learning techniques revolutionizes the wireless system by the data-driven philosophy. However, the ever exploding data volume and model complexity will limit centralized solutions to learn and respond within a reasonable time. Therefore, scalability becomes a critical issue…
A scalable algorithm for GP regression selects relevant covariates efficiently.
problem Scalable variable selection in large GP regression models.
method VGPR algorithm using Vecchia approximation for sparse precision matrix, mini-batch subsampling.
result Improved scalability and accuracy in selecting relevant covariates.
Developing stable and scalable probabilistic ODE solvers for stiff and high-dimensional problems.
problem Stiff and high-dimensional ODEs
method Matrix-free update step and iterative re-linearization
result Improved stability and scalability
The vast quantity of information brought by big data as well as the evolving computer hardware encourages success stories in the machine learning community. In the meanwhile, it poses challenges for the Gaussian process (GP) regression, a well-known non-parametric and interpretable Bayesian model, which suffers from cu…
Scalable model for slate recommendation learns reward probabilities.
problem Scalable personalized slate recommendation in large action spaces.
method Probabilistic Rank and Reward (PRR) model combining reward, interaction, and rank.
result PRR outperforms existing methods and is scalable to large action spaces.
TensorHyper-VQC improves VQC scalability and robustness.
problem Scalability and noise sensitivity in VQC.
method Tensor-train-guided hypernetwork framework.
result TensorHyper-VQC achieves superior performance and robust noise tolerance.
Paper combines scalable BMF algorithms for web-scale datasets.
problem High computational cost of Bayesian Matrix Factorization.
method Combines Posterior Propagation and asynchronous distributed implementation.
result Substantial improvements in scalability on web-scale datasets.
Machine learning techniques have been widely applied in Internet companies for various tasks, acting as an essential driving force, and feature engineering has been generally recognized as a crucial tache when constructing machine learning systems. Recently, a growing effort has been made to the development of automati…
We introduce a new structured kernel interpolation (SKI) framework, which generalises and unifies inducing point methods for scalable Gaussian processes (GPs). SKI methods produce kernel approximations for fast computations through kernel interpolation. The SKI framework clarifies how the quality of an inducing point a…
S3VDC improves DC methods for scalability, stability, and simplicity.
problem Poor scalability, instability, and lack of simplicity in DC methods.
method Four algorithmic improvements: initial γ-training, periodic β-annealing, mini-batch GMM initialization, and inverse min-max transform. S3VDC incorporates all improvements. result S3VDC outperforms state-of-the-art methods on benchmark and industrial datasets.
A scalable MARL algorithm using local rewards for cooperative multi-agent learning.
problem Scalability issues in cooperative multi-agent reinforcement learning due to large state and action spaces.
method LOMAQ algorithm incorporating local rewards in centralized training and decentralized execution.
result LOMAQ scales well compared to other methods, improving performance and convergence speed.
Scalable PnP-ADMM for large-scale imaging problems.
problem Heavy computational and memory requirements of current PnP algorithms.
method Incremental variant of PnP-ADMM with theoretical convergence guarantees.
result Fast convergence and scalability compared to existing PnP algorithms.
New method for scalable barycenter computation using Wasserstein gradient flows.
problem Scalability and integration of label information in barycenter computation.
method Gradient flows in Wasserstein space, time discretization, mini-batch optimal transport, modular regularization, task-aware functions, supervised information integration.
result Empirically validated new state-of-the-art barycenter solver with labeled barycenters outperforming unlabeled ones.
Paper proposes scalable algorithm to estimate intervention targets in linear models.
problem Estimating intervention targets in linear models from observational and interventional data.
method The paper proposes a scalable algorithm that estimates intervention sites from the difference between precision matrices of observational and interventional datasets.
result The algorithm consistently identifies all intervention targets and updates observational Markov equivalence classes to interventional ones.
In this paper, we present a general framework to scale graph autoencoders (AE) and graph variational autoencoders (VAE). This framework leverages graph degeneracy concepts to train models only from a dense subset of nodes instead of using the entire graph. Together with a simple yet effective propagation mechanism, our…
The book covers scalable MCMC methods for Bayesian learning.
problem Scalability issues in Bayesian learning with large datasets.
method Advanced MCMC algorithms, including stochastic gradient, non-reversible, and continuous time methods.
result Substantial advances in practical and theoretical Bayesian computation.
We propose a new scalable algorithm for holistic linear regression building on Bertsimas & King (2016). Specifically, we develop new theory to model significance and multicollinearity as lazy constraints rather than checking the conditions iteratively. The resulting algorithm scales with the number of samples n in th…
This study improves scalability of randomized smoothing for certifying classifier robustness.
problem Certifying machine learning classifiers against adversarial attacks is challenging and scalable solutions are needed.
method The study reviews and explores randomized smoothing and its derivatives, focusing on scalability.
result The study provides theoretical guarantees and discusses scalability challenges of randomized smoothing.
This paper surveys scalable automated alignment methods for LLMs.
problem Scalability issues in traditional human-annotated alignment methods for LLMs.
method Categorizes and discusses various automated alignment methods.
result Emerging automated alignment methods are effective and scalable.
Improved GP models for fast training and good performance.
problem Training scalable Gaussian process models efficiently.
method Cross-validation and nearest neighbor truncation for scalable GP training.
result Our method offers fast training and excellent predictive performance.
SOL is an open-source library for scalable online learning algorithms, and is particularly suitable for learning with high-dimensional data. The library provides a family of regular and sparse online learning algorithms for large-scale binary and multi-class classification tasks with high efficiency, scalability, porta…
Study improves scalability of cell-free massive MIMO networks by optimizing UE-AP association.
problem Optimizing UE-AP association in cell-free massive MIMO networks.
method Deep learning algorithm using Bidirectional Long Short-Term Memory cells and hybrid probabilistic weight updating.
result Enhanced scalability without retraining, robust against pilot contamination.
Sep-SpectralNet improves SE for broader applicability and scalability.
problem Three main drawbacks of current SE implementations: generalizability, scalability, and eigenvectors separation.
method Sep-SpectralNet extends SpectralNet with an eigenvector separation post-processing step.
result Sep-SpectralNet achieves consistent SE approximation and generalization, enhancing scalability and applicability.
DKL-KAN combines deep learning and kernel methods for scalable, expressive models.
problem Combining deep learning's depth with kernel methods' flexibility for scalable models.
method DKL-KAN uses Kolmogorov-Arnold Networks (KAN) to optimize kernel attributes within a Gaussian process framework.
result DKL-KAN outperforms DKL-MLP on datasets with a low number of observations and DKL-MLP on large datasets.
SAFLe solves federated learning's trade-off between non-linearity and scalability.
problem Federated Learning's high communication overhead and performance collapse on non-IID data.
method SAFLe introduces a structured head of bucketed features and sparse, grouped embeddings, mathematically equivalent to a high-dimensional linear regression.
result SAFLe achieves a new state-of-the-art in analytic FL, outperforming linear AFL and multi-round DeepAFL.
Scalable method learns context-specific models for hundreds of variables.
problem Learning context-specific models for large numbers of variables.
method Order-based Markov chain Monte-Carlo search with context-specific sparsity assumption.
result Method scales to hundreds of variables and learns accurate models.
Meta-learning interpretable decision trees with synthetic data.
problem Lack of efficient, scalable methods for generating synthetic data for decision tree meta-learning.
method Synthetic generation of near-optimal decision trees using the MetaTree transformer architecture.
result Meta-learning of decision trees achieves performance comparable to real-world data or optimal decision trees, with significant computational cost reduction.
Develops scalable differentiable physics for complex object interactions.
problem Limited scalability of existing differentiable physics solvers.
method Adopting meshes for arbitrary geometry, localized collision handling, and accelerated implicit differentiation.
result Significantly reduces memory and computation requirements compared to particle-based methods.
Bayesian learning made scalable with posteriors library.
problem Computational challenges in Bayesian learning with modern models.
method Introducing posteriors library and tempered MCMC.
result Bayesian approximations are useful and scalable.
Unified framework for scalable black-box optimization.
problem Expensive black-box evaluations in scientific and engineering domains.
method Integrates active learning, multi-armed bandits, and distributed computing.
result Consistently outperforms state-of-the-art black-box optimizers.
L2G2G improves graph autoencoder accuracy without sacrificing scalability.
problem Scalability issues in graph autoencoders for large networks.
method Dynamic synchronization of latent node representations in each epoch.
result L2G2G achieves higher accuracy than standard Local2Global approach on large and dense networks.
Scalable method completes ill-conditioned matrices from few samples.
problem Matrix completion from few samples for ill-conditioned matrices.
method Iterative algorithm combining IRLS, smoothing Newton, and proximal gradient methods.
result Local quadratic convergence rate and well-conditioned linear systems.
BSA-TNP improves NP scalability and accuracy for spatiotemporal data.
problem Scalability and accuracy trade-off in Neural Processes.
method Introduces KRBlocks, group-invariant attention biases, and BSA for scalable spatiotemporal inference.
result BSA-TNP matches or exceeds accuracy of best models while training faster.
Novel method for scalable neural network-based blackbox optimization.
problem Scalability challenges in high-dimensional Bayesian Optimization.
method SNBO: Adds new samples using separate criteria for exploration and exploitation, adaptively controlling the sampling region.
result SNBO achieves better function values with 40-60% fewer function evaluations and reduced runtime.